A skeleton display method, device, storage medium and electronic equipment

Abstract

The application discloses a skeleton display method and device, a storage medium and an electronic device, and relates to the technical field of data collection. The method comprises the following steps: determining hand parameter information of a user performing a target hand action based on electromyographic signal data of the user; and displaying a target hand skeleton of the user based on the hand parameter information, wherein the target hand skeleton is used to represent the hand state change of the user during the execution of the target hand action. Compared with the related art, the application can determine the hand parameter information of the user when the user performs the target hand action, so that the target hand skeleton of the user can be displayed according to the hand parameter information of the user, so that the application can recognize the hand parameter by using the electromyographic signal and display the skeleton, and does not need to rely on the hand image to display the hand action, and thus the hand action at the blocked position can be accurately displayed when the hand image is blocked by the object gripped by the hand.

Description

Technical Field

[0001] This application relates to the field of data acquisition technology, and in particular to a skeleton display method, device, storage medium and electronic device. Background Technology

[0002] In related technologies, user hand movements are mainly identified by collecting images of the user's hand. However, this method can lead to situations where objects being grasped by the hand obscure the hand image, making it impossible to accurately display the hand movements at the obscured locations. Summary of the Invention

[0003] In view of this, this application provides a skeleton display method, device, storage medium and electronic device. This application uses electromyography signals to identify hand parameters and display the skeleton, without relying on hand images to display hand movements. Therefore, even when the hand image is obscured by an object being grasped by the hand, the hand movements at the obscured position can still be accurately displayed.

[0004] Firstly, this application provides a method for displaying a skeleton, including:

[0005] Based on the user's electromyographic signal data, determine the hand parameter information of the user performing the target hand movement; Based on the hand parameter information, the target hand skeleton of the user is displayed, which is used to characterize the changes in the hand state of the user during the execution of the target hand action.

[0006] Optionally, the target hand skeleton includes the joints of the user's hand and connecting lines connecting the joints. During the user's execution of the target hand action, the positions of the joints and the connecting lines change with the user's hand action.

[0007] Optionally, displaying the user's hand skeleton includes: Obtain the execution screen of the user performing the target hand action; The target hand skeleton of the user is displayed at the user's hand position in the execution screen.

[0008] Optionally, displaying the target hand skeleton of the user at the user's hand position in the execution screen includes: When the user's hand is obscured by a grasping object, the target hand skeleton of the user is displayed at the unobscured position of the user's hand and at the position where the grasping object obscures the user's hand, based on the user's grasping information of the grasping object.

[0009] Optionally, based on the user's grasping information of the grasping object, displaying the target hand skeleton of the user at the unobstructed position of the user's hand and at the position where the grasping object obstructs the user's hand includes: Based on the contact distribution information between the user and the object being grasped and the user's finger gripping force information on the object being grasped, the occlusion position in the execution screen is determined; The user's target hand skeleton is displayed in both the unobstructed and obstructed positions.

[0010] The method further includes: Based on the execution screen, determine whether the hand position is obscured by the grasping object.

[0011] Optionally, the method further includes: The execution screen of the user performing the target hand action is obtained through a head-mounted device, which is a device with image capture function.

[0012] Optionally, the method further includes: Obtain target distance information between the head-mounted device and the user's hand; The target distance information is displayed at the distance display position corresponding to the target hand skeleton.

[0013] Optionally, displaying the user's target hand skeleton based on the hand parameter information includes: Based on the user's hand size information, the user's basic hand skeleton is displayed; Based on the user's hand state information, the state of the basic hand skeleton is adjusted to generate the target hand skeleton.

[0014] Optionally, the step of adjusting the state of the basic hand skeleton based on the user's hand state information to generate the target hand skeleton includes: The degree of finger opening and closing of the user is determined based on the joint angle of the user's hand; Based on the degree of finger opening and closing, the basic hand skeleton is adjusted to adjust the finger opening and closing state to generate the target hand skeleton.

[0015] Optionally, adjusting the finger opening / closing state of the basic hand skeleton based on the degree of finger opening / closing to generate the target hand skeleton includes: Based on the degree of finger opening and closing, determine the joint positions of the finger joints in the basic hand skeleton and the connection angle of the finger connecting lines connecting the finger joints; Based on the joint position and the connection angle, the finger opening and closing state of the basic hand skeleton is adjusted to generate the target hand skeleton.

[0016] Optionally, displaying the user's basic hand skeleton based on the user's hand size information includes: Based on the user's palm size and finger length, determine the user's skeletal adjustment parameters; Based on the skeleton adjustment parameters, the size of the preset hand skeleton is adjusted to generate the user's basic hand skeleton.

[0017] Optionally, adjusting the size of the preset hand skeleton based on the skeleton adjustment parameters to generate the user's basic hand skeleton includes: Based on the skeleton adjustment parameters, the initial joint point position of the user's hand and the length of the initial connecting line connecting the initial joint point are determined. Based on the initial joint position and the length of the connecting line, the size of the preset hand skeleton is adjusted to generate the user's basic hand skeleton.

[0018] Optionally, determining the hand parameter information for the user to perform the target hand movement based on the user's electromyographic signal data includes: Based on the electromyographic signal data, the coordinates of the key joints of the user's hand are determined; Based on the coordinates of the key joint points, the hand parameter information for the user performing the target hand action is determined.

[0019] Optionally, the method further includes: The user's electromyography (EMG) signal data is obtained through a wearable wristband device, which is a device with EMG signal acquisition function.

[0020] Secondly, this application provides a skeleton display method, including: The determination module is configured to determine the hand parameter information of the user performing the target hand action based on the user's electromyographic signal data; The display module is configured to display the target hand skeleton of the user based on the hand parameter information. The target hand skeleton is used to characterize the changes in the hand state of the user during the execution of the target hand action.

[0021] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the skeleton display method described in the first aspect.

[0022] Fourthly, this application provides an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement the skeleton display method described in the first aspect.

[0023] Fifthly, this application provides a program product including computer instructions for causing a computer to execute the skeleton display method described in the first aspect.

[0024] By utilizing the above technical solutions, this application provides a skeleton display method, device, storage medium, and electronic device. Compared with related technologies, this application, by using the user's electromyographic signal data, can determine the user's hand parameter information when the user performs a target hand movement. Based on the user's hand parameter information, the target hand skeleton of the user can be displayed. The target hand skeleton can characterize the changes in the hand state during the user's performance of the target hand movement. Thus, this application uses electromyographic signals to identify hand parameters and display the skeleton without relying on hand images to display hand movements. Therefore, even when the hand image is obscured by an object being grasped, the hand movement at the obscured position can still be accurately displayed.

[0025] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating a skeleton display method provided in an embodiment of this application is shown; Figure 2 A flowchart illustrating a skeleton display method provided in an embodiment of this application is shown; Figure 3 This illustration shows one of the schematic diagrams of a skeleton display example provided in an embodiment of this application; Figure 4This is a second schematic diagram illustrating an example of a skeleton display provided in an embodiment of this application; Figure 5 This paper shows a schematic diagram of the structure of a skeleton display device provided in an embodiment of this application; Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0029] Embodiments of the present application will now be described in more detail with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in the present application can be combined with each other.

[0030] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, they do not violate public order and good morals, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0031] To address the issue that current technologies primarily rely on capturing images of the user's hand to recognize hand movements, which can lead to problems such as the hand being grasped or its movements being obscured, this embodiment provides a skeleton display method. Figure 1 As shown, the method includes: Step 101: Based on the user's electromyographic signal data, determine the hand parameter information for the user to perform the target hand movement.

[0032] In the embodiments of this application, the electromyographic signal data can be generated by superimposing the motor unit movements when the user's skeletal muscles contract. For example, when the user's arm skeletal muscles contract, an electromyographic signal corresponding to the arm can be generated; when the user's leg skeletal muscles contract, an electromyographic signal corresponding to the leg can be generated, and so on. No further examples will be given here.

[0033] In this embodiment, the user's electromyographic signal data may include, but is not limited to, the electromyographic signals of the user's arm, the user's wrist, the user's fingers, etc., and is not specifically limited here. For example, by collecting the user's electromyographic signal data, this application embodiment can determine the hand parameter information of the user during the execution of the target hand movement.

[0034] For example, the target hand movement can be a hand movement that requires electromyography signal acquisition. For instance, the target hand movement can include, but is not limited to, grasping, picking up, finger pinching, hand lifting, palm dragging, etc., without specific limitations.

[0035] As an alternative approach, if the target hand action is picking up a cup, the user's electromyography (EMG) signal data can be collected to determine the hand parameters during the cup-picking process; if the target hand action is folding a blanket, the user's EMG signal data can be collected to determine the hand parameters during the blanket-folding process.

[0036] In some examples, hand parameter information can be the parameter information of the user's hand during the execution of the target hand action; for example, hand parameter information includes, but is not limited to, palm size information, finger length information, joint position information, joint angle information, contact heat distribution information with the contacting object, contact force information with the contacting object, etc., without specific limitations.

[0037] As an alternative approach, when the target hand action is picking up a cup, electromyography (EMG) data can be collected from the user to determine the joint position, joint angle, heat distribution, and contact force during the cup-picking process. Similarly, when the target hand action is folding a blanket, EMG data can be collected from the user to determine the joint position, joint angle, heat distribution, and contact force during the blanket-folding process.

[0038] Step 102: Based on the hand parameter information, display the user's target hand skeleton.

[0039] The target hand skeleton is used to characterize the changes in hand state during the user's execution of the target hand action.

[0040] It should be noted that the target hand skeleton in this application embodiment can be a skeleton that represents the change in hand state during the user's execution of the target hand action; for example, if the hand state changes during the user's execution of the target hand action, the target hand skeleton will also change accordingly.

[0041] In this application embodiment, the target hand skeleton can be the hand skeleton of the user during the execution of the target hand action; for example, when the target hand action is the action of picking up a cup, the user's electromyographic signal data can be collected, and the joint position information, joint angle information, contact heat distribution information with the cup, contact force information with the cup, etc. of the user during the picking up of the cup can be determined based on the user's electromyographic signal data. Then, based on the determined hand parameter information, the target hand skeleton of the user during the picking up of the cup can be displayed, so that this application can clearly display the changes in the user's hand state during the picking up of the cup through the target hand skeleton.

[0042] As an alternative approach, when the target hand action is grasping a toy ball, electromyography (EMG) signal data of the user can be collected. Based on the user's EMG signal data, information such as joint position, joint angle, heat distribution, and contact force during the grasping process can be determined. Then, based on the determined hand parameter information, the target hand skeleton during the grasping process can be displayed, allowing this application to clearly show the changes in the user's hand state during the grasping process through the target hand skeleton.

[0043] Optionally, the target hand skeleton includes the joints of the user's hand and the connecting lines connecting the joints. During the user's execution of the target hand movement, the positions of the joints and connecting lines change with the user's hand movement.

[0044] It should be noted that, in the embodiments of this application, during the process of the user's hand performing the target hand action, the positions of the joints and connecting lines of the hand will change with the hand action; for example, when the user performs a picking action, the hand will first change from an extended state to a state of pinching or holding an object, and then change from a state of pinching or holding an object back to an extended state. During the process of the hand changing from an extended state to a state of pinching or holding an object, the joints of the fingers will move towards the palm based on the extended position, thereby realizing the action of the fingers clenching into a fist towards the palm, thus pinching or holding the object. During this process, the position of the joints of the fingers will move towards the palm as the fingers move. When the joints move, the position of the connecting points connecting the joints will also move with the joints, that is, the position of the connecting lines will also move towards the palm, thereby realizing that the positions of the joints and connecting lines change with the user's hand action.

[0045] In this embodiment of the application, the joints of the hand may include the joints of the palm and the joints of the fingers; the joints of the palm can be used to show the changes of the palm during the execution of the target hand action; the joints of the fingers can be used to show the changes of the fingers during the execution of the target hand action.

[0046] In this embodiment, the connecting lines connecting the joints may include connecting lines connecting the palm joints, connecting lines connecting the palm and fingers joints, and connecting lines connecting the finger joints. The connecting lines connecting the palm joints can represent the change in the palm's opening and closing angle during the execution of the target hand movement; the connecting lines connecting the palm and fingers joints can represent the change in the opening and closing angle at the connection between the palm and fingers during the execution of the target hand movement; and the connecting lines connecting the finger joints can represent the change in the finger's opening and closing angle during the execution of the target hand movement. No specific limitations are imposed here.

[0047] As an alternative approach, when the target hand action is to pick up a cup, the positions of the joints and connecting lines of the user's hand will change accordingly if the user performs the action of picking up a cup; when the target hand action is to fold clothes, the positions of the joints and connecting lines of the user's hand will change accordingly if the user performs the action of folding clothes, thereby enabling the embodiments of this application to clearly demonstrate the changes in the hand during the execution of the target hand action.

[0048] Compared with related technologies, this embodiment uses the user's electromyography (EMG) signal data to determine the user's hand parameter information when the user performs a target hand movement. Based on the user's hand parameter information, the target hand skeleton can be displayed. The target hand skeleton can characterize the changes in the hand state during the user's performance of the target hand movement. Therefore, this embodiment uses EMG signals to identify hand parameters and display the skeleton without relying on hand images to display hand movements. Thus, even when the hand is grasped by an object that obscures the hand image, the hand movement at the obscured position can still be accurately displayed.

[0049] Optionally, when performing the "display the user's hand skeleton" action, the following methods may be used, but are not limited to them: Figure 2 As shown, the method includes: Step 201: Obtain the execution screen of the user performing the target hand action.

[0050] In this embodiment of the application, the execution screen of the user performing a hand action can be the execution screen captured by a camera device; for example, if the target hand action is to pick up a cup, the execution screen of the user performing the action of picking up a cup can be captured by a camera device; if the target hand action is to open a cabinet, the execution screen of the user performing the action of opening a cabinet can be captured by a camera device, and no specific limitation is made here.

[0051] It should be noted that, by obtaining the execution screen of the user performing the target hand action, the embodiments of this application can clearly display the screen of the user performing the target hand action, thereby enabling the user's target hand skeleton to be further displayed on the screen of the user performing the target hand action.

[0052] Optionally, this embodiment may also employ the following methods, but not limited to these: obtaining the execution screen of the user performing the target hand action through a head-mounted device, wherein the head-mounted device is a device with screen capture function.

[0053] In this application embodiment, the head-mounted device can be a device that can be worn on the user's head. It should be noted that the head-mounted device in this application embodiment can be a head-mounted device with image capture function, wherein the image capture function can include, but is not limited to, image capture function, video capture function, animation capture function, etc., and is not specifically limited here.

[0054] In some examples, the head-mounted device may include, but is not limited to, helmets, headbands, hats, etc. with image capture capabilities. The specific form of the head-mounted device is not limited in the embodiments of this application.

[0055] For example, if the target hand action is to pick up a cup, the execution screen of the user performing the action of picking up a cup can be captured by the head-mounted device; if the target hand action is to open a cabinet, the execution screen of the user performing the action of opening a cabinet can be captured by the head-mounted device, without any specific limitation here.

[0056] Optionally, this embodiment may also employ the following methods, but not limited to these: obtaining target distance information between the head-mounted device and the user's hand; displaying the target distance information at the distance display position corresponding to the target hand skeleton.

[0057] In this embodiment of the application, the distance information between the head-mounted device and the user's hand can also be obtained during the user's execution of the target hand action, and the determined distance information can be displayed at the distance display position corresponding to the target hand skeleton. The distance information can more clearly show the actual acquisition angle of the execution screen of the user's execution of the target hand action, thereby more clearly showing the user the execution screen of the execution of the target hand action.

[0058] For example, if the target hand action is to pick up a cup, the head-mounted device can capture the execution of the user's action of picking up the cup. It can also determine the distance between the head-mounted device and the user's hand during the process of picking up the cup, thereby more clearly showing the forward and backward movement of the user's hand during the process of picking up the cup.

[0059] For example, if the target hand action is folding clothes, the head-mounted device can capture the execution of the user's folding action. It can also determine the distance between the head-mounted device and the user's hand during the folding action, thus more clearly showing the forward and backward movement of the user's hand and the changes in the layers of the clothes.

[0060] Step 202: Display the user's hand skeleton at the user's hand position on the execution screen.

[0061] In this embodiment of the application, in the execution screen where the user performs the target hand action, the target hand skeleton of the user can be displayed at the position of the user's hand; for example, the target hand skeleton of the user can be displayed on the upper layer of the user's hand; or the target hand skeleton can be matched with the user's actual hand skeleton, and the target hand skeleton can be displayed at the position of the user's actual skeleton, so as to clearly show the changes in the skeleton during the user's performance of the target hand action.

[0062] In some examples, if the target hand action is to pick up a cup, the target hand skeleton can be mapped to the user's actual hand skeleton, and the target hand skeleton can be displayed at the user's actual skeleton position during the process of picking up the cup, thus clearly showing the changes in the skeleton during the user's action of picking up the cup.

[0063] Optionally, when performing the action "displaying the user's hand skeleton at the user's hand position in the execution screen", the following methods can be used, but are not limited to these: when the user's hand position is obscured by a grasping object, based on the user's grasping information of the grasping object, displaying the user's hand skeleton at the unobscured position of the user's hand and at the position where the grasping object obscures the user's hand.

[0064] In this embodiment of the application, when the user's hand is obscured by a grasping object, the position of the hand obscured by the grasping object can be determined based on the user's grasping information of the grasping object. Thus, the user's hand skeleton can be displayed at the unobscured position of the user's hand and at the position where the grasping object obscures the user's hand.

[0065] For example, if the target hand action is to hold a cup, and if the fingers are obscured by the cup while the user is gripping the cup, the user's hand skeleton can be displayed at the unobscured position of the user's hand and at the position where the cup obscures the user's hand, based on the user's gripping information. This allows the target finger skeleton to be clearly displayed even in the area obscured by the cup in the execution screen, thus clearly showing the changes in the finger skeleton during the cup gripping process to the user.

[0066] For example, such as Figure 3 As shown, if the target hand action is to pick up a document, and if the user's fingers are obscured by the document while picking it up, the system can display the user's hand skeleton at the locations where the user's hand is not obscured by the document and at the locations where the document obscures the user's hand, based on the user's document-picking information. For example... Figure 3 In the diagram, the area outlined by the dashed line represents the position where the document obscures the user's hand, while the area not outlined by the dashed line represents the position where the user's hand is not obscured by the document. In this embodiment, when the document obscures the hand, the user's hand skeleton can be displayed in both the obscured position and the unobscured position. This allows the user's hand skeleton to be clearly and completely displayed in the execution screen even when the hand is obscured, thus clearly showing the changes in the finger skeleton during the process of holding the document.

[0067] For example, such as Figure 4 The image shown is an example of a user's hand performing a target hand action that is obscured. Figure 4 In this diagram, circular circles represent joints of the hand, connecting lines between these circles represent connecting lines between the joints, and numbers on the circles indicate the joint numbers. The area enclosed by a dashed line represents the position where the user's hand is obscured by a grasping object, while the area not enclosed by a dashed line represents the position where the user's hand is not obscured. The grasping object can be any object. In this embodiment, even when the user's hand is obscured by a grasping object, the user's hand skeleton can be displayed in both the obscured and unobscured positions. This allows for a clear and complete display of the user's hand skeleton in the execution screen, even when the hand is obscured, clearly showing the changes in the finger skeleton during the execution of the target hand movement.

[0068] Optionally, when performing the action of "displaying the user's hand skeleton in the unobstructed position of the user's hand and the occluded position of the user's hand based on the user's gripping information of the object being grasped", the following methods can be used, but are not limited to these: determining the occlusion position in the execution screen based on the contact distribution information between the user and the object being grasped and the user's finger gripping force information of the object being grasped; and displaying the user's target hand skeleton in the unobstructed position and the occluded position.

[0069] In this embodiment of the application, the contact distribution information between the user and the object being grasped can be the distribution information of the contact between the user's hand and the object during the grasping process; for example, it can be represented by a distribution heat map; the finger gripping force information of the user on the object can be the force between the user's fingers and the object during the grasping process, which is not specifically limited here.

[0070] For example, by using the contact distribution information between the user and the object being grasped and the user's finger gripping force information on the object, the contact position between the user's hand and the object being grasped in the execution screen can be accurately determined, thereby determining the occlusion position in the execution screen; then, at the hand position and the occlusion position, the user's target hand skeleton can be displayed, so that even when the grasping object occludes the user's hand, the user's hand skeleton can still be clearly displayed, thereby clearly showing the changes in the hand joints during the user's grasping process.

[0071] As an alternative approach, if the target hand action is to hold a cup, and the user's fingers are obscured by the cup during the gripping process, the contact distribution information during the gripping process can be identified to determine the contact between the user's hand and the cup. The gripping force information can also be identified to determine the force exerted by the user's fingers on the cup. Based on the contact distribution and gripping force information, the contact position between the user's hand and the cup can be identified, thus determining the obscured position in the execution screen. Furthermore, the user's target hand skeleton can be displayed at both the unobscured and obscured positions, ensuring that even when the cup obscures the user's hand, the hand skeleton can still be clearly displayed, thus clearly showing the changes in the hand joints during the gripping process.

[0072] Optionally, this embodiment may also employ the following methods, but not limited to these, including: determining whether the hand position is obscured by the grasping object based on the execution screen.

[0073] In this embodiment of the application, it can be determined whether the hand position is obscured by a grasping object based on the execution screen; for example, the execution screen can show the process of the user performing the target hand action. If the complete hand image of the user cannot be identified in the execution screen, it can be determined that the hand position is obscured by a grasping object.

[0074] As an alternative approach, if the target hand action is to pick up a cup, and the user's complete hand cannot be recognized in the execution screen of the user picking up the cup, then it can be determined that the hand is obscured by the cup; if the target hand action is to grab a toy ball, and the user's complete hand cannot be recognized in the execution screen of the user grabbing the toy ball, then it can be determined that the hand is obscured by the toy ball, and so on. Examples will not be given here.

[0075] Optionally, when performing "displaying the user's target hand skeleton based on hand parameter information", the following methods can be used, but are not limited to: displaying the user's basic hand skeleton based on the user's hand size information; adjusting the state of the basic hand skeleton based on the user's hand state information to generate the target hand skeleton.

[0076] In some examples, the basic hand skeleton can be a basic hand skeleton constructed based on the user's hand size; for example, if the basic hand skeleton is in the state of fully extended fingers, then the basic hand skeleton of the target user can be a hand skeleton with fully extended fingers that is the same size as the user's hand; if the basic hand skeleton is in the state of fully clenched fists, then the basic hand skeleton of the target user can be a hand skeleton with fully clenched fists that is the same size as the user's hand, without any specific limitation.

[0077] In this embodiment, based on the user's hand state information, the hand state can be adjusted on the basis of the basic skeleton. For example, if the basic skeleton is a hand skeleton with the same size as the user's hand and the fingers fully extended, then according to the hand state, such as the state of grasping an object, the target hand skeleton can be adjusted from the hand skeleton with the fingers fully extended to the state of grasping an object. Examples will not be listed here.

[0078] Optionally, when performing the action of "adjusting the state of the basic hand skeleton based on the user's hand state information to generate the target hand skeleton", the following methods can be used, but are not limited to these: determining the user's finger opening and closing degree based on the joint angle of the user's hand; adjusting the finger opening and closing state of the basic hand skeleton according to the finger opening and closing degree to generate the target hand skeleton.

[0079] In this embodiment of the application, the joint angle can be the relative rotation angle between bones; for example, the user's finger joint angle can be the relative rotation angle between the bones of the finger; the joint angle between the user's palm and fingers can be the relative rotation angle between the bones of the finger and the palm.

[0080] In this embodiment, the degree of finger opening and closing can be determined based on the joint angle. For example, with the finger fully extended as the reference, when the finger is bent or clenched into a fist, the joint angle value increases, which can indicate that the degree of finger opening and closing decreases, that is, the finger is closed; when the finger is fully open and extended, the joint angle value decreases, which can indicate that the degree of finger opening and closing increases, that is, the finger is open. No specific limitation is made here.

[0081] For example, the opening and closing state of the fingers can be adjusted based on the degree of finger opening and closing to generate a target hand skeleton. For instance, if the basic hand skeleton is a hand skeleton with fingers fully extended that is the same size as the user's hand, the fully extended hand skeleton can be adjusted to a curved hand skeleton that conforms to the degree of finger opening and closing to obtain the target hand skeleton. No specific limitation is made here.

[0082] Optionally, when performing the action of "adjusting the finger opening and closing state of the basic hand skeleton according to the degree of finger opening and closing to generate the target hand skeleton", the following methods can be used, but are not limited to these: determining the joint position of the finger joints in the basic hand skeleton and the connection angle of the finger connecting lines connecting the finger joints according to the degree of finger opening and closing; adjusting the finger opening and closing state of the basic hand skeleton based on the joint position and connection angle to generate the target hand skeleton.

[0083] In this embodiment, the position of the finger joints and the connection angle of the connecting lines connecting the finger joints are different under different finger opening and closing degrees. Therefore, the joint position of the finger joints and the connection angle of the connecting lines connecting the finger joints in the basic hand skeleton can be determined according to the finger opening and closing degree. Based on the joint position and connection angle, the finger opening and closing state of the basic hand skeleton can be adjusted to generate the target hand skeleton.

[0084] For example, when a user grips a cup tightly, the degree of finger opening and closing is relatively small; when the user extends their fingers, the degree of finger opening and closing is relatively large. That is, the degree of finger opening and closing can change from large to small as the user moves from extending their fingers to gripping the cup. Based on this change in the degree of opening and closing, the positional changes of the finger joints and the changes in the connection angles of the connecting lines to the finger joints can be determined. Thus, based on the joint positions and connection angles, the finger opening and closing state of the basic hand skeleton can be adjusted to generate the target hand skeleton when the user moves from extending their fingers to gripping the cup. Further examples will not be provided here.

[0085] Optionally, when performing the "displaying the user's basic hand skeleton based on the user's hand size information", the following methods can be used, but are not limited to: determining the user's skeleton adjustment parameters based on the user's palm size and finger length; adjusting the skeleton size of the preset hand skeleton based on the skeleton adjustment parameters to generate the user's basic hand skeleton.

[0086] In this embodiment, the palm size can be the dimensions of the palm, such as, but not limited to, the width, length, and outline of the palm; the finger length can include the total length of each finger, the length of each finger joint, etc.; the user's skeletal adjustment parameters can be determined based on the user's palm size and finger length; and the size of the preset hand skeleton can be adjusted according to the skeletal adjustment parameters so that the preset hand skeleton size can be adjusted to a basic hand skeleton that is the same as the user's hand size.

[0087] For example, if the user has a large hand, the user's skeletal adjustment parameters can be determined by the size of the user's palm and the length of the fingers. Based on the skeletal adjustment parameters, the size of the preset hand skeleton can be adjusted so that the preset hand skeleton can be enlarged based on the preset hand skeleton size, thereby adjusting the preset hand skeleton to a basic hand skeleton with the same size as the user's hand.

[0088] Optionally, when performing the action of "adjusting the size of the preset hand skeleton based on the skeleton adjustment parameters to generate the user's basic hand skeleton", the following methods can be used, but are not limited to these: determining the initial joint point positions of the user's hand and the length of the initial connecting lines connecting the initial joint points based on the skeleton adjustment parameters; adjusting the size of the preset hand skeleton based on the initial joint point positions and the length of the connecting lines to generate the user's basic hand skeleton.

[0089] In the embodiments of this application, the joint positions and the lengths of the connecting lines between the joints are different for hands of different sizes. For example, if the user's hand is small, the joint positions can be more compact and the lengths of the connecting lines between the joints can be shorter. If the user's palm is small, the joint positions can be more compact and the distances between the connecting lines between different fingers can be shorter. If the user's hand is large, the joint positions can be more spacious and the lengths of the connecting lines between the joints can be longer. Examples will not be listed here.

[0090] As an alternative approach, since different users have different hand sizes, it is necessary to make adaptive adjustments based on the preset hand skeleton. Specifically, the initial joint point positions of the user's hand and the length of the initial connecting lines connecting the initial joint points can be adjusted based on the preset hand skeleton. Thus, the size of the preset hand skeleton can be adjusted according to the initial joint point positions and the length of the connecting lines to generate the user's basic hand skeleton.

[0091] Optionally, when performing the "determining hand parameter information of the user performing the target hand movement based on the user's electromyographic signal data", the following methods can be used, but are not limited to: determining the coordinates of the key joints of the user's hand based on the electromyographic signal data; and determining the hand parameter information of the user performing the target hand movement based on the coordinates of the key joints.

[0092] In this embodiment of the application, based on electromyographic signal data, the coordinates of key joints of the user's hand can be determined. The coordinates of key joints can be three-dimensional key point coordinates, thereby determining the hand parameter information of the user performing the target hand action through the key point coordinates.

[0093] For example, by using the coordinates of key joints in the hand, the precise position of the hand joints can be determined during the user's execution of the target hand action. Based on the precise position of the joints, the user's hand size information and hand state information can be determined, without any specific limitations.

[0094] As an alternative approach, during the user's action of picking up a cup, electromyographic (EMG) signal data of the user's hand can be obtained. Based on the EMG signal data, the coordinates of the key joints of the user's hand can be determined. By using the coordinates of the key joints of the hand, the accurate position of the joints of the hand during the action of picking up a cup can be determined. Based on the accurate position of the joints, the user's hand size information and hand state information can be determined. No specific limitations are made here.

[0095] In some examples, determining the coordinates of key joints in a user's hand based on electromyography (EMG) signal data can be achieved by preprocessing the EMG signal data. This preprocessing may include one or more of the following: rectification, smoothing, baseline drift removal, motion artifact removal, filtering and denoising, power frequency notch filtering, and normalization. Rectification converts negative values ​​in the EMG signal data to positive values, allowing for subsequent analyses such as averaging and root mean square calculation. Smoothing extracts the macroscopic trends in the EMG signal data and reduces point-to-point random fluctuations. Baseline drift removal eliminates slow fluctuations superimposed on the EMG signal data caused by factors such as electrode slippage or sweating. Motion artifact removal addresses interference from large limb movements and can be suppressed using methods such as high-pass filtering. Power frequency notch filtering eliminates power frequency interference noise. Filtering removes noise signals outside a specified frequency band to preserve the effective frequency components of the EMG signal. Normalization can map the amplitude values ​​of electromyographic signal data to a uniform numerical range, thereby enabling the embodiments of this application to eliminate the magnitude differences in the amplitude of electromyographic signal data under different acquisition conditions, which is not specifically limited here.

[0096] The specific preprocessing method can be selected based on the actual application scenario and signal characteristics. In some cases, the signal can be filtered to remove interference components within a specific frequency range. In other cases, amplitude normalization can be performed to make the electromyographic signal data have a similar numerical range. In still other cases, baseline drift removal can be performed. It should be noted that any of the above processing methods can be used individually or in combination; no specific limitations are imposed here.

[0097] Furthermore, the preprocessed electromyography (EMG) signals can be input into the prediction model, where the EMG signal data can be converted to a standard format. The prediction model is pre-trained and possesses the ability to infer the coordinates of key hand joints from the EMG signals, and then identify hand size and state information (i.e., hand parameter information in this embodiment) based on these coordinates. In some cases, the prediction model can be deployed in an electronic device, which can directly call the model for inference locally. In other cases, the prediction model can also be deployed on a remote server, allowing the electronic device to send signal data to the server for inference via a network and receive the returned results.

[0098] For example, the predictive model can predict hand size and hand state information corresponding to a user's hand movements based on input electromyography (EMG) signal data. Illustratively, the hand size information reflects the size of the user's hand, such as palm size and finger length. The hand state information reflects the hand's movement during the execution of the target hand movement. The predictive model can output both types of information simultaneously. In some cases, the predictive model outputs a multi-dimensional vector, where one dimension corresponds to hand size information and another dimension corresponds to hand state information; this is not specifically limited here.

[0099] Optionally, this embodiment may also employ the following methods, but not limited thereto: obtaining the user's electromyographic signal data through a wearable wristband device, wherein the wearable wristband device is a device with electromyographic signal acquisition function.

[0100] In this embodiment, the wearable wristband device can be a device with electromyography (EMG) signal acquisition function, such as, but not limited to, an EMG acquisition wristband, an EMG acquisition watch, an EMG acquisition wristwatch, etc. It should be noted that the specific device type for EMG acquisition is not specifically limited here.

[0101] As an alternative approach, when a user wears a wearable bracelet on their hand or arm, the wearable bracelet can acquire electromyographic (EMG) signal data of the user's hand or arm, thereby determining the hand parameter information for the user to perform the target hand movement based on the acquired EMG signal data.

[0102] For example, if a user is wearing a wearable wristband on their hand or wrist while performing the action of picking up a cup, the wearable wristband can obtain electromyographic signal data of the user's hand, thereby determining the hand parameter information of the user performing the action of picking up a cup based on the collected electromyographic signal data.

[0103] Compared with related technologies, this embodiment uses the user's electromyography (EMG) signal data to determine the user's hand parameter information when the user performs a target hand movement. Based on the user's hand parameter information, the target hand skeleton can be displayed. The target hand skeleton can characterize the changes in the hand state during the user's performance of the target hand movement. Therefore, this embodiment uses EMG signals to identify hand parameters and display the skeleton without relying on hand images to display hand movements. Thus, even when the hand is grasped by an object that obscures the hand image, the hand movement at the obscured position can still be accurately displayed.

[0104] Furthermore, as Figure 1 To illustrate the specific implementation of the method shown, this embodiment provides a skeleton display device, such as... Figure 5 As shown, it includes: a determination module 31 and a display module 32; The determination module 31 is configured to determine the hand parameter information of the user performing the target hand action based on the user's electromyographic signal data; The display module 32 is configured to display the target hand skeleton of the user based on the hand parameter information. The target hand skeleton is used to characterize the changes in the hand state of the user during the execution of the target hand action.

[0105] In some examples of this embodiment, the target hand skeleton includes the joints of the user's hand and connecting lines connecting the joints. During the user's execution of the target hand action, the positions of the joints and the connecting lines change with the user's hand action.

[0106] In some examples of this embodiment, the display module 32 is specifically configured to obtain an execution screen of the user performing the target hand action; and to display the target hand skeleton of the user at the position of the user's hand in the execution screen.

[0107] In some examples of this embodiment, the display module 32 is further configured to, when the user's hand position is obscured by a grasping object, display the user's target hand skeleton at the user's hand position and at the position where the grasping object obscures the user's hand, based on the user's grasping information of the grasping object.

[0108] In some examples of this embodiment, the display module 32 is further configured to determine the occlusion position in the execution screen based on the contact distribution information between the user and the grasping object and the user's finger gripping force information on the grasping object; and to display the user's target hand skeleton at the hand position and the occlusion position.

[0109] In some examples of this embodiment, the display module 32 is further configured to determine, based on the execution screen, whether the hand position is obscured by the grasping object.

[0110] In some examples of this embodiment, the display module 32 is further configured to obtain the execution screen of the user performing the target hand action through a head-mounted device, wherein the head-mounted device is a device with image acquisition function.

[0111] In some examples of this embodiment, the display module 32 is further configured to obtain target distance information between the head-mounted device and the user's hand; and display the target distance information at the distance display position corresponding to the target hand skeleton.

[0112] In some examples of this embodiment, the display module 32 is specifically configured to display the user's basic hand skeleton based on the user's hand size information; and to adjust the state of the basic hand skeleton based on the user's hand state information to generate the target hand skeleton.

[0113] In some examples of this embodiment, the display module 32 is further configured to determine the degree of finger opening and closing of the user based on the joint angle of the user's hand; and to adjust the finger opening and closing state of the basic hand skeleton according to the degree of finger opening and closing to generate the target hand skeleton.

[0114] In some examples of this embodiment, module 32 is further configured to determine the joint positions of the finger joints in the basic hand skeleton and the connection angles of the finger connecting lines connecting the finger joints based on the degree of finger opening and closing; and to adjust the finger opening and closing state of the basic hand skeleton based on the joint positions and the connection angles to generate the target hand skeleton.

[0115] In some examples of this embodiment, the display module 32 is further configured to determine the user's skeleton adjustment parameters based on the user's palm size and finger length; and to adjust the skeleton size of a preset hand skeleton based on the skeleton adjustment parameters to generate the user's basic hand skeleton.

[0116] In some examples of this embodiment, module 32 is further configured to determine the initial joint point position of the user's hand and the length of the initial connecting line connecting the initial joint point based on the skeleton adjustment parameters; and to adjust the skeleton size of the preset hand skeleton according to the initial joint point position and the connecting line length to generate the user's basic hand skeleton.

[0117] In some examples of this embodiment, the determining module 31 is specifically configured to determine the coordinates of the key joints of the user's hand based on the electromyographic signal data; and to determine the hand parameter information of the user performing the target hand action based on the coordinates of the key joints.

[0118] In some examples of this embodiment, the determining module 31 is further configured to obtain the user's electromyographic signal data through a wearable wristband device, which is a device with electromyographic signal acquisition function.

[0119] It should be noted that other corresponding descriptions of the functional units involved in the communication system provided in this embodiment can be found in [reference needed]. Figure 1 and Figure 2 The corresponding description in [the document] will not be repeated here.

[0120] A1. A method for displaying a skeleton, comprising: Based on the user's electromyographic signal data, determine the hand parameter information of the user performing the target hand movement; Based on the hand parameter information, the target hand skeleton of the user is displayed, which is used to characterize the changes in the hand state of the user during the execution of the target hand action.

[0121] A2. According to the method described in A1, the target hand skeleton includes the joints of the user's hand and connecting lines connecting the joints. During the user's execution of the target hand action, the positions of the joints and the connecting lines change with the user's hand action.

[0122] A3. According to the method described in A1, displaying the user's hand skeleton includes: Obtain the execution screen of the user performing the target hand action; The target hand skeleton of the user is displayed at the user's hand position in the execution screen.

[0123] A4. According to the method described in A3, displaying the target hand skeleton of the user at the user's hand position in the execution screen includes: When the user's hand is obscured by a grasping object, the target hand skeleton of the user is displayed at the unobscured position of the user's hand and at the position where the grasping object obscures the user's hand, based on the user's grasping information of the grasping object.

[0124] A5. According to the method described in A4, the step of displaying the user's target hand skeleton at the unobstructed position of the user's hand and the obstructed position of the user's hand by the grasping object, based on the user's grasping information of the grasping object, includes: Based on the contact distribution information between the user and the object being grasped and the user's finger gripping force information on the object being grasped, the occlusion position in the execution screen is determined; The user's target hand skeleton is displayed in both the unobstructed and obstructed positions.

[0125] A6. The method according to A3 further includes: Based on the execution screen, determine whether the hand position is obscured by the grasping object.

[0126] A7. The method according to any one of A3 to A6, the method further comprising: The execution screen of the user performing the target hand action is obtained through a head-mounted device, which is a device with image capture function.

[0127] A8. The method according to A7 further includes: Obtain target distance information between the head-mounted device and the user's hand; The target distance information is displayed at the distance display position corresponding to the target hand skeleton.

[0128] A9. According to the method described in A1, the step of displaying the user's target hand skeleton based on the hand parameter information includes: Based on the user's hand size information, the user's basic hand skeleton is displayed; Based on the user's hand state information, the state of the basic hand skeleton is adjusted to generate the target hand skeleton.

[0129] A10. According to the method described in A9, the step of adjusting the state of the basic hand skeleton based on the user's hand state information to generate the target hand skeleton includes: The degree of finger opening and closing of the user is determined based on the joint angle of the user's hand; Based on the degree of finger opening and closing, the basic hand skeleton is adjusted to adjust the finger opening and closing state to generate the target hand skeleton.

[0130] A11. According to the method described in A10, the step of adjusting the finger opening / closing state of the basic hand skeleton based on the finger opening / closing degree to generate the target hand skeleton includes: Based on the degree of finger opening and closing, determine the joint positions of the finger joints in the basic hand skeleton and the connection angle of the finger connecting lines connecting the finger joints; Based on the joint position and the connection angle, the finger opening and closing state of the basic hand skeleton is adjusted to generate the target hand skeleton.

[0131] A12. The method according to any one of A9 to A11, wherein displaying the user's basic hand skeleton based on the user's hand size information includes: Based on the user's palm size and finger length, determine the user's skeletal adjustment parameters; Based on the skeleton adjustment parameters, the size of the preset hand skeleton is adjusted to generate the user's basic hand skeleton.

[0132] A13. According to the method described in A12, adjusting the size of the preset hand skeleton based on the skeleton adjustment parameters to generate the user's basic hand skeleton includes: Based on the skeleton adjustment parameters, the initial joint point position of the user's hand and the length of the initial connecting line connecting the initial joint point are determined. Based on the initial joint position and the length of the connecting line, the size of the preset hand skeleton is adjusted to generate the user's basic hand skeleton.

[0133] A14. According to the method described in A1, determining the hand parameter information of the user performing the target hand movement based on the user's electromyographic signal data includes: Based on the electromyographic signal data, the coordinates of the key joints of the user's hand are determined; Based on the coordinates of the key joint points, the hand parameter information of the user performing the target hand action is determined.

[0134] A15. The method according to A1 further includes: The user's electromyography (EMG) signal data is obtained through a wearable wristband device, which is a device with EMG signal acquisition function.

[0135] A16. A skeleton display device, comprising: The determination module is configured to determine the hand parameter information of the user performing the target hand action based on the user's electromyographic signal data; The display module is configured to display the target hand skeleton of the user based on the hand parameter information. The target hand skeleton is used to characterize the changes in the hand state of the user during the execution of the target hand action.

[0136] A17. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any one of A1 to A15.

[0137] A18. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor, when executing the computer program, implements the method of any one of claims A1 to A15.

[0138] A19. A program product comprising computer instructions for causing a computer to perform the method of any one of claims A1 to A15.

[0139] Based on the above, Figure 1 and Figure 2 Accordingly, this embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. Figure 1 and Figure 2 The method shown.

[0140] Based on this understanding, the technical solutions of the embodiments of this application can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the embodiments of this application.

[0141] like Figure 6 The diagram shown is a hardware structure schematic of an electronic device according to the present invention, comprising: At least one processor 401; and, A memory 402 is communicatively connected to at least one of the processors 401; wherein, The memory 402 stores instructions that can be executed by at least one of the processors to enable the at least one of the processors to perform the vehicle control method as described above.

[0142] Figure 6 Take a processor 401 as an example.

[0143] The electronic device may also include an input device 403 and a display device 404.

[0144] The processor 401, memory 402, input device 403, and display device 404 can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0145] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the vehicle control method in the embodiments of this application. Figure 1 and Figure 2 The method flow is shown. The processor 401 executes various functional applications and data processing by running non-volatile software programs, instructions, and modules stored in the memory 402, thereby implementing the vehicle control method in the above embodiments.

[0146] Memory 402 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the vehicle control method, etc. Furthermore, memory 402 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 402 may optionally include memory remotely located relative to processor 401, and these remote memories may be connected to the apparatus performing the vehicle control method via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0147] The input device 403 can receive user clicks and generate signal inputs related to user settings and function control of the vehicle control method. The display device 404 may include a display screen or other display equipment.

[0148] The one or more modules are stored in the memory 402, and when run by the one or more processors 401, they execute the vehicle control method in any of the above method embodiments.

[0149] In some embodiments, the aforementioned physical device may further include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, an input unit such as a keyboard, etc., and optionally, a USB interface, a card reader interface, etc. In some embodiments, the network interface may include a standard wired interface, a wireless interface (such as a Wi-Fi interface), etc.

[0150] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0151] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0152] Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments of this application can be implemented by means of software plus necessary general-purpose hardware platforms, or by hardware. By applying the solution of this embodiment, compared with related technologies, this embodiment can determine the user's hand parameter information when the user performs a target hand movement by using the user's electromyographic signal data. Based on the user's hand parameter information, the target hand skeleton of the user can be displayed. The target hand skeleton can characterize the changes in hand state during the user's performance of the target hand movement. Thus, this embodiment uses electromyographic signals to identify hand parameters and display the skeleton without relying on hand images to display hand movements. Therefore, even when the object grasped by the hand occludes the hand image, the hand movement at the occluded position can still be accurately displayed.

[0153] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0154] The above descriptions are merely specific implementations of the embodiments of this application, enabling those skilled in the art to understand or implement the embodiments of this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the embodiments of this application. Therefore, the embodiments of this application are not to be limited to the embodiments described herein, but are to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for displaying a skeleton, characterized in that, include: Based on the user's electromyographic signal data, determine the hand parameter information of the user performing the target hand movement; Based on the hand parameter information, the target hand skeleton of the user is displayed, which is used to characterize the changes in the hand state of the user during the execution of the target hand action.

2. The method according to claim 1, characterized in that, The target hand skeleton includes the joints of the user's hand and connecting lines connecting the joints. During the user's execution of the target hand movement, the positions of the joints and the connecting lines change with the user's hand movement.

3. The method according to claim 1, characterized in that, The display of the user's hand skeleton includes: Obtain the execution screen of the user performing the target hand action; The target hand skeleton of the user is displayed at the user's hand position in the execution screen.

4. The method according to claim 3, characterized in that, The display of the user's target hand skeleton at the user's hand position in the execution screen includes: When the user's hand is obscured by a grasping object, the target hand skeleton of the user is displayed at the unobscured position of the user's hand and at the position where the grasping object obscures the user's hand, based on the user's grasping information of the grasping object.

5. The method according to claim 3 or 4, characterized in that, The method further includes: The execution screen of the user performing the target hand action is obtained through a head-mounted device, which is a device with image capture function.

6. The method according to claim 5, characterized in that, The method further includes: Obtain target distance information between the head-mounted device and the user's hand; The target distance information is displayed at the distance display position corresponding to the target hand skeleton.

7. The method according to claim 1, characterized in that, The step of displaying the user's target hand skeleton based on the hand parameter information includes: Based on the user's hand size information, the user's basic hand skeleton is displayed; Based on the user's hand state information, the state of the basic hand skeleton is adjusted to generate the target hand skeleton.

8. The method according to claim 7, characterized in that, The step of adjusting the state of the basic hand skeleton based on the user's hand state information to generate the target hand skeleton includes: The degree of finger opening and closing of the user is determined based on the joint angle of the user's hand; Based on the degree of finger opening and closing, the basic hand skeleton is adjusted to adjust the finger opening and closing state to generate the target hand skeleton.

9. The method according to claim 7 or 8, characterized in that, The step of displaying the user's basic hand skeleton based on the user's hand size information includes: Based on the user's palm size and finger length, determine the user's skeletal adjustment parameters; Based on the skeleton adjustment parameters, the size of the preset hand skeleton is adjusted to generate the user's basic hand skeleton.

10. A program product, characterized in that, Includes computer instructions for causing a computer to perform the method of any one of claims 1 to 9.

Images