Data processing method, device, electronic device and storage medium for manipulator

Abstract

The present application discloses a data processing method, device, electronic device and storage medium for a manipulator, the method comprising: controlling the display screen of a user device to display an action execution interface, the action execution interface comprising target image information; in response to an image confirmation message uploaded by the user device, determining the image type of the target image information, and according to the image type, controlling the display screen of the user device to jump to display a hand shape determination interface, the hand shape determination interface being used to display the target hand shape, the target hand shape being the content determined by analyzing the target image information, the image type comprising a static image and a dynamic image; when the image type is a dynamic image, there are multiple target hand shapes; in response to a hand shape confirmation message uploaded by the user device, sending adjustment information to the user device to instruct the manipulator to be adjusted to the target hand shape. The present application is conducive to improving the data processing capability of the hand function training system, realizing the execution of daily activity operations assisted by the manipulator, and improving rehabilitation efficiency and convenience of life.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a data processing method, device, electronic device and storage medium for a manipulator. Background Art

[0002] Many behaviors and activities in daily life are determined based on the usage habits of the left and right hands. For example, the default way to unscrew a bottle cap is with the right hand, so unscrewing it counterclockwise is more in line with the right-handed usage habits. In real life, a user's hand injury will greatly affect their daily life. For example, when the right hand is injured, it is inconvenient to use force when using the left hand to perform activities that are suitable for the right hand, such as turning the car key, making the action difficult to perform. The current manipulator can assist users in rehabilitation training according to pre-configured actions, but it cannot meet the user's daily activity needs and has limited capabilities. Summary of the invention

[0003] The embodiments of the present application provide a data processing method, device, electronic device and storage medium for a manipulator, in order to improve the data processing capability of a hand function training system, realize the function of assisting the user in performing daily activities through a manipulator, and improve rehabilitation efficiency and convenience of life.

[0004] In a first aspect, an embodiment of the present application provides a data processing method for a manipulator, which is applied to a server of a hand function training system, wherein the hand function training system includes the server, a user device, and a manipulator, wherein the server is communicatively connected to the user device, and the user device is communicatively connected to the manipulator, and the method includes:

[0005] Controlling the display screen of the user device to display an action execution interface, wherein the action execution interface includes target image information, wherein the target image information is image content uploaded by the manipulator in response to an imitation instruction, wherein the image content includes hand shape information of a user's hand without the manipulator, wherein the imitation instruction is content sent from the user device to the manipulator, and wherein the imitation instruction is used to instruct the manipulator to capture the image content;

[0006] In response to the image confirmation message uploaded by the user device, determine the image type of the target image information, and control the display screen of the user device to jump to display a hand shape determination interface according to the image type, wherein the hand shape determination interface is used to display the target hand shape, the target hand shape is determined by analyzing the target image information, and the image type includes a static image and a dynamic image; when the image type is a dynamic image, there are multiple target hand shapes;

[0007] In response to a hand shape confirmation message uploaded by the user device, adjustment information is sent to the user device, wherein the adjustment information is used to instruct the user device to send an adjustment instruction to the manipulator, wherein the adjustment instruction is used to instruct the manipulator to adjust to a target hand shape according to a control scheme, wherein the control scheme includes control content for adjusting the manipulator from a current hand shape to the target hand shape.

[0008] In a possible example of the first aspect, the adjustment instruction includes a first adjustment content and a second adjustment content;

[0009] The first adjustment content is used to instruct the manipulator to adjust to a reference hand shape, and the reference hand shape and the target hand shape are of the same hand shape type;

[0010] The second adjustment content is used to instruct the manipulator to continuously increase the adjustment angle after adjusting to the reference hand shape until the real-time pressure of the manipulator is greater than the preset pressure, so as to adjust to the target hand shape.

[0011] In a possible example of the first aspect, the real-time pressure of the manipulator is the real-time pressure of a target knuckle of the manipulator, and the target knuckle is determined by the following steps:

[0012] Determining whether a target object exists according to the target image information;

[0013] If so, the knuckle in contact with the target object is determined as the target knuckle;

[0014] If not present, a preset knuckle is determined as the target knuckle, and the preset knuckle is the knuckle that contacts each other in the target hand shape.

[0015] In a possible example of the first aspect, the adjustment instruction also includes a third adjustment content, and the third adjustment content includes: when the real-time pressure of the manipulator is greater than the preset pressure, determining whether the actual force point of the manipulator is a reference force point, and when the actual force point is not the reference force point, adjusting the conventional force point of the manipulator to the position of the actual force point, and the conventional force point is the force point corresponding to the target hand shape.

[0016] In a possible example of the first aspect, the third adjustment content also includes: before adjusting the conventional force point of the manipulator to the position of the actual force point, determining whether the actual force point is the conventional force point, so as to adjust the conventional force point of the manipulator to the position of the actual force point when the actual force point is not the conventional force point.

[0017] In a possible example of the first aspect, the reference force point is determined by the following steps:

[0018] Determining the movement direction of the manipulator according to the reference hand shape and the target hand shape;

[0019] A reference force point of a target knuckle in the manipulator is determined according to the movement direction.

[0020] In a possible example of the first aspect, controlling the display screen of the user equipment to jump to display the hand shape determination interface according to the image type includes:

[0021] When it is determined that the image type is a dynamic image, determining target states corresponding to a plurality of target hand shape changes according to the target image information;

[0022] Determine display information according to the target hand shapes and the target states corresponding to the target hand shapes when they are changed, wherein the display information is the display content of the hand shape determination interface;

[0023] Control the display screen of the user equipment to jump to display the hand shape determination interface.

[0024] In a second aspect, an embodiment of the present application provides a data processing device for a manipulator, which is applied to a server of a hand function training system, wherein the hand function training system comprises the server, a user device, and a manipulator, wherein the server is communicatively connected to the user device, and the user device is communicatively connected to the manipulator, and the data processing device for the manipulator comprises:

[0025] a control unit, configured to control the display screen of the user device to display an action execution interface, wherein the action execution interface includes target image information, wherein the target image information is image content uploaded by the manipulator in response to an imitation instruction, wherein the image content includes hand shape information of a user's hand without the manipulator, and wherein the imitation instruction is content sent from the user device to the manipulator, and wherein the imitation instruction is used to instruct the manipulator to capture the image content;

[0026] A first response unit is used to respond to the image confirmation message uploaded by the user device, determine the image type of the target image information, and control the display screen of the user device to jump to display a hand shape determination interface according to the image type, wherein the hand shape determination interface is used to display the target hand shape, the target hand shape is determined by analyzing the target image information, and the image type includes a static image and a dynamic image; when the image type is a dynamic image, there are multiple target hand shapes;

[0027] A second response unit is used to send adjustment information to the user device in response to a hand shape confirmation message uploaded by the user device, wherein the adjustment information is used to instruct the user device to send an adjustment instruction to the manipulator, and the adjustment instruction is used to instruct the manipulator to adjust to a target hand shape according to a control scheme, and the control scheme includes control content for adjusting the manipulator from a current hand shape to the target hand shape.

[0028] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps in the first aspect of the embodiment of the present application.

[0029] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium that stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute some or all of the steps described in the first aspect of this embodiment.

[0030] It can be seen that in this embodiment, the server controls the display screen of the user device to display the action execution interface, the action execution interface includes target image information, the target image information is the image content uploaded by the robot in response to the imitation instruction, the image content includes the hand shape information of the user's hand without wearing the robot, the imitation instruction is the content sent by the user device to the robot, and the imitation instruction is used to instruct the robot to collect image content; then, in response to the image confirmation message uploaded by the user device, the image type of the target image information is determined, and the display screen of the user device is controlled to jump to display the hand shape determination interface according to the image type, the hand shape determination interface is used to display the target hand shape, the target hand shape is the content determined by analyzing the target image information, and the image type includes static image and dynamic image; when the image type is a dynamic image, there are multiple target hand shapes; then, in response to the hand shape confirmation message uploaded by the user device, adjustment information is sent to the user device, the adjustment information is used to instruct the user device to send an adjustment instruction to the robot, and the adjustment instruction is used to instruct the robot to adjust to the target hand shape according to the control scheme, and the control scheme includes control content for adjusting the robot from the current hand shape to the target hand shape. It can be seen that the present application can control the user device to display the action execution interface to display the target image information uploaded by the user, and then control the user device bar to display the hand shape confirmation interface of the user displaying the target hand shape based on the image confirmation message uploaded by the user for the target image information, and then send adjustment information to the user device based on the hand shape confirmation message uploaded by the user for the target hand shape, to instruct the robotic arm to adjust to the target hand shape, so that the robotic arm assists the user to use the injured hand to perform the actions required for daily activities based on the target hand shape, and then realize the function of assisting the user in performing daily activities operations through the robotic arm, which is beneficial to improving the data processing capability of the hand function training system and the accuracy of processed data, and improving the user's rehabilitation efficiency and life convenience. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 It is a schematic diagram of the architecture of a hand function training system provided in an embodiment of the present application;

[0033] Figure 2 It is a structural schematic diagram of a manipulator provided in an embodiment of the present application;

[0034] Figure 3 It is a flow chart of a data processing method for a manipulator provided in an embodiment of the present application;

[0035] Figure 4 is a schematic diagram of an action execution interface provided in an embodiment of the present application;

[0036] Figure 5 is a schematic diagram of a hand shape determination interface provided in an embodiment of the present application;

[0037] Figure 6 is an interface schematic diagram of another action execution interface provided in an embodiment of the present application;

[0038] Figure 7 This is a schematic diagram of another action execution interface provided in an embodiment of the present application;

[0039] Figure 8 is a schematic diagram of another hand shape determination interface provided in an embodiment of the present application;

[0040] Fig. 9 is a composition example diagram of an electronic device provided in an embodiment of the present application;

[0041] Fig.10 It is a functional unit composition block diagram of a first data processing device for a manipulator provided by an embodiment of the present application;

[0042] Fig.11 This is a block diagram of the functional units of a second data processing device for a manipulator provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0044] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0045] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0046] The embodiments of the present application are described below with reference to the accompanying drawings.

[0047] The technical solution of this application can be applied to Figure 1 The hand function training system 10 for a manipulator shown includes a server 110, a user device 120, and a manipulator 130. The server is connected to the user device for communication, and the user device is connected to the manipulator for communication.

[0048] Among them, the server 110 can be an outsourced server, a cloud server, an edge server, an intelligent robot, etc., which is not limited here.

[0049] The user device 120 may be a terminal device configured with a software application provided by a server for controlling the manipulator, and the user device 120 may realize the relevant functions of the software application by communicating with the server. For example, the user device 120 may be a smart phone (such as an Android phone, an iOS phone, a Windows Phone phone, etc.), a smart computer (such as a tablet computer, a PDA, a laptop computer, a desktop computer, etc.), a driving recorder, a mobile Internet device (MID, Mobile Internet Device) or a wearable device (such as a smart watch, a Bluetooth headset), etc. The above are only examples, not exhaustive, and include but are not limited to the above devices.

[0050] Among them, the manipulator is a rehabilitation training tool. The manipulator is mainly used for continuous passive training activities of the metacarpophalangeal joints, proximal interphalangeal joints and distal interphalangeal joints. Based on the theory of neuroplasticity and motor relearning, through repeated task-oriented training of the patient's injured hand, it promotes the reorganization and compensation of the nervous system, enhances the proprioception of the affected hand, induces active movement, prevents muscle atrophy, improves joint mobility, and assists the patient's injured hand to gradually recover fine movements such as grasping, holding, and pinching.

[0051] In this application, the robot arm can also be adjusted to an adaptive target hand shape according to the user's daily activity needs, so that the user can use the injured hand with the assistance of the robot arm to perform the actions required for daily activities based on the target hand shape, thereby meeting the user's daily activity needs. Among them, the target hand shape is the content analyzed and confirmed based on the target image information uploaded by the user.

[0052] In a specific implementation, the manipulator may be configured with a communication unit, which is used to communicate with the user device to receive control instructions from the user device, so as to control the various joints of the manipulator to adjust according to the control content carried by the control instructions. For example, the user device may be a smart phone or smart computer equipped with a software application for controlling the manipulator. The manipulator may be connected to the user device via Bluetooth to receive control instructions issued by the user device through the software application, thereby reducing production costs and ensuring the reliability of training.

[0053] Please refer to FIG. 2 , which is a schematic diagram of the structure of a manipulator provided in an embodiment of the present application. Figure 2 The example structure 20 is an example structure of a manipulator. Specifically, the example structure 20 includes multiple joint drivers 21, a palm fixing belt 22 and a finger fixing belt 23. Each finger of the example structure 20 corresponds to at least one joint driver 21. The joint driver 21 outputs driving information based on the control instruction, so that the corresponding finger on the example structure 20 moves, so that the example structure 20 can make a corresponding action. When the user uses the example structure 20 for rehabilitation training or activities, the palm fixing belt 22 and the finger fixing belt 23 are used to bind the injured hand to the example structure 20, and then the example structure 20 makes a corresponding action based on the control instruction, and drives the user's hand to make a corresponding action, so as to realize the user's passive activity.

[0054] See also Figure 3 , Figure 3 is a flow chart of a data processing method for a manipulator provided in an embodiment of the present application. The method can be applied in Figure 1 The server 110 in the hand function training system shown in FIG. Figure 3 As shown, the data processing method for the manipulator comprises the following steps:

[0055] S310, controlling the display screen of the user equipment to display an action execution interface.

[0056] Among them, the action execution interface includes target image information, which is the image content uploaded by the robot in response to the imitation instruction. The image content includes the hand shape information of the user's hand without wearing the robot. The imitation instruction is the content sent by the user device to the robot. The imitation instruction is used to instruct the robot to collect image content.

[0057] The hand not wearing the robot arm can be the user's own hand. For example, if the user's right hand is injured, the hand not wearing the robot arm can be the user's left hand. Alternatively, the hand not wearing the robot arm can also be the hand of someone other than the user. In this case, the hand can be the left hand or the right hand. When the hand corresponding to the image content is inconsistent with the hand wearing the robot arm, the server will automatically identify and proofread it.

[0058] Specifically, the robot is equipped with a shooting device, such as a camera. The image content is the content shot by the robot.

[0059] In a specific implementation, the user device can enable the daily use function through the software application of the robot to send an imitation instruction to the robot. After receiving the imitation instruction, the robot activates the shooting device, so that the user can shoot the hand movements without wearing the robot through the shooting device of the robot, thereby generating image content and uploading it to the user device.

[0060] Specifically, during the shooting process, the robot can upload the captured image content to the user device in real time, and confirm whether to adopt the captured image content through the user device, so as to ensure that the robot accurately captures the hand movements without the robot. Alternatively, the robot can be further configured with a display device, such as a display screen, for displaying the captured image content, so as to improve the convenience of shooting while ensuring that the robot accurately captures the hand movements without the robot.

[0061] In a specific implementation, the action execution interface also includes an image confirmation control. The user can perform a selection operation (such as single-click, double-click, or long press, etc.) on the image confirmation control in the action execution interface displayed on the user device. The user device will send an image confirmation message to the server based on the selection operation to enable the server to determine the data for processing.

[0062] For example, see Figure 4, the action acquisition interface 410 includes an information display control 411 and an upload control 412. Among them, the information display control 411 is used to indicate the display area of ​​the target image information. The user can perform a selection operation on the information display control 411 or the upload control 412 to send an imitation instruction to the robot. After the robot shoots and uploads the image content according to the imitation instruction, the server will control the display content of the user device to jump from the action acquisition interface 410 to the action execution interface 420 after confirming that the robot has uploaded the image content to the user device. The action execution interface 420 includes an image display control 421, a re-upload control 423, and an image confirmation control 422. Among them, the image display control 421 is used to display the target image information so that the user can review whether the target image information used for server processing is accurate. If the user determines that the target image information displayed on the action execution interface 420 is incorrect during the review, the user can perform a selection operation on the re-upload control 423 to send the imitation instruction to the robot again, so that the robot can re-shoot and upload new image content. If the user determines during review that the target image information displayed on the action execution interface 420 is accurate, the user may perform a selection operation on the image confirmation control 422 so that the user device sends an image confirmation message to the server.

[0063] S320: In response to the image confirmation message uploaded by the user device, determine the image type of the target image information, and control the display screen of the user device to jump to display a hand shape determination interface according to the image type.

[0064] The hand shape determination interface is used to display the target hand shape, which is determined by analyzing the target image information. The image type includes static image and dynamic image; when the image type is dynamic image, there are multiple target hand shapes.

[0065] Among them, in the case of static images, the target image information is a single picture, and in the case of dynamic images, the target image information is multiple pictures or videos.

[0066] Specifically, the image confirmation message can carry target image information, so that the image content uploaded by the robot can be uploaded to the server after confirmation by the user. Compared with the robot uploading the image content to the user device and then directly uploading it to the server, the target image information confirmed by the user device can be sent to the server, so as to reduce the data processing and storage capacity of the server while ensuring the accuracy of the data processed by the server.

[0067] In the specific implementation, after receiving the image confirmation message uploaded by the user device, the server will analyze and process the target image information to obtain the target hand shape. After the server processes and obtains the target hand shape, it will control the display content of the user device to jump from the action execution interface to the hand shape determination interface, so as to display the target hand shape in the hand shape determination interface, thereby providing it to the user for inspection and confirmation, and further ensuring the accuracy of the hand shape adjusted by the robot.

[0068] Specifically, the target hand shape displayed in the hand shape determination interface may include the target hand shape type, the knuckles touching the target object (or the knuckles touching each other when there is no target object), and the joint angle adjustment information corresponding to the target hand shape.

[0069] For example, see Figure 5 , the hand shape determination interface 510 includes a hand shape display control 511, a hand shape type display control 512, an angle display control 513 and a hand shape confirmation control 514. Among them, the hand shape display control 511 is used to display the hand shape corresponding to the hand wearing the manipulator when performing actions required for daily activities. The content displayed by the hand shape display control 511 may include the target objects required when performing actions required for daily activities, so as to provide action references for users. The hand shape type display control 512 is used to display the target hand shape type analyzed and processed by the server, and the target hand shape type is the hand shape type of the target hand shape. The hand shape types include two-finger pinching, side pinching, hook-shaped grasping, columnar grasping, spherical grasping, etc., which are not further restricted here. The angle display control 513 is used to display the angle adjustment information of each joint corresponding to the target hand shape, and the adjustment angle is determined by the server analyzing and processing the target image information. When the user verifies that the contents displayed by the hand shape display control 511, the hand shape type display control 512, and the angle display control 513 are correct, a selection operation can be performed on the hand shape confirmation control 514, so that the user device sends a hand shape confirmation message to the server.

[0070] S330: In response to the hand shape confirmation message uploaded by the user equipment, send adjustment information to the user equipment.

[0071] Among them, the adjustment information is used to instruct the user device to send an adjustment instruction to the manipulator, and the adjustment instruction is used to instruct the manipulator to adjust to the target hand shape according to the control scheme, and the control scheme includes control content for adjusting the manipulator from the current hand shape to the target hand shape.

[0072] In a specific implementation, after receiving the hand shape confirmation message, the server will generate a control scheme according to the target hand shape, and the control scheme includes control content for adjusting the manipulator from the current hand shape of the manipulator to the target hand shape. The server will send adjustment information carrying the control scheme to the user device, so that the user device sends the control scheme to the manipulator through the adjustment instruction, so that the manipulator is adjusted to the target hand shape according to the control scheme.

[0073] It can be seen that in this embodiment, the server controls the display screen of the user device to display the action execution interface, the action execution interface includes target image information, the target image information is the image content uploaded by the robot in response to the imitation instruction, the image content includes the hand shape information of the user's hand without wearing the robot, the imitation instruction is the content sent by the user device to the robot, and the imitation instruction is used to instruct the robot to collect image content; then, in response to the image confirmation message uploaded by the user device, the image type of the target image information is determined, and the display screen of the user device is controlled to jump to display the hand shape determination interface according to the image type, the hand shape determination interface is used to display the target hand shape, the target hand shape is the content determined by analyzing the target image information, and the image type includes static image and dynamic image; when the image type is a dynamic image, there are multiple target hand shapes; then, in response to the hand shape confirmation message uploaded by the user device, adjustment information is sent to the user device, the adjustment information is used to instruct the user device to send an adjustment instruction to the robot, and the adjustment instruction is used to instruct the robot to adjust to the target hand shape according to the control scheme, and the control scheme includes control content for adjusting the robot from the current hand shape to the target hand shape. It can be seen that the present application can control the user device to display the action execution interface to display the target image information uploaded by the user, and then control the user device bar to display the hand shape confirmation interface of the user displaying the target hand shape based on the image confirmation message uploaded by the user for the target image information, and then send adjustment information to the user device based on the hand shape confirmation message uploaded by the user for the target hand shape, to instruct the robotic arm to adjust to the target hand shape, so that the robotic arm assists the user to use the injured hand to perform the actions required for daily activities based on the target hand shape, and then realize the function of assisting the user in performing daily activities operations through the robotic arm, which is beneficial to improving the data processing capability of the hand function training system and the accuracy of processed data, and improving the user's rehabilitation efficiency and life convenience.

[0074] In one possible example, the adjustment instruction includes a first adjustment content and a second adjustment content; the first adjustment content is used to instruct the manipulator to adjust to a reference hand shape, and the reference hand shape and the target hand shape are of the same hand type; the second adjustment content is used to instruct the manipulator to continuously increase the adjustment angle after adjusting to the reference hand shape until the real-time pressure of the manipulator is greater than the preset pressure, so as to adjust to the target hand shape.

[0075] The adjustment angle refers to the angle of each joint that the manipulator needs to further adjust according to the second adjustment content. The preset pressure is the content preset according to the experience value. In the specific implementation, the preset pressure can be configured differently according to different hand types.

[0076] Specifically, the adjustment angle of each joint corresponding to the reference hand shape is smaller than the adjustment angle of each joint corresponding to the target hand shape. That is, taking the existence of a target object as an example, the distance between the knuckles of the reference hand shape and the target object is greater than the distance between the knuckles of the target hand shape and the target object. The reference hand shape is determined by the server. Specifically, the server can determine the target angle adjustment information of each joint corresponding to the target hand shape based on the target image information, and then process the angle adjustment information according to preset rules to obtain the reference hand shape. Among them, the target angle adjustment information is used to characterize the angle value that the joint needs to be adjusted from the angle corresponding to the current hand shape to the target hand shape. The preset rules can be determined according to actual needs. For example, the preset rule can be that the angle adjustment information corresponding to the reference hand shape is less than the target angle adjustment information, and the difference is 10°.

[0077] Specifically, the robot is equipped with a pressure sensor for detecting real-time pressure.

[0078] In a specific implementation, the real-time pressure of the manipulator is the real-time pressure of the target knuckles in the manipulator, and the target knuckles are determined by the following steps: determining whether a target object exists based on the target image information; if so, determining the knuckles in contact with the target object as the target knuckles; if not, determining the preset knuckles as the target knuckles, and the preset knuckles are the knuckles in contact with each other in the target hand shape.

[0079] Among them, the target object is an object grasped by a hand without a robotic arm.

[0080] Specifically, when analyzing and processing the target image information, the server will analyze and identify whether the hand without the robot gripper is holding the target object, so as to determine whether the target object exists in the target image information. If the target object exists, the server can further analyze the target image information to determine the knuckles of the hand without the robot gripper that are in contact with the target object, and determine them as the target knuckles. For example, refer to Figure 4 , if the action required for daily activities is turning a key, the target knuckles can be the first knuckle of the thumb and the second knuckle of the index finger. When there is no target object, the server further analyzes the image information to determine whether there are knuckles that are in contact with each other on the hand that is not wearing the robot arm, and if there are knuckles that are in contact with each other, the knuckles that are in contact with each other are determined as target knuckles. For example, taking the action of knocking on the door as an example, the target knuckles can be determined as the first knuckle of the thumb and the first knuckle of the ring finger.

[0081] It can be seen that in this example, by defining the real-time pressure as the real-time pressure of the target knuckle, and allowing the server to determine the target knuckle by analyzing and processing the target image information, the accuracy of the target knuckle used to monitor the real-time pressure can be guaranteed, making the real-time pressure monitoring more in line with the actual usage of the target hand shape, thereby improving the accuracy of motion adjustment monitoring.

[0082] In a specific implementation, after receiving the adjustment instruction, the manipulator will first adjust from the current hand shape to the reference hand shape according to the first adjustment content. After the manipulator adjusts to the reference hand shape according to the first adjustment content, it can determine the joints that need to be further adjusted according to the second adjustment content, thereby continuously increasing the adjustment angle of each joint that needs to be adjusted according to the second adjustment content, so as to gradually adjust and shorten the distance between the target knuckle and the target object, so that the user can hold the target object firmly, and increase the pressure between the target object and the target knuckle, so as to increase the real-time pressure of the target knuckle under the action of the reaction force. Similarly, if there is no target object, the adjustment angle of each joint that needs to be adjusted is continuously increased according to the second adjustment content, that is, the distance between the target knuckles is gradually adjusted and shortened, so as to increase the pressure between the target knuckles, so as to increase the real-time pressure of the target knuckle under the action of the reaction force. When the real-time pressure reaches the preset pressure, the manipulator can determine that it has been adjusted to the target hand shape.

[0083] Specifically, the process of determining each joint that needs to be further adjusted according to the second adjustment content includes: obtaining the angle information of each joint corresponding to the reference hand shape, and the angle information of each joint corresponding to the target hand shape, so as to determine the angle difference of each joint, and then determine the joints that need to be adjusted according to the second adjustment content according to the angle difference. For example, if the angle difference of a joint is 0°, it is determined that the joint is not a joint that needs to be adjusted according to the second adjustment content. If the angle information corresponding to a joint in the target hand shape is greater than the corresponding angle information in the reference hand shape, or the angle difference of a joint is not 0°, it is determined that the joint is a joint that needs to be further adjusted according to the second adjustment content. The above process can be performed by the server before sending the adjustment information to the user device, so as to carry the content in the adjustment information, and send it to the manipulator through the adjustment instruction. Alternatively, the above process can also be performed by the manipulator after adjusting to the reference hand shape.

[0084] It can be seen that in this example, the manipulator is adjusted through the first adjustment content and the second adjustment content to determine that the manipulator has been adjusted to the target hand shape when the real-time pressure of the target knuckle is greater than the preset pressure. Compared with the manipulator directly adjusting according to the joint angle information corresponding to the target hand shape, it is more adapted to the actual situation and can avoid the impact of the server's analysis error of the target image information on the grasping effect.

[0085] In a possible example, the adjustment instruction also includes a third adjustment content, and the third adjustment content includes: when the real-time pressure of the manipulator is greater than the preset pressure, determining whether the actual force point of the manipulator is a reference force point, and when the actual force point is not the reference force point, adjusting the conventional force point of the manipulator to the position of the actual force point, and the conventional force point is the force point corresponding to the target hand shape.

[0086] Among them, the actual force point is the maximum force position of the manipulator, and the actual force point is determined according to the assembly position of the pressure sensor that detects the pressure. The conventional force point can be a force point of the hand type corresponding to the preset target hand type to improve the convenience of configuration. Alternatively, the conventional force point can also be the force point with the most force times determined based on the data of the same hand type corresponding to the target hand type in the historical data to improve the adaptability of use. It can be set specifically according to the needs and no further restrictions are made here.

[0087] In a specific implementation, before the manipulator determines that the hand shape has been adjusted to the target hand shape, it can further determine whether the actual force point of the manipulator is the reference force point when the real-time pressure of the target knuckle is greater than the preset pressure. If the actual force point is the reference force point, it indicates that the hand shape is correct, and it can be determined that the manipulator has been adjusted to the target hand shape. If the actual force point is not the reference force point, it indicates that the hand shape is incorrect, and the manipulator can be further adjusted at this time. For example, if the user wants to press a switch, the second knuckle of the thumb should be placed on the switch, that is, the actual force point should be the second knuckle of the thumb. However, due to the user's hand position error, the first knuckle of the thumb is pressed on the switch, that is, the actual force point is actually the first knuckle of the thumb. At this time, although the target knuckle will also have pressure, and the corresponding real-time pressure may reach the preset pressure, it is still difficult for the user to exert force at this time, so it is necessary to adjust the position of the thumb so that the second knuckle is pressed on the switch.

[0088] Specifically, the manipulator can be further adjusted by adjusting the reference force point of the manipulator to the actual force point, so that the force is in line with the user's usage habits. Alternatively, the manipulator can be further adjusted by adjusting the conventional force point of the manipulator to the actual force point, so that the force during use is in line with healthy usage habits, reducing the damage to the hands caused by differences in personal habits of users.

[0089] Specifically, the third adjustment content also includes: before adjusting the conventional force point of the manipulator to the position where the actual force point is located, determining whether the actual force point is the conventional force point, so as to adjust the conventional force point of the manipulator to the position where the actual force point is located when the actual force point is not the conventional force point. In this example, by determining whether the actual force point is the conventional force point before adjusting the conventional force point of the manipulator to the position where the actual force point is located, it can be directly determined that the manipulator has been adjusted to the target hand shape when the actual force point is already the conventional force point, so that the user can perform daily activities based on the target hand shape, thereby avoiding the situation where the manipulator recognizes the adjustment error due to the lack of adjustment, thereby improving the accuracy and reliability of the adjustment.

[0090] Specifically, the reference force point is determined by the following steps: determining the movement direction of the manipulator according to the reference hand shape and the target hand shape; determining the reference force point of the target knuckle in the manipulator according to the movement direction. In this example, the movement direction is used to indicate the pressing direction of the target knuckle of the manipulator, that is, the moving direction toward the target object or the contact knuckle. Determining the reference force point of the target knuckle according to the movement direction is to determine the contact position between the target knuckle and the target object according to the pressing direction, thereby determining the reference force point of the target knuckle. The reference force point determined in this way has a high degree of adaptability to the content corresponding to the target image information, which can improve the accuracy of data processing by the server.

[0091] It can be seen that in this example, when the real-time pressure of the manipulator is greater than the preset pressure, it is determined whether the actual force point of the manipulator is the reference force point, and when the actual force point is not the reference force point, the conventional force point of the manipulator is adjusted to the position of the actual force point. This can solve the problem of force errors caused by deviations caused by the movement of the target object during the adjustment process in actual use, avoid the execution of daily activities due to force errors, and avoid further damage to the user's hand due to force errors, which is conducive to the accuracy of the hand shape corresponding to the actions adjusted by the manipulator for performing daily activities.

[0092] In a possible example, controlling the display screen of the user device to jump to display a hand shape determination interface according to the image type includes: when it is determined that the image type is a dynamic image, determining a plurality of target states corresponding to changes in the target hand shapes according to the target image information; determining display information according to the plurality of target hand shapes and the target states corresponding to changes in the target hand shapes, the display information being the display content of the hand shape determination interface; and controlling the display screen of the user device to jump to display the hand shape determination interface.

[0093] Among them, the target state refers to the time state or the target object state, etc. The time state is used to characterize the time nodes corresponding to the conversion of each target hand shape. The target object state can be represented by the pressure value change and / or force position change detected by the manipulator in real time. For example, if the manipulator detects that the maximum force position has changed from the first joint of the thumb to the second joint of the index finger, it indicates that the state of the target object has changed, and it can be adjusted to the next target hand shape. Specifically, when the dynamic image is a plurality of pictures, the hand shape adjustment order of the plurality of target hand shapes can be determined according to the order of the uploaded pictures. When the dynamic image is a video, the hand shape adjustment order of the plurality of target hand shapes can be determined according to the order of the time nodes corresponding to the plurality of target hand shapes in the video.

[0094] Among them, the displayed information includes the hand type corresponding to each target hand shape, the target state, the knuckles touching the target object (or the knuckles touching each other when there is no target object), or the joint angle adjustment information corresponding to the target hand shape, etc., and no further restrictions are made here.

[0095] In a specific implementation, after receiving the target image information, the server may first determine the image type of the target image information. If the image type is a static image, the single target hand shape determined according to the target image information may be directly determined as the display content of the hand shape determination interface. If the image type is a dynamic image, multiple target hand shapes may be determined according to the target image information, and the target state corresponding to each target hand shape may be determined according to the target image information, thereby determining the display information according to the multiple target hand shapes and the corresponding target states, thereby controlling the display screen of the user device to jump to display the hand shape determination interface.

[0096] For example, see Figure 4 and Figure 6 If the image content is uploaded through the action acquisition interface 410 and it is confirmed that the image content is multiple pictures, the server controls the user device to jump to the action execution interface, such as Figure 6 The interface 610 shown in (6a) or Figure 6 Interface 620 shown in (6b) in FIG. Figure 6 The interface 610 shown in (6a) includes a first picture display control 611 and a first picture confirmation control 612. The first picture display control 611 is used to display multiple pictures according to the order in which the pictures are uploaded. By selecting the first picture confirmation control 612, the user device can upload an image confirmation message to the server. Figure 6 The interface 620 shown in (6b) includes a second picture display control 621, a picture adjustment control 622, and a second picture confirmation control 623. The second picture display control 621 is used to display pictures. The user can perform a selection operation on the picture adjustment control 622 to switch the pictures displayed by the user device according to the picture upload sequence. The user can perform a selection operation on the second picture confirmation control 623 to enable the user device to upload an image confirmation message to the server. In this example, the user can perform a selection operation on the displayed picture to enlarge the displayed image.

[0097] Or, for example, see Figure 7 , combined with reference to Figure 4 and Figure 7 If the image content is a video after uploading it through the action acquisition interface 410, the server can control the user device to jump to the action execution interface, such as Figure 7The interface 710 shown includes a video display control 711, a play control 712, and a video confirmation control 713. The video display control 711 is used to display the video uploaded by the robot, and the user can control the start and stop of the video playback by performing a selection operation on the play control 712, and the user can perform a selection operation on the video confirmation control 713 to enable the user device to upload an image confirmation message to the server.

[0098] For example, referring to FIG8 , when the target image information is a dynamic image, the hand shape confirmation interface is Figure 5 Based on the above, content adjustment controls can be further included, such as Figure 8 The content adjustment control 811 in the illustrated interface 810. The user performs a selection operation on the content adjustment control 811 to switch the target hand shape displayed by the user device according to the hand shape adjustment sequence.

[0099] It can be seen that in this example, by determining that there are multiple target hand shapes when the image type is determined to be a dynamic image; determining the target states corresponding to the changes of multiple target hand shapes based on the target image information; and determining the display information based on the multiple target hand shapes and the target states corresponding to the changes of multiple target hand shapes, the display information is the display content of the hand shape determination interface, which can enrich the data type processed by the server, improve the data processing capability of the server, and improve the functionality of the software applications corresponding to the manipulator, so that the manipulator can more accurately assist users in performing daily activities.

[0100] The electronic device 90 in the present application may be structured as follows: Fig. 9 As shown, the electronic device 90 is a server, which can be used to execute the above method. Specifically, the electronic device 90 may include a processor 910, a memory 920, a communication interface 930, and one or more programs 921, wherein the one or more programs 921 are stored in the above memory 920 and are configured to be executed by the above processor 910, and the one or more programs 921 include instructions for executing any step in the above method embodiment.

[0101] Among them, the communication interface 930 is used to support the communication between the electronic device 90 and other devices. The processor 910 can be, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, units and circuits described in conjunction with the disclosure of the embodiments of the present application. The processor can also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0102] The memory 920 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory are available, such as static random access memory (Static RAM, SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synch Link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DRRAM).

[0103] In a specific implementation, the processor 910 is used to execute any step in the above method embodiment, and when performing data transmission such as sending, the communication interface 930 can be selectively called to complete the corresponding operation.

[0104] It should be noted that the structural diagram of the above-mentioned electronic device 90 is only an example, and the specific components included may be more or less, and no sole limitation is made here.

[0105] The present application can divide the functional units of the electronic device according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0106] Fig.10 1 is a functional unit block diagram of a first data processing device for a manipulator provided in an embodiment of the present application. The first data processing device 1000 for a manipulator includes:

[0107] The control unit 1010 is used to control the display screen of the user device to display an action execution interface, wherein the action execution interface includes target image information, wherein the target image information is the image content uploaded by the manipulator in response to the imitation instruction, wherein the image content includes the hand shape information of the user's hand without wearing the manipulator, and the imitation instruction is the content sent by the user device to the manipulator, and the imitation instruction is used to instruct the manipulator to collect the image content;

[0108] The first responding unit 1020 is used to respond to the image confirmation message uploaded by the user device, determine the image type of the target image information, and control the display screen of the user device to jump to display a hand shape determination interface according to the image type, wherein the hand shape determination interface is used to display the target hand shape, the target hand shape is determined by analyzing the target image information, and the image type includes a static image and a dynamic image; when the image type is a dynamic image, there are multiple target hand shapes;

[0109] The second response unit 1030 is used to send adjustment information to the user device in response to the hand shape confirmation message uploaded by the user device, and the adjustment information is used to instruct the user device to send an adjustment instruction to the manipulator, and the adjustment instruction is used to instruct the manipulator to adjust to the target hand shape according to the control scheme, and the control scheme includes control content for adjusting the manipulator from the current hand shape to the target hand shape.

[0110] In one possible example, the adjustment instruction includes a first adjustment content and a second adjustment content; the first adjustment content is used to instruct the manipulator to adjust to a reference hand shape, and the reference hand shape and the target hand shape are of the same hand type; the second adjustment content is used to instruct the manipulator to continuously increase the adjustment angle after adjusting to the reference hand shape until the real-time pressure of the manipulator is greater than the preset pressure, so as to adjust to the target hand shape.

[0111] In one possible example, the real-time pressure of the manipulator is the real-time pressure of the target knuckles in the manipulator, and the target knuckles are determined by the following steps: determining whether there is a target object based on the target image information; if so, determining the knuckles in contact with the target object as the target knuckles; if not, determining preset knuckles as the target knuckles, and the preset knuckles are the knuckles in contact with each other in the target hand shape.

[0112] In a possible example, the adjustment instruction also includes a third adjustment content, and the third adjustment content includes: when the real-time pressure of the manipulator is greater than the preset pressure, determining whether the actual force point of the manipulator is a reference force point, and when the actual force point is not the reference force point, adjusting the conventional force point of the manipulator to the position of the actual force point, and the conventional force point is the force point corresponding to the target hand shape.

[0113] In a possible example, the third adjustment content also includes: before adjusting the conventional force point of the manipulator to the position of the actual force point, determining whether the actual force point is the conventional force point, so as to adjust the conventional force point of the manipulator to the position of the actual force point when the actual force point is not the conventional force point.

[0114] In a possible example, the reference force point is determined by the following steps: determining the movement direction of the manipulator according to the reference hand shape and the target hand shape; and determining the reference force point of the target knuckle in the manipulator according to the movement direction.

[0115] In one possible example, in terms of controlling the display screen of the user device to jump to display a hand shape determination interface according to the image type, the first response unit is specifically used to: when it is determined that the image type is a dynamic image, determine a plurality of target states corresponding to changes in the target hand shapes according to the target image information; determine display information according to the plurality of target hand shapes and the target states corresponding to changes in the target hand shapes, the display information being the display content of the hand shape determination interface; and control the display screen of the user device to jump to display the hand shape determination interface.

[0116] In the case of using an integrated unit, the functional unit composition block diagram of the second data processing device for a manipulator provided in an embodiment of the present application is as follows: Fig.11 As shown. Fig.11 In the embodiment, the second data processing device 1100 for the robot comprises: a processing module 1120 and a communication module 1110. The processing module 1120 is used to control and manage the actions of the first data processing device 1000 for the robot, for example, the steps executed by the control unit 1010, the first response unit 1020 and the second response unit 1030, and / or other processes for executing the technology described herein. The communication module 1110 is used to support the interaction between the second data processing device 1100 for the robot and other devices. Fig.11 As shown, the second data processing device 1100 for a robot may further include a storage module 1130 , and the storage module 1130 is used to store program codes and data of the second data processing device 1100 for a robot.

[0117] Among them, the processing module 1120 can be a processor or a controller, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the embodiments of the present application. The processor can also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication module 1110 can be a transceiver, an RF circuit or a communication interface, and the like. The storage module 1130 can be a memory.

[0118] Among them, all relevant contents of each scenario involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here. The above first data processing device for the manipulator or the second data processing device for the manipulator can execute the above Figure 3 The steps performed by the server in the data processing method for a robot are shown.

[0119] An embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps of any method recorded in the above method embodiments, and the above computer includes a server.

[0120] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0121] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0122] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0123] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0125] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the above-mentioned methods of each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk, etc., and other media that can store program codes.

[0126] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the relevant hardware through a program, and the program may be stored in a computer-readable memory, which may include a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0127] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data processing method for a robot, characterized in that: A server applied to a hand function training system, the hand function training system comprising the server, a user device and a manipulator, the server being communicatively connected to the user device, the user device being communicatively connected to the manipulator, the method comprising: Controlling the display screen of the user device to display an action execution interface, wherein the action execution interface includes target image information, wherein the target image information is image content uploaded by the manipulator in response to an imitation instruction, wherein the image content includes hand shape information of a user's hand without the manipulator, wherein the imitation instruction is content sent from the user device to the manipulator, and wherein the imitation instruction is used to instruct the manipulator to capture the image content; In response to the image confirmation message uploaded by the user device, determine the image type of the target image information, and control the display screen of the user device to jump to display a hand shape determination interface according to the image type, wherein the hand shape determination interface is used to display the target hand shape, the target hand shape is determined by analyzing the target image information, and the image type includes a static image and a dynamic image; when the image type is a dynamic image, there are multiple target hand shapes; In response to a hand shape confirmation message uploaded by the user device, adjustment information is sent to the user device, the adjustment information is used to instruct the user device to send an adjustment instruction to the manipulator, the adjustment instruction includes a first adjustment content and a second adjustment content; the first adjustment content is used to instruct the manipulator to adjust to a reference hand shape, and the reference hand shape and the target hand shape are of the same hand type; the second adjustment content is used to instruct the manipulator to continuously increase the adjustment angle after adjusting to the reference hand shape until the real-time pressure of the manipulator is greater than the preset pressure, so as to adjust to the target hand shape.

2. The method according to claim 1, characterized in that The real-time pressure of the manipulator is the real-time pressure of the target knuckle of the manipulator, and the target knuckle is determined by the following steps: Determining whether a target object exists according to the target image information; If so, the knuckle in contact with the target object is determined as the target knuckle; If not present, a preset knuckle is determined as the target knuckle, and the preset knuckle is the knuckle that contacts each other in the target hand shape.

3. The method according to claim 1, characterized in that The adjustment instruction also includes a third adjustment content, and the third adjustment content includes: when the real-time pressure of the manipulator is greater than the preset pressure, determining whether the actual force point of the manipulator is a reference force point, and when the actual force point is not the reference force point, adjusting the conventional force point of the manipulator to the position of the actual force point, and the conventional force point is the force point corresponding to the target hand shape.

4. The method according to claim 3, characterized in that The third adjustment content also includes: before adjusting the conventional force point of the manipulator to the position of the actual force point, determining whether the actual force point is the conventional force point, so as to adjust the conventional force point of the manipulator to the position of the actual force point when the actual force point is not the conventional force point.

5. The method according to claim 3, characterized in that The reference force point is determined by the following steps: Determining the movement direction of the manipulator according to the reference hand shape and the target hand shape; A reference force point of a target knuckle in the manipulator is determined according to the movement direction.

6. The method according to claim 1, characterized in that The step of controlling the display screen of the user device to jump to display the hand shape determination interface according to the image type includes: When it is determined that the image type is a dynamic image, determining target states corresponding to a plurality of target hand shape changes according to the target image information; Determine display information according to the target hand shapes and the target states corresponding to the target hand shapes when they are changed, wherein the display information is the display content of the hand shape determination interface; Control the display screen of the user equipment to jump to display the hand shape determination interface.

7. A data processing device for a manipulator, characterized in that: A server applied to a hand function training system, the hand function training system comprising the server, a user device and a manipulator, the server being communicatively connected to the user device, the user device being communicatively connected to the manipulator, and the data processing device for the manipulator comprising: a control unit, configured to control the display screen of the user device to display an action execution interface, wherein the action execution interface includes target image information, wherein the target image information is image content uploaded by the manipulator in response to an imitation instruction, wherein the image content includes hand shape information of a user's hand without the manipulator, and wherein the imitation instruction is content sent from the user device to the manipulator, and wherein the imitation instruction is used to instruct the manipulator to capture the image content; A first response unit is used to respond to the image confirmation message uploaded by the user device, determine the image type of the target image information, and control the display screen of the user device to jump to display a hand shape determination interface according to the image type, wherein the hand shape determination interface is used to display the target hand shape, the target hand shape is determined by analyzing the target image information, and the image type includes a static image and a dynamic image; when the image type is a dynamic image, there are multiple target hand shapes; A second response unit is used to send adjustment information to the user device in response to a hand shape confirmation message uploaded by the user device, wherein the adjustment information is used to instruct the user device to send an adjustment instruction to the manipulator, and the adjustment instruction includes a first adjustment content and a second adjustment content; the first adjustment content is used to instruct the manipulator to adjust to a reference hand shape, and the reference hand shape and the target hand shape are of the same hand type; the second adjustment content is used to instruct the manipulator to continuously increase the adjustment angle after adjusting to the reference hand shape until the real-time pressure of the manipulator is greater than the preset pressure, so as to adjust to the target hand shape.

8. An electronic device, characterized in that: The method comprises a processor, a memory, a communication interface and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for executing the steps in the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: A computer program for electronic data exchange is stored, wherein the computer program enables a computer to execute the steps in the method according to any one of claims 1 to 6.

Images