Massage information generation method and device, electronic equipment and storage medium

Abstract

The invention provides a massage information generation method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: identifying a target hand diagnosis gesture corresponding to at least one hand in a hand diagnosis video stream, wherein the hand diagnosis video stream comprises a shot hand diagnosis gesture acting on a target area; a target hand diagnosis gesture corresponding to the at least one hand is recorded, a hand diagnosis result is generated, and the hand diagnosis result is used for indicating a target massage position and / or a target massage path of the target area; according to the method, the massage position can be determined through the hand diagnosis operation, the method is not limited to acupuncture points recognized by a massage robot any more, personalized massage is conveniently achieved, and the applicability is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a massage information generation method and device, electronic equipment and a storage medium. BACKGROUND

[0002] A massage robot is a robot that simulates the techniques of a professional masseur to provide massage services for users. The massage robot can relieve body pain, muscle tension and promote blood circulation, and is suitable for people of different ages.

[0003] In related technologies, when a massage robot is used for massage, the massage robot needs to identify the acupoints of the human body first, and then massage the acupoints.

[0004] The above massage scheme mainly depends on the acupoints identified by the massage robot, and it is difficult to achieve personalized massage for non-acupoint areas. SUMMARY

[0005] To address the above problems in the related art, the present application provides a massage information generation method and device, electronic equipment and a storage medium, which can conveniently achieve personalized massage.

[0006] In a first aspect, the present application provides a massage information generation method, comprising:

[0007] identifying a target hand diagnosis gesture corresponding to at least one hand in a hand diagnosis video stream, the hand diagnosis video stream comprising a hand diagnosis gesture applied to a target area;

[0008] recording the target hand diagnosis gesture corresponding to the at least one hand to generate a hand diagnosis result, the hand diagnosis result being used to indicate a target massage position and / or a target massage path of the target area;

[0009] generating massage information according to the hand diagnosis result.

[0010] In a second aspect, the present application provides a massage information generation device, comprising:

[0011] an identification module configured to identify a target hand diagnosis gesture corresponding to at least one hand in a hand diagnosis video stream, the hand diagnosis video stream comprising a hand diagnosis gesture applied to a target area;

[0012] a recording module configured to record the target hand diagnosis gesture corresponding to the at least one hand to generate a hand diagnosis result, the hand diagnosis result being used to indicate a target massage position and / or a target massage path of the target area;

[0013] a generation module configured to generate massage information according to the hand diagnosis result.

[0014] In a third aspect, an electronic device is provided, and the electronic device includes a processor, a storage medium, and a bus. The storage medium stores machine readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium through the bus. The processor executes the machine readable instructions to perform the steps of the massage information generation method in the first aspect.

[0015] In a fourth aspect, a storage medium is provided. The storage medium stores a computer program. When the computer program is run by a processor, the steps of the massage information generation method in the first aspect are performed.

[0016] The massage information generation method, device, electronic device, and storage medium provided in the embodiments of the present application have the following beneficial effects:

[0017] The massage information generation method, device, electronic device, and storage medium provided in the embodiments of the present application have the following beneficial effects: BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be considered as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0019] Figure 1 A flowchart of a massage information generation method provided in an embodiment of the present application is shown in the figure.

[0020] Figure 2 A flowchart of another massage information generation method provided in an embodiment of the present application is shown in the figure.

[0021] Figure 3 A distribution diagram of key points in a left hand provided in an embodiment of the present application is shown in the figure.

[0022] Figure 4 A flowchart of another massage information generation method provided in an embodiment of the present application is shown in the figure.

[0023] Figure 5 A flowchart of another massage information generation method provided in an embodiment of the present application is shown in the figure.

[0024] Figure 6 A flowchart of another massage information generation method provided by an embodiment of the present application is shown in FIG. 6;

[0025] Figure 7 A state transition diagram provided by an embodiment of the present application is shown in FIG. 7;

[0026] Figure 8 A hand diagnosis gesture diagram provided by an embodiment of the present application is shown in FIG. 8;

[0027] Figure 9 A flowchart of another massage information generation method provided by an embodiment of the present application is shown in FIG. 6;

[0028] Figure 10 A hand diagnosis gesture diagram provided by an embodiment of the present application is shown in FIG. 8;

[0029] Figure 11 A massage scheme diagram provided by an embodiment of the present application is shown in FIG. 9;

[0030] Figure 12 A functional module diagram of a massage information generation device provided by an embodiment of the present application is shown in FIG. 10;

[0031] Figure 13 An electronic device structure diagram provided by an embodiment of the present application is shown in FIG. 11. DETAILED DESCRIPTION

[0032] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, but not all embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0033] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0034] It should be noted that: similar reference numbers and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0035] In the related art, in the process of performing massage by using a massage robot, sometimes the part to be massaged cannot be recognized in a fully automatic manner, for example, nodules on the body cannot be detected by a visual algorithm, and for example, for some fat people, some bones (for example, shoulder blades) are difficult to identify by vision.

[0036] Therefore, the embodiment of the present application provides a massage information generation method, which can determine the massage position through hand diagnosis operation, so as to no longer be limited to the acupoints recognized by the massage robot, conveniently realize personalized massage, and improve the applicability.

[0037] Figure 1 A flowchart of a massage information generation method provided by the embodiment of the present application is shown, the method can be applied to a massage scene, and the execution subject can be a computer, a server, a processor or the like. The electronic device can interact with the massage robot to obtain a hand diagnosis video stream, and send the generated massage information to the massage robot. Of course, in some embodiments, the method can also be applied to the massage robot, and the massage robot can perform corresponding massage operation according to the obtained massage information.

[0038] In order to better understand the present application, the execution subject is taken as the massage robot as an example for description, wherein the massage robot can include a processing unit, a mechanical arm, a massage head and a camera, wherein the massage head and the camera are respectively arranged on the mechanical arm, and the mechanical arm, the massage head and the camera are respectively electrically connected with the processing unit. The processing unit can control the mechanical arm to drive the massage head and the camera to move according to a mechanical arm control instruction. When the movement is controlled to a target position, the processing unit can send a shooting instruction to the camera to shoot a hand diagnosis gesture acting on a target area to generate a hand diagnosis video stream. The massage information generation method based on the present application can generate a massage scheme after obtaining the massage information.

[0039] As shown in the figure, the massage information generation method includes: Figure 1

[0040] S101, identifying a target hand diagnosis gesture corresponding to at least one hand in a hand diagnosis video stream, the hand diagnosis video stream including a hand diagnosis gesture acting on a target area.

[0041] Optionally, the hand diagnosis video stream can be generated by a camera in the massage robot to shoot a hand diagnosis gesture acting on a target area of a human body.

[0042] The hand diagnosis gesture is an indication gesture of a hand diagnosis user using a specified gesture action for a target massage position and / or a target massage path of the target area.

[0043] ​Optionally, the specified gesture action can include, but is not limited to, a C-shaped gesture (i.e., the thumb and the index finger are bent towards each other), a V-shaped gesture (i.e., the victory gesture, in which the index finger and the middle finger form a V shape), a single-finger line-drawing gesture, a multi-finger line-drawing gesture (e.g., the index finger and the middle finger are drawn together), and the like, without limitation. In some embodiments, the C-shaped gesture and the V-shaped gesture can be used to indicate a target massage position of the target region, and the single-finger line-drawing gesture and the multi-finger line-drawing gesture can be used to indicate a target massage path of the target region.

[0044] Optionally, the target massage position can be determined according to the position of the palm diagnosis gesture. Optionally, the target massage path can be determined according to the path of the palm diagnosis gesture.

[0045] In the hand diagnosis video stream, each video frame can include one or more hands, such as the left hand and the right hand of the palm diagnosis user, or any one hand of the palm diagnosis user.

[0046] In some embodiments, when performing identification, a preset identification algorithm can be used to achieve the identification function. Optionally, the preset identification algorithm can be implemented based on a deep learning algorithm, a neural network model, and the like, without limitation.

[0047] S102, record the target palm diagnosis gesture corresponding to the at least one hand, and generate a palm diagnosis result. The palm diagnosis result is used to indicate a target massage position and / or a target massage path of the target region.

[0048] The palm diagnosis result can indicate the relevant information of the target region, and specifically, can indicate the target massage position and / or the target massage path of the target region.

[0049] Optionally, the target massage position can be any region position in the target region, and the application does not limit the size of the target massage position region. The target massage path can be any path line in the target region. In some embodiments, the target massage path can be a straight path or a curved path, without limitation.

[0050] Based on the above description, after identifying each target palm diagnosis gesture, a palm diagnosis sequence can be recorded. According to the palm diagnosis sequence, a palm diagnosis result can be further generated. Optionally, the palm diagnosis sequence can include a plurality of target palm diagnosis gestures, which can indicate a change sequence of the target palm diagnosis gestures.

[0051] S103, generate massage information according to the palm diagnosis result.

[0052] The massage information can be used to indicate a massage scheme corresponding to the target region. Alternatively, when the massage information is generated according to the hand diagnosis result, the massage information can be generated based on a preset massage rule and in combination with the hand diagnosis result, and then the target region can be comprehensively massaged based on the massage information, so that the massage is no longer limited to the acupoints recognized by the massage robot, and personalized massage is conveniently realized, and the applicability is improved.

[0053] Alternatively, the preset massage rule can indicate a massage sequence of a plurality of target massage positions, a massage starting point and a massage ending point of a target massage path, a massage sequence of the target massage positions and the target massage path, and the like, without limitation.

[0054] It should be noted that in some embodiments, the present application can also be used in cooperation with an acupoint massage method. It can be understood that the method provided by the present application can be applied to massage of any target region, so that the massage is no longer limited to the acupoints recognized by the massage robot, and personalized massage is conveniently realized, and the applicability is improved. In addition, compared with the existing massage robot, no new hardware needs to be introduced, and the cost is low.

[0055] In summary, the embodiment of the present application provides a massage information generation method, which comprises: identifying at least one hand gesture corresponding to a target hand gesture of a hand in a hand diagnosis video stream, the hand diagnosis video stream comprising a hand gesture applied to a target region; recording the target hand gesture corresponding to the at least one hand, generating a hand diagnosis result, the hand diagnosis result indicating the target massage position and / or the target massage path of the target region; and generating massage information according to the hand diagnosis result. The application of the method can determine the massage position through hand diagnosis operation, so that the massage is no longer limited to the acupoints recognized by the massage robot, and personalized massage is conveniently realized, and the applicability is improved.

[0056] Alternatively, the recording of the target hand gesture corresponding to the at least one hand and the generation of the hand diagnosis result comprise: recording the target hand gesture corresponding to the at least one hand based on a preset conversion relationship, and generating the hand diagnosis result.

[0057] The preset conversion relationship comprises a state conversion relationship between the hand gestures, and a state conversion relationship between a hand diagnosis recording state and a non-recording state.

[0058] It can be understood that different hand gestures can be used for different target regions during hand diagnosis, and the hand gestures can be converted to each other. In addition, it should be noted that each hand gesture can be a static gesture, or can be a dynamic gesture, without limitation. Of course, the hand gesture can also be converted from a non-hand diagnosis gesture, that is, the hand gesture can be converted from any gesture.

[0059] Wherein, after obtaining the target hand diagnosis gesture corresponding to at least one hand in each video frame, it is determined whether the target hand diagnosis gesture needs to be recorded according to a preset conversion relationship, and if so, the target hand diagnosis gesture is recorded to generate a hand diagnosis result. Alternatively, if not, the target hand diagnosis gesture can also be stored in a preset location for subsequent use according to the actual application scenario.

[0060] It should be noted that, in order to accurately record each hand, a preset conversion relationship corresponding to each hand can be set, so that when recording each hand based on the preset conversion relationship corresponding to each hand, misrecording can be avoided.

[0061] Alternatively, the above-mentioned identifying a target hand diagnosis gesture corresponding to at least one hand in the hand diagnosis video stream comprises: using a hand diagnosis gesture recognition model to identify a target hand diagnosis gesture corresponding to at least one hand in the hand diagnosis video stream, wherein the hand diagnosis gesture recognition model is obtained by training a training sample set, and the training sample set comprises a plurality of training samples, and each training sample is labeled with a hand diagnosis gesture.

[0062] Wherein, the specified gesture action corresponding to the hand diagnosis gesture labeled by each training sample can include but is not limited to: C-shaped gesture, V-shaped gesture, single-finger line drawing gesture, multi-finger line drawing gesture, etc., without limitation.

[0063] Alternatively, the training sample set can include a plurality of sample video streams, each sample video stream corresponds to a training sample, and each sample video stream includes a hand diagnosis gesture acting on a sample region, and the sample region can be a human body region, such as a human back region, a leg region, a head region, a face region, etc., without limitation.

[0064] By training, the hand diagnosis gesture recognition model can learn the feature information of the hand diagnosis gesture of each training sample in the training sample set, and be used to identify a target hand diagnosis gesture corresponding to at least one hand in a hand diagnosis video stream, so as to realize rapid construction of the hand diagnosis gesture recognition model.

[0065] Figure 2 Another flowchart of a massage information generation method provided by an embodiment of the present application is shown in Figure 3 A distribution diagram of key points in a left hand provided by an embodiment of the present application is shown in Figure 2 As shown in the above-mentioned identifying a target hand diagnosis gesture corresponding to at least one hand in the hand diagnosis video stream, comprising:

[0066] S201, determining at least one hand region in each video frame of a hand diagnosis video stream, and identifying the key point positions of each hand region in each video frame.

[0067] In some embodiments, since each video frame can include multiple hand regions, the hand regions corresponding to the same hand in each video frame can be tracked first, and then the key point positions in the hand regions corresponding to the same hand are identified.

[0068] Optionally, as shown in Figure 3 The key points of each hand region can include 21 finger tips and knuckle joints.

[0069] It should be noted that each video frame can include hand regions corresponding to the same hand, and can also include hand regions corresponding to different hands. For example, a certain hand diagnosis video stream includes 30 frames. Optionally, the first 10 frames include hand regions corresponding to the left and right hands of a hand diagnosis user, and the last 20 frames include hand regions corresponding to the left hand of the hand diagnosis user, which is not limited herein.

[0070] S202, according to the key point positions of each hand region in each video frame and the previous result queue corresponding to each hand in each video frame, identify the target hand diagnosis gesture corresponding to each hand in each video frame.

[0071] The previous result queue corresponding to each hand in each video frame includes an initial hand diagnosis gesture corresponding to each hand and the key point positions of each hand.

[0072] It should be noted that different hands can correspond to different result queues. For example, a first hand can correspond to a first result queue, and a second hand can correspond to a second result queue.

[0073] Optionally, the initial hand diagnosis gesture corresponding to each hand can be a line drawing hand diagnosis gesture or a region range indication gesture. The region range indication gesture can be used to indicate the diameter or boundary of a target region. For example, it can be used to indicate a nodule in a human body, and the corresponding target region can correspond to a circular region. The line drawing hand diagnosis gesture can be used to indicate the direction of a bone or a meridian, and the corresponding target region can correspond to a curve, which is not limited herein.

[0074] When identifying the target hand diagnosis gesture corresponding to each hand in each video frame, the previous result queue corresponding to each hand in the video frame and the key point positions of each hand in the video frame can be comprehensively judged, which can improve the accuracy of judgment compared with the judgment mode using only the key point positions of each hand in the video frame.

[0075] It should be noted that for the first frame in the hand diagnosis video stream, since there is no previous video frame, the previous result queue corresponding thereto is empty.

[0076] Optionally, the determining at least one hand region in each video frame of the hand diagnosis video stream, and identifying the key point position of each hand in each video frame based on the at least one hand region, comprises:

[0077] Optionally, the bounding box of the at least one hand region in each video frame of the hand diagnosis video stream is identified, and the at least one hand in each video frame is tracked and determined according to the bounding box of the at least one hand region in each video frame, and the key point position of each hand in each video frame is identified by using the hand key point detection model.

[0078] Optionally, the bounding box can be identified by using a target detection model, and in some embodiments, the target detection model can be constructed based on a detection network such as yolo (you only look once), SSD (Single Shot MultiBox Detector), Fast R-CNN, nanodet, etc., which is not limited herein. It should be noted that if a video frame includes multiple hand regions, the bounding box of all hand regions can be identified by using the target detection model.

[0079] In some embodiments, a hand tracking algorithm can be used to identify the at least one hand in each video frame. Optionally, the hand tracking algorithm can be set based on a deep learning algorithm, or can be set based on the Intersection Over Union (IOU) of the bounding box of the hand region or the hand key point bounding box, which is not limited herein. The hand key point detection model can be constructed based on an algorithm such as hrnet (High-Resolution Network), vitpose (visiontransformer baseline), Mediapipe, etc., which is not limited herein.

[0080] In some embodiments, the target detection model can be trained and obtained according to a first sample set, and the first sample set includes a plurality of first hand diagnosis video streams, and the first video frame in each first hand diagnosis video stream is labeled with a bounding box of a hand region. The hand key point detection model can be trained and obtained according to a second sample set, and the second sample set can include a plurality of second video frames, and the hand in each second video frame can be labeled with a key point position. Optionally, the second sample set can be obtained from the first hand diagnosis video stream, that is, the second video frame can be partially the same as the first video frame in the first hand diagnosis video stream, so as to improve the sample collection efficiency.

[0081] When tracking based on the bounding box of the hand region, the same hand in each video frame can be marked based on difference operation of the bounding box of the hand region. For the hand region corresponding to the same hand determined by tracking, the key point positions of each hand can be further identified by using the hand key point detection model. Alternatively, the key point positions of each hand can be determined by the image positions of the key points of each hand in the video frame.

[0082] Figure 4 A flowchart of another massage information generation method provided by an embodiment of the present application is shown in FIG. 6. Alternatively, as shown in FIG. 6, the above-mentioned identifying the target hand diagnosis gesture corresponding to each hand in each video frame according to the key point positions of each hand in each video frame and the previous result queue corresponding to each hand in each video frame includes: Figure 4

[0083] S301, identifying an initial hand diagnosis gesture corresponding to a target hand in a target video frame according to the key point positions of the target hand in the target video frame.

[0084] Alternatively, the target video frame can be any video frame in the hand diagnosis video stream, and the target hand can be any hand in the target video frame, which is not limited herein.

[0085] In some embodiments, the hand diagnosis gesture recognition model can be constructed based on a deep learning algorithm. Alternatively, the network structure of the constructed hand diagnosis gesture recognition model can sequentially include a first full connection layer, a first batch normalization (BN) layer, a first activation layer, a first Dropout layer, a second full connection layer, a second batch normalization (BN) layer, a second activation layer, a second Dropout layer (when training), a third full connection layer, and a softmax.

[0086] ​In the identification process, a 34-dimensional vector corresponding to the key point position of the target hand in the target video frame can be input into the first full connection layer, the 34-dimensional vector input is mapped to a higher dimensional space through the first full connection layer, and a nonlinear feature is introduced; the first batch normalization BN layer is used to perform batch normalization on the output of the first full connection layer, which helps to speed up the training process and reduce the gradient vanishing problem; the first activation layer is used to introduce nonlinearity, such as a ReLU activation function, to increase the expression ability of the network; the first Dropout layer is used to randomly discard neurons with a certain probability during the training process to prevent overfitting; the second full connection layer is used for linear transformation and nonlinear feature extraction again; the second batch normalization BN layer is used to perform batch normalization again on the output of the second full connection layer; the second activation layer is used to introduce nonlinearity again; the second Dropout layer is used to randomly discard neurons with a certain probability during the training process to prevent overfitting again; the third full connection layer is used to obtain the final feature representation; and the softmax is used to convert the final feature representation into a probability distribution of each hand diagnosis gesture category, and finally the hand diagnosis gesture type corresponding to the highest probability value is selected as the initial hand diagnosis gesture corresponding to the target hand in the target video frame.

[0087] By inputting the key point position of the target hand in the target video frame into the hand diagnosis gesture recognition model, the initial hand diagnosis gesture corresponding to the target hand in the target video frame can be identified according to the embodiments of the present application.

[0088] Optionally, the initial hand diagnosis gesture corresponding to the target hand in the target video frame can be a line drawing hand diagnosis gesture, a region range indication gesture, etc., which is not limited herein.

[0089] S302, generating a current gesture result according to the initial hand diagnosis gesture corresponding to the target hand in the target video frame and the key point position of the target hand.

[0090] S303, adding the current gesture result to a previous result queue corresponding to the target hand to obtain a target result queue corresponding to the target hand.

[0091] S304, determining a target hand diagnosis gesture corresponding to the target hand in the target video frame according to the target result queue corresponding to the target hand.

[0092] Based on the generation content of the current gesture result, it can be seen that the current gesture result not only stores the initial hand diagnosis gesture corresponding to the target hand in the target video frame, but also stores the key point position of the target hand. The previous result queue corresponding to the target hand includes the previous gesture result corresponding to the target hand in each previous video frame of the target video frame, that is, includes the initial hand diagnosis gesture corresponding to the target hand in each previous video frame of the target video frame and the key point position of the target hand.

[0093] Wherein, after obtaining the current gesture result, it can be added to the tail of the previous result queue corresponding to the target hand, so as to obtain the target result queue corresponding to the target hand, and then according to the target result queue corresponding to the target hand, the target hand gesture corresponding to the target hand can be determined, and the initial hand diagnosis gesture corresponding to the target hand in the previous video frame can be comprehensively considered, so that the static gesture and the logical motion can be combined, which can be applied to the recognition of dynamic gestures, and the accuracy of recognition is also improved.

[0094] It should be noted that the hand diagnosis gestures corresponding to each hand in each video frame in the hand diagnosis video stream can be determined according to the processes of steps S301 to S304, which will not be described here.

[0095] Figure 5 The flowchart of another massage information generation method provided by the embodiment of the present application. Optionally, the target hand gesture corresponding to the target hand in the target video frame is determined according to the target result queue corresponding to the target hand, including:

[0096] S401, calculate the gesture class proportion of the gesture class of the initial hand diagnosis gesture corresponding to the target hand in the target result queue, and obtain the average motion speed of the preset key point in the target hand based on the target result queue.

[0097] Wherein, for the target result queue, the gesture class of the hand diagnosis gesture can be counted, and the gesture class proportion of the gesture class of the initial hand diagnosis gesture corresponding to the target hand in the target result queue can be calculated by counting.

[0098] It should be noted that according to the gesture class of the initial hand diagnosis gesture corresponding to the target hand, the preset key point in the target hand can be different. For example, the preset key point corresponding to the line drawing hand diagnosis gesture can be the tip of the thumb and the tip of the index finger, and the preset key point corresponding to the area range indicating gesture can be the tip of the index finger and the tip of the middle finger. Of course, the specific selection is not limited thereto, and the motion of the hand diagnosis gesture can be reflected.

[0099] Optionally, the average motion speed of the preset key point in the target hand can be calculated according to the displacement of the preset key point of the target hand in the adjacent two video frames and the frame interval. Specifically, the motion speed of the preset key point in the target hand corresponding thereto can be calculated according to the displacement of the preset key point of the target hand in the adjacent two video frames and the frame interval; then, the average motion speed of the preset key point in the target hand is calculated by mean value calculation.

[0100] S402, if it is determined that the gesture category proportion meets the first preset condition, it is determined that the gesture category of the target hand diagnosis gesture corresponding to the target hand in the target video frame is the same as the gesture category of the initial hand diagnosis gesture.

[0101] Optionally, the first preset condition can indicate a preset gesture category proportion threshold, in some embodiments, the preset gesture category proportion threshold can be 60%, 70%, 80%, etc., which is not limited herein.

[0102] Wherein, if it is determined that the gesture category proportion is greater than the preset gesture category proportion threshold, it is determined that the gesture category of the initial hand diagnosis gesture corresponding to the target hand in the target video frame is correct, that is, the gesture category of the target hand diagnosis gesture corresponding to the target hand in the target video frame is the same as the gesture category of the initial hand diagnosis gesture.

[0103] For example, assuming that the initial hand diagnosis gesture corresponding to the target hand in the target video frame is a line drawing hand diagnosis gesture, when the gesture category proportion of the initial hand diagnosis gesture corresponding to the target hand in the target video frame in the gesture category queue is greater than 70%, it is determined that the target hand diagnosis gesture corresponding to the target hand in the target video frame is a line drawing hand diagnosis gesture.

[0104] In summary, by applying the embodiments of the present application, the gesture category of the target hand can be comprehensively judged by the gesture category proportion, which can improve the accuracy of the gesture category judgment result.

[0105] Figure 6 A flowchart of another massage information generation method provided by the embodiments of the present application is shown.

[0106] Figure 7 A state transition diagram provided by the embodiments of the present application is shown. Figure 8 A hand diagnosis gesture diagram provided by the embodiments of the present application is shown. Figure 6 As shown, the above recording of the target hand diagnosis gesture corresponding to at least one hand in each video frame based on the preset transition relationship to generate the hand diagnosis result includes:

[0107] S501, if it is determined that the target hand diagnosis gesture corresponding to the target hand in the target video frame is the first preset gesture, and the average motion speed of the preset key point corresponding to the target hand in the target video frame is less than or equal to the first preset threshold, it is determined to enter the hand diagnosis recording state, and the position of the preset key point in the target hand is obtained.

[0108] S502, generating a hand diagnosis result according to the position of the preset key point in the target hand.

[0109] Wherein, by comparing whether the average motion speed of the preset key points corresponding to the target hand in the target video frame is less than or equal to the first preset threshold, the robustness to shaking can be increased, the false triggering of the recording state can be avoided, and the accuracy of recording can be improved.

[0110] Optionally, the first preset gesture can be a static gesture, such as Figure 8 As shown, it can be a range indication gesture, but is not limited thereto. Wherein, the range indication gesture can indicate the diameter of the nodule in the human body, and optionally, when recording the nodule, the center is the center of the line connecting the thumb tip and the index finger tip in the range indication gesture, and the diameter is the distance between the two.

[0111] Referring to Figure 7 As shown, if the first preset gesture is the range indication gesture pinch, the hand diagnosis user can switch from any hand diagnosis gesture to the first preset gesture, wherein if the target hand gesture corresponding to the target hand in the target video frame in the hand diagnosis video stream is the first preset gesture, and the average motion speed of the preset key points corresponding to the target hand in the target video frame is less than or equal to the first preset threshold, that is, the conversion path in Figure 7 Gesture == pinch and v <= Thres1, it indicates that the first preset gesture acted by the hand diagnosis user is a valid static gesture, at this time, the hand diagnosis recording state can be entered, and the positions of the preset key points in the target hand can be obtained, and then based thereon, the hand diagnosis result can be further generated; and when any condition in the conversion path is not met, that is, v <= Thres1 or Gesture!= pinch, the recording is stopped.

[0112] In summary, by applying the embodiments of the present application, the hand diagnosis gesture in a static state will not be recorded repeatedly, for example, when recording the nodule by the first preset gesture, if the hand is always placed in the target area without moving, it will not be recorded, avoiding false recording and improving the reliability of the method of the present application.

[0113] In some embodiments, if it is determined to enter the hand diagnosis recording state, optionally, the massage robot can be set to play a preset voice to remind the hand diagnosis user that the current hand diagnosis gesture is valid, and the intelligence of the massage robot can be improved.

[0114] Figure 9 A flowchart of another massage information generation method provided by the embodiments of the present application.

[0115] Figure 10 Another hand diagnosis gesture diagram provided by the embodiments of the present application. Optionally, as shown in Figure 9 As shown above, based on the preset conversion relationship, the target hand diagnosis gesture corresponding to at least one hand in each video frame is recorded to generate a hand diagnosis result, which includes:

[0116] S601, if it is determined that the target hand gesture corresponding to the target hand in the target video frame is a second preset hand gesture, and the average motion speed of the preset key point corresponding to the target hand in the target video frame meets the preset requirement, it is determined to enter a hand diagnosis recording state, and the position of the preset key point in the target hand is obtained.

[0117] S602, generating a hand diagnosis result according to the position of the preset key point in the target hand.

[0118] The second preset gesture can be a dynamic gesture, such as Figure 10 As shown, it can be a line drawing hand diagnosis gesture, through which the hand diagnosis path can be indicated, that is, the path walked in the line drawing hand diagnosis gesture.

[0119] The preset requirement can indicate the change rule of the average motion speed of the preset key point in the target hand.

[0120] Referring to Figure 7 As shown, if the second preset gesture is a line drawing hand diagnosis gesture pointer, the hand diagnosis user can switch from any hand diagnosis gesture to the second preset gesture, wherein if the target hand gesture corresponding to the target hand in the target video frame is the second preset gesture, and the average motion speed of the preset key point corresponding to the target hand in the target video frame meets the preset requirement, it is determined that the second preset gesture is a valid dynamic gesture, at this time, the hand diagnosis recording state can be entered, and the position of the preset key point in the target hand is obtained, and then based on this, the hand diagnosis result can be further generated.

[0121] In summary, by applying the embodiments of the present application, it can be avoided that a dynamic gesture is triggered and there is no motion all the time, and a very short hand diagnosis path is recorded by mistake.

[0122] Optionally, the average motion speed of the preset key point corresponding to the target hand in the target video frame meeting the preset requirement comprises:

[0123] If the average motion speed of the preset key point corresponding to the target hand in the target video frame is greater than a second preset threshold in a first time period, and is less than or equal to the second preset threshold in a second time period, it is determined that the average motion speed of the preset key point corresponding to the target hand in the target video frame meets the preset requirement.

[0124] Referring to Figure 7As shown, if the second preset gesture is the line drawing gesture, when the condition of the conversion path S1 is met, Gesture == = pointer and v <= Thres2, the user can switch from any gesture to the second preset gesture, wherein the hand diagnosis process can be divided into two stages, wherein in the first stage corresponding to the first time period (T1), the motion speed of the preset key point in the target hand is greater than the second preset threshold, that is, v > Thres2, and in the second stage corresponding to the second time period (T2), the motion speed of the preset key point in the target hand is less than or equal to the second preset threshold, that is, v <= Thres2, which indicates that the average speed of the preset key point in the target hand has experienced a deceleration operation from the first time period to the second time period during the hand diagnosis process. Specifically, during the hand diagnosis process, the user first accelerates the motion at the hand diagnosis start position Line_Start with the second preset gesture, and then decelerates the motion to the hand diagnosis end position Line_End.

[0125] It should be noted that from the above Figure 7 It can also be seen from the above that when the second preset gesture is the line drawing gesture and is at the hand diagnosis start position, if the motion speed of the preset key point in the target hand is less than or equal to the second preset threshold, that is, v <= Thres2, it indicates that the second preset gesture does not move at this time, and in this case, the static state is not recorded repeatedly, avoiding the misrecording of a very short hand diagnosis path after the dynamic gesture is triggered without movement. Figure 7 It can also be seen from the above that if the user switches the second preset gesture to any other gesture during the hand diagnosis process, it will not be recorded, which can avoid misrecording, that is, corresponding to the conversion paths S2, S3 and S4 in the above Figure 7 Furthermore, when the second preset gesture moves to the hand diagnosis end position Line_End, if the average speed of the preset key point in the target hand experiences a deceleration motion and then an acceleration motion, that is, corresponding to the conversion path S5 in the above Figure 7 In this case, it also will not be recorded repeatedly, which can avoid the continuous recording after the dynamic gesture motion stops, resulting in a very long hand diagnosis path.

[0126] It can be seen that the preset conversion relationship provided in the present application fully considers the state conversion between various hand diagnosis gestures, the state conversion between the hand diagnosis recording state and the non-recording state, and thus a more accurate hand diagnosis result can be obtained, improving the reliability of the method of the present application.

[0127] In addition, it can be seen from the above Figure 7 that the recording of the first preset gesture and the second preset gesture is an independent and complete process, and only one hand can record two states at the same time, and if there is an interruption in between, misrecording will not occur.

[0128] Optionally, the massage information is generated according to the hand diagnosis result, including: determining the massage parameters corresponding to the target region according to the hand diagnosis result, the massage parameters including target massage positions and / or target massage paths; and generating the massage information according to the massage parameters corresponding to the target region, the massage information being used to indicate a massage scheme corresponding to the target region.

[0129] Optionally, the massage parameters can include the relative positions of the target massage positions in the target region, and / or the relative positions of the target massage paths in the target region. Optionally, the relative positions of the target massage positions and / or the target massage paths in the target region can be marked by pixel positions in the target region.

[0130] Based on the determined massage parameters corresponding to the target region, massage information corresponding to the target region can be further generated. In some embodiments, the massage information can include massage intensity, massage duration, massage strategy, etc., which are not limited herein.

[0131] Optionally, the massage intensity and the massage duration can be a preset value, and the massage strategy can be determined according to massage experience. The massage strategy is used to indicate the massage order of the plurality of target massage positions and the massage order of the plurality of target massage paths. For example, in some embodiments, the determined massage strategy can be to cross massage a preset number of target massage positions and a preset number of target massage paths according to a preset rule (a nearest selection principle or a symmetric selection principle), until all target massage positions and target massage paths are completed.

[0132] It should be noted that, in some embodiments, the hand diagnosis result can also indicate an excluded massage region of the target region. Optionally, the excluded massage region can be indicated and determined by a hand diagnosis user using a preset gesture on a specified position in the target region. The preset gesture can be a single-finger line drawing gesture, a single-finger circle drawing gesture, etc., which are not limited herein.

[0133] For example, if the massage information generated according to the hand diagnosis result indicates two nodes and one massage line on the back of the human body, a massage scheme can be further generated according to the massage information. The massage scheme can indicate the massage order of the nodes and the massage line. In some embodiments, the massage of the nodes can be performed first, and then the massage of the massage path can be performed. For each node, the massage order of each node can be determined according to the position of each node on the human body, such as from top to bottom and from left to right. For the massage line, the massage line can be sorted before or after the nodes, which are not limited herein and can be flexibly set according to actual application scenarios.

[0134] Figure 11A massage scheme diagram provided by an embodiment of the present application. In some embodiments, the massage robot can also include a display device, and the above hand diagnosis result can also be synchronously displayed in the display device for further confirmation by the hand diagnosis user. If the result is confirmed to be correct, the hand diagnosis record can be retained, otherwise, the hand diagnosis record can be manually modified, or re-recorded, which is not limited herein. Optionally, as shown in Figure 11 when the hand diagnosis result is displayed through the display device, the hand diagnosis result can be displayed based on the image and the text. For example, when the hand diagnosis result is displayed by using the text, the hand diagnosis result can be synchronously marked in the human body picture by using a preset mark. For example, a preset circle can be used to mark the position of the nodule in the human body, which is not limited herein. As shown in Figure 11 there are two nodules in the left shoulder and one nodule in the right shoulder. The positions of the nodules can be referred to the marked positions in the human body.

[0135] Figure 12 A functional module diagram of a massage information generation device provided by an embodiment of the present application. The basic principle and the technical effects generated by the device are the same as those of the corresponding method embodiments described above. For brief description, the parts not mentioned in the present embodiment can be referred to the corresponding contents in the method embodiments. As shown in Figure 12 the massage information generation device 100 includes:

[0136] a recognition module 110, configured to recognize at least one target hand diagnosis gesture corresponding to a hand in a hand diagnosis video stream, the hand diagnosis video stream including a hand diagnosis gesture acting on a target region and being photographed;

[0137] a recording module 120, configured to record the at least one target hand diagnosis gesture corresponding to the hand, and generate a hand diagnosis result, the hand diagnosis result being used to indicate a target massage position and / or a target massage path of the target region;

[0138] a generation module 130, configured to generate massage information according to the hand diagnosis result.

[0139] In some embodiments, the hand diagnosis video stream is generated by a camera in the massage robot and the hand diagnosis gesture acting on the target region of the human body.

[0140] In some embodiments, the hand diagnosis gesture is an indicating gesture of the hand diagnosis user for the target massage position and / or the target massage path of the target region by using a specified gesture action.

[0141] In some embodiments, the recording module 120 is specifically configured to record the at least one target hand diagnosis gesture corresponding to the hand based on a preset conversion relationship, and generate the hand diagnosis result, the preset conversion relationship including a state conversion relationship between the hand diagnosis gestures, and a state conversion relationship between a hand diagnosis record state and a non-record state.

[0142] In some embodiments, the identification module 110 is specifically configured to identify a target hand diagnosis gesture corresponding to at least one hand in the hand diagnosis video stream by using a hand diagnosis gesture recognition model, the hand diagnosis gesture recognition model is obtained by training according to a training sample set, and the training sample set includes a plurality of training samples, and each training sample is labeled with a sample hand diagnosis gesture.

[0143] In some embodiments, the identification module 110 is specifically configured to determine at least one hand region in each video frame of the hand diagnosis video stream, and identify the key point position of each hand in each video frame based on the at least one hand region.

[0144] According to the key point position of each hand in each video frame and the previous result queue corresponding to each hand in each video frame, the target hand diagnosis gesture corresponding to each hand in each video frame is identified, and the previous result queue corresponding to each hand in each video frame includes the initial hand diagnosis gesture corresponding to each hand and the key point position of each hand.

[0145] In some embodiments, the identification module 110 is specifically configured to identify the bounding box of at least one hand region in each video frame of the hand diagnosis video stream.

[0146] According to the bounding box of at least one hand region in each video frame, at least one hand in each video frame is tracked and determined, and the key point position of each hand in each video frame is identified by using a hand key point detection model.

[0147] In some embodiments, the identification module 110 is specifically configured to identify the initial hand diagnosis gesture corresponding to the target hand in the target video frame according to the key point position of the target hand in the target video frame.

[0148] According to the initial hand diagnosis gesture corresponding to the target hand in the target video frame and the key point position of the target hand, a current gesture result is generated.

[0149] The current gesture result is added to the previous result queue corresponding to the target hand to obtain a target result queue corresponding to the target hand.

[0150] According to the target result queue corresponding to the target hand, the target hand diagnosis gesture corresponding to the target hand in the target video frame is determined.

[0151] In some embodiments, the identification module 110 is specifically configured to calculate the gesture class proportion of the gesture class of the initial hand diagnosis gesture corresponding to the target hand in the target result queue, and obtain the average motion speed of a preset key point in the target hand based on the target result queue.

[0152] If it is determined that the gesture class proportion meets a first preset condition, it is determined that the gesture class of the target hand diagnosis gesture corresponding to the target hand in the target video frame is the same as the gesture class of the initial hand gesture diagnosis.

[0153] In some embodiments, the recording module 120 is specifically configured to determine to enter a hand diagnosis recording state and acquire the positions of the preset key points in the target hand if it is determined that the target hand gesture corresponding to the target hand in the target video frame is a first preset hand gesture, and the average motion speed of the preset key points corresponding to the target hand in the target video frame is less than or equal to a first preset threshold value.

[0154] The hand diagnosis result is generated according to the positions of the preset key points in the target hand.

[0155] In some embodiments, the recording module 120 is specifically configured to determine to enter a hand diagnosis recording state and acquire the positions of the preset key points in the target hand if it is determined that the target hand gesture corresponding to the target hand in the target video frame is a second preset hand gesture, and the average motion speed of the preset key points corresponding to the target hand in the target video frame meets a preset requirement.

[0156] The hand diagnosis result is generated according to the positions of the preset key points in the target hand.

[0157] In some embodiments, the average motion speed of the preset key points corresponding to the target hand in the target video frame meeting the preset requirement comprises:

[0158] If the average motion speed of the preset key points corresponding to the target hand in the target video frame meeting the preset requirement is greater than a second preset threshold value in a first time period and is less than or equal to the second preset threshold value in a second time period, the average motion speed of the preset key points corresponding to the target hand in the target video frame meeting the preset requirement.

[0159] In some embodiments, the generation module 130 is specifically configured to determine a massage parameter corresponding to the target region according to the hand diagnosis result, the massage parameter comprising the target massage position and / or the target massage path.

[0160] Massage information is generated according to the massage parameter corresponding to the target region, the massage information being used to indicate a massage scheme corresponding to the target region.

[0161] The above device is used to execute the method provided by the foregoing embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0162] The above modules can be one or more integrated circuits configured to implement the above methods, for example: one or more application specific integrated circuits (ASICs), or one or more microprocessors, or one or more field programmable gate arrays (FPGAs), etc. For another example, when a certain module above is implemented in the form of a processing element scheduling code, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can invoke program code. For another example, the modules can be integrated together to implement in the form of a system on a chip (SOC).

[0163] Figure 13 An electronic device structure schematic diagram is provided for an embodiment of the present application. The electronic device can be integrated into the massage information generation device described above. As shown in the figure, the electronic device can include a processor 210, a storage medium 220, and a bus 230. The storage medium 220 stores machine-readable instructions executable by the processor 210. When the electronic device is running, the processor 210 communicates with the storage medium 220 through the bus 230. The processor 210 executes the machine-readable instructions to perform the following method embodiments: Figure 13

[0164] Identify a target hand diagnosis gesture corresponding to at least one hand in a hand diagnosis video stream, the hand diagnosis video stream including a hand diagnosis gesture acting on a target area;

[0165] Record the target hand diagnosis gesture corresponding to the at least one hand, and generate a hand diagnosis result, the hand diagnosis result being used to indicate a target massage position and / or a target massage path of the target area;

[0166] Generate massage information according to the hand diagnosis result.

[0167] In some embodiments, the hand diagnosis video stream is generated by a camera in a massage robot capturing a hand diagnosis gesture acting on a target area of a human body; the target area is a body area of a human body.

[0168] In some embodiments, the hand diagnosis gesture is an indication gesture of a hand diagnosis user adopting a specified gesture action for a target massage position and / or a target massage path of the target area.

[0169] In some embodiments, recording the target hand diagnosis gesture corresponding to the at least one hand and generating a hand diagnosis result includes:

[0170] ​The target hand diagnosis gesture corresponding to the at least one hand in the hand diagnosis video stream is identified based on a preset conversion relationship, and a hand diagnosis result is generated, the preset conversion relationship including a state conversion relationship between hand diagnosis gestures, a state conversion relationship between a hand diagnosis recording state and a non-recording state.

[0171] In some embodiments, the target hand diagnosis gesture corresponding to the at least one hand in the hand diagnosis video stream is identified by:

[0172] A hand diagnosis gesture recognition model is used to identify the target hand diagnosis gesture corresponding to the at least one hand in the hand diagnosis video stream, the hand diagnosis gesture recognition model being obtained by training according to a training sample set, the training sample set including a plurality of training samples, each of the training samples being labeled with a sample hand diagnosis gesture.

[0173] In some embodiments, the target hand diagnosis gesture corresponding to the at least one hand in the hand diagnosis video stream is identified by:

[0174] At least one hand region in each video frame of the hand diagnosis video stream is determined, and the key point positions of each hand in each video frame are identified based on the at least one hand region;

[0175] The target hand diagnosis gesture corresponding to each hand in each video frame is identified according to the key point positions of each hand in each video frame and the previous result queue corresponding to each hand in each video frame, the previous result queue corresponding to each hand in each video frame including an initial hand diagnosis gesture corresponding to each hand and the key point positions of each hand.

[0176] In some embodiments, at least one hand region in each video frame of the hand diagnosis video stream is determined, and the key point positions of each hand in each video frame are identified based on the at least one hand region, by:

[0177] A bounding box of at least one hand region in each video frame of the hand diagnosis video stream is identified;

[0178] At least one hand in each video frame is determined according to the bounding box of at least one hand region in each video frame, and the key point positions of each hand in each video frame are identified using a hand key point detection model.

[0179] In some embodiments, the target hand diagnosis gesture corresponding to each hand in each video frame is identified according to the key point positions of each hand in each video frame and the previous result queue corresponding to each hand in each video frame, by:

[0180] An initial hand diagnosis gesture corresponding to the target hand in the target video frame is identified according to the key point positions of the target hand in the target video frame;

[0181] A current gesture result is generated according to the initial hand diagnosis gesture corresponding to the target hand in the target video frame and the key point positions of the target hand.

[0182] add the current gesture result to a previous result queue corresponding to the target hand to obtain a target result queue corresponding to the target hand;

[0183] determine a target hand diagnosis gesture corresponding to the target hand in the target video frame according to the target result queue corresponding to the target hand.

[0184] In some embodiments, determining a target hand diagnosis gesture corresponding to the target hand in the target video frame according to the target result queue corresponding to the target hand includes:

[0185] calculating a gesture category proportion of the gesture category of the initial hand diagnosis gesture in the target result queue, and obtaining an average motion speed of a preset key point in the target hand based on the target result queue;

[0186] If it is determined that the gesture category proportion meets a first preset condition, it is determined that the gesture category of the target hand diagnosis gesture corresponding to the target hand in the target video frame is the same as the gesture category of the initial hand diagnosis gesture.

[0187] In some embodiments, recording the target hand diagnosis gesture corresponding to at least one hand in each video frame based on a preset conversion relationship to generate a hand diagnosis result includes:

[0188] If it is determined that the target hand diagnosis gesture corresponding to the target hand in the target video frame is a first preset gesture, and the average motion speed of the preset key point corresponding to the target hand in the target video frame is less than or equal to a first preset threshold, it is determined to enter a hand diagnosis recording state, and a position of the preset key point in the target hand is obtained.

[0189] generating a hand diagnosis result according to the position of the preset key point in the target hand.

[0190] In some embodiments, recording the target hand diagnosis gesture corresponding to at least one hand in each video frame based on a preset conversion relationship to generate a hand diagnosis result includes:

[0191] If it is determined that the target hand diagnosis gesture corresponding to the target hand in the target video frame is a second preset gesture, and the average motion speed of the preset key point corresponding to the target hand in the target video frame meets a preset requirement, it is determined to enter a hand diagnosis recording state, and a position of the preset key point in the target hand is obtained.

[0192] generating a hand diagnosis result according to the position of the preset key point in the target hand.

[0193] In some embodiments, the average motion speed of the preset key point corresponding to the target hand in the target video frame meets the preset requirement, including:

[0194] If the average motion speed of the preset key points corresponding to the target hand in the target video frame meets the preset requirement and is greater than the second preset threshold in the first time period and is less than or equal to the second preset threshold in the second time period, the average motion speed of the preset key points corresponding to the target hand in the target video frame meets the preset requirement.

[0195] In some embodiments, the massage information is generated according to the hand diagnosis result, including:

[0196] According to the hand diagnosis result, a massage parameter corresponding to the target area is determined, and the massage parameter includes the target massage position and / or the target massage path.

[0197] According to the massage parameter corresponding to the target area, massage information is generated, and the massage information is used to indicate a massage scheme corresponding to the target area.

[0198] The specific implementation manners and technical effects of the method are similar to those of the above method, and will not be described here.

[0199] Optionally, the application also provides a storage medium, and the storage medium stores a computer program. When the computer program is run by a processor, the steps of the method embodiments are executed.

[0200] A target hand diagnosis gesture corresponding to at least one hand in a hand diagnosis video stream is recognized, and the hand diagnosis video stream includes a hand diagnosis gesture acting on a target area.

[0201] The target hand diagnosis gesture corresponding to the at least one hand is recorded to generate a hand diagnosis result, and the hand diagnosis result is used to indicate a target massage position and / or a target massage path of the target area.

[0202] The massage information is generated according to the hand diagnosis result.

[0203] In some embodiments, the hand diagnosis video stream is generated by a camera in a massage robot and the hand diagnosis gesture acting on a target area of a human body.

[0204] In some embodiments, the hand diagnosis gesture is an indication gesture in which a hand diagnosis user adopts a specified gesture action to indicate a target massage position and / or a target massage path of the target area.

[0205] In some embodiments, the target hand diagnosis gesture corresponding to the at least one hand is recorded to generate the hand diagnosis result, including:

[0206] The target hand diagnosis gesture corresponding to the at least one hand is recorded to generate the hand diagnosis result based on a preset conversion relationship, and the preset conversion relationship includes a state conversion relationship between hand diagnosis gestures, a state conversion relationship between a hand diagnosis recording state and a non-recording state.

[0207] In some embodiments, the identifying the target hand diagnosis gesture corresponding to the at least one hand in the hand diagnosis video stream comprises:

[0208] adopting a hand diagnosis gesture recognition model to identify the target hand diagnosis gesture corresponding to the at least one hand in the hand diagnosis video stream, the hand diagnosis gesture recognition model being obtained according to training of a training sample set, the training sample set comprising a plurality of training samples, each of the training samples being labeled with a sample hand diagnosis gesture.

[0209] In some embodiments, the identifying the target hand diagnosis gesture corresponding to the at least one hand in the hand diagnosis video stream comprises:

[0210] determining at least one hand region in each video frame of the hand diagnosis video stream, and identifying key point positions of each hand in each video frame based on the at least one hand region;

[0211] identifying the target hand diagnosis gesture corresponding to each hand in each video frame according to the key point positions of each hand in each video frame and a previous result queue corresponding to each hand in each video frame, the previous result queue corresponding to each hand in each video frame comprising an initial hand diagnosis gesture corresponding to each hand and the key point positions of each hand.

[0212] In some embodiments, the determining at least one hand region in each video frame of the hand diagnosis video stream, and identifying key point positions of each hand in each video frame based on the at least one hand region comprises:

[0213] identifying a bounding box of the at least one hand region in each video frame of the hand diagnosis video stream;

[0214] tracking and determining at least one hand in each video frame according to the bounding box of the at least one hand region in each video frame, and identifying key point positions of each hand in each video frame by adopting a hand key point detection model.

[0215] In some embodiments, the identifying the target hand diagnosis gesture corresponding to each hand in each video frame according to the key point positions of each hand in each video frame and a previous result queue corresponding to each hand in each video frame comprises:

[0216] identifying an initial hand diagnosis gesture corresponding to a target hand in a target video frame according to the key point positions of the target hand in the target video frame;

[0217] generating a current gesture result according to the initial hand diagnosis gesture corresponding to the target hand in the target video frame and the key point positions of the target hand in the target video frame;

[0218] adding the current gesture result to a previous result queue corresponding to the target hand to obtain a target result queue corresponding to the target hand;

[0219] According to the target result queue corresponding to the target hand, a target hand diagnosis gesture corresponding to the target hand in the target video frame is determined.

[0220] In some embodiments, according to the target result queue corresponding to the target hand, a target hand diagnosis gesture corresponding to the target hand in the target video frame is determined, including:

[0221] A gesture category proportion of the gesture category of the initial hand diagnosis gesture corresponding to the target hand in the target result queue is calculated, and an average motion speed of a preset key point in the target hand is obtained based on the target result queue;

[0222] If it is determined that the gesture category proportion meets the first preset condition, it is determined that the gesture category of the target hand diagnosis gesture corresponding to the target hand in the target video frame is the same as the gesture category of the initial hand gesture diagnosis.

[0223] In some embodiments, based on the preset conversion relationship, a target hand diagnosis gesture corresponding to at least one hand in each video frame is recorded to generate a hand diagnosis result, including:

[0224] If it is determined that the target hand diagnosis gesture corresponding to the target hand in the target video frame is a first preset gesture, and the average motion speed of the preset key point corresponding to the target hand in the target video frame is less than or equal to a first preset threshold, it is determined to enter a hand diagnosis recording state, and a position of the preset key point in the target hand is obtained.

[0225] According to the position of the preset key point in the target hand, a hand diagnosis result is generated.

[0226] In some embodiments, based on the preset conversion relationship, a target hand diagnosis gesture corresponding to at least one hand in each video frame is recorded to generate a hand diagnosis result, including:

[0227] If it is determined that the target hand diagnosis gesture corresponding to the target hand in the target video frame is a second preset gesture, and the average motion speed of the preset key point corresponding to the target hand in the target video frame meets a preset requirement, it is determined to enter a hand diagnosis recording state, and a position of the preset key point in the target hand is obtained.

[0228] According to the position of the preset key point in the target hand, a hand diagnosis result is generated.

[0229] In some embodiments, the average motion speed of the preset key point corresponding to the target hand in the target video frame meets the preset requirement, including:

[0230] If the average motion speed of the preset key point corresponding to the target hand in the target video frame meets the preset requirement is greater than a second preset threshold within a first time period, and is less than or equal to the second preset threshold within a second time period, the average motion speed of the preset key point corresponding to the target hand in the target video frame meets the preset requirement.

[0231] In some embodiments, the generating the massage information according to the hand diagnosis result comprises:

[0232] According to the hand diagnosis result, determining the massage parameter corresponding to the target area, the massage parameter comprising the target massage position and / or the target massage path; and generating massage information according to the massage parameter corresponding to the target area, the massage information being used for indicating the massage scheme corresponding to the target area.

[0233] The specific implementation manners and technical effects of the method are similar to those of the method described above, and thus are not described herein.

[0234] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are only illustrative, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0235] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0236] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional unit.

[0237] The integrated unit implemented in the form of software functional units can be stored in a computer readable storage medium. The software functional unit is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to execute part of the steps of the method of each embodiment of the present application. And the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, for short: ROM), random access memory (English: Random Access Memory, for short: RAM), magnetic disk or optical disk and various program code storage media.

[0238] It should be noted that, in this paper, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0239] The above is only the preferred embodiment of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that: similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. The above is only the preferred embodiment of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A massage information generation method characterized by comprising: The method comprises: identifying a target hand diagnosis gesture corresponding to at least one hand in a hand diagnosis video stream, the hand diagnosis video stream comprising a hand diagnosis gesture applied to a target region captured by a camera in a massage robot; recording the target hand diagnosis gesture corresponding to the at least one hand to generate a hand diagnosis result, the hand diagnosis result being used to indicate a target massage position and / or a target massage path of the target region; generating massage information according to the hand diagnosis result.

2. The massage information generation method according to claim 1, characterized by, The hand diagnosis video stream is generated by the camera in the massage robot capturing the hand diagnosis gesture applied to the target region of the human body; and the target region is a body region of the human body.

3. The massage information generation method according to claim 1, characterized by, The hand diagnosis gesture is an indication gesture of a hand diagnosis user adopting a specified gesture action for the target massage position and / or the target massage path of the target region.

4. The massage information generation method according to claim 1, characterized by, The recording of the target hand diagnosis gesture corresponding to the at least one hand to generate the hand diagnosis result comprises: recording the target hand diagnosis gesture corresponding to the at least one hand to generate the hand diagnosis result based on a preset conversion relationship, the preset conversion relationship comprising a state conversion relationship between the hand diagnosis gestures, and a state conversion relationship between a hand diagnosis recording state and a non-recording state.

5. The massage information generation method according to claim 1, characterized by, The identification of the target hand diagnosis gesture corresponding to the at least one hand in the hand diagnosis video stream comprises: identifying the target hand diagnosis gesture corresponding to the at least one hand in the hand diagnosis video stream by using a hand diagnosis gesture recognition model, the hand diagnosis gesture recognition model being obtained by training according to a training sample set, the training sample set comprising a plurality of training samples, and each training sample being labeled with a sample hand diagnosis gesture.

6. The massage information generation method according to claim 1, characterized by, The identification of the target hand diagnosis gesture corresponding to the at least one hand in the hand diagnosis video stream comprises: determining at least one hand region in each video frame of the hand diagnosis video stream, and identifying key point positions of each hand in each video frame based on the at least one hand region; identifying the target hand diagnosis gesture corresponding to each hand in each video frame according to the key point positions of each hand in each video frame and a previous result queue corresponding to each hand in each video frame, the previous result queue corresponding to each hand in each video frame comprising an initial hand diagnosis gesture corresponding to each hand and the key point positions of each hand.

7. The massage information generation method according to claim 6, characterized by, The determination of the at least one hand region in each video frame of the hand diagnosis video stream and the identification of the key point positions of each hand in each video frame based on the at least one hand region comprise: identifying a bounding box of the at least one hand region in each video frame of the hand diagnosis video stream; tracking and determining at least one hand in each video frame according to the bounding box of the at least one hand region in each video frame, and identifying the key point positions of each hand in each video frame.

8. The massage information generation method according to claim 6, characterized by, The identification of the target hand diagnosis gesture corresponding to each hand in each video frame according to the key point positions of each hand in each video frame and the previous result queue corresponding to each hand in each video frame comprises: identifying an initial hand diagnosis gesture corresponding to a target hand in a target video frame according to the key point positions of the target hand in the target video frame; generating a current gesture result according to the initial hand diagnosis gesture corresponding to the target hand in the target video frame and the key point positions of the target hand; adding the current gesture result to the previous result queue corresponding to the target hand to obtain a target result queue corresponding to the target hand; determining the target hand diagnosis gesture corresponding to the target hand in the target video frame according to the target result queue corresponding to the target hand.

9. The massage information generation method according to claim 8, characterized by, According to the target result queue corresponding to the target hand, a target hand diagnosis gesture corresponding to the target hand in the target video frame is determined, including: The gesture category proportion of the initial hand diagnosis gesture corresponding to the target hand in the target result queue is calculated, and the average motion speed of the preset key point in the target hand is obtained based on the target result queue; If it is determined that the gesture category proportion meets the first preset condition, it is determined that the gesture category of the target hand diagnosis gesture corresponding to the target hand in the target video frame is the same as that of the initial hand diagnosis gesture.

10. The massage information generation method according to claim 4, characterized by, The target hand diagnosis gesture corresponding to at least one hand in each video frame is recorded based on the preset conversion relationship to generate a hand diagnosis result, including: If it is determined that the target hand diagnosis gesture corresponding to the target hand in the target video frame is a first preset gesture, and the average motion speed of the preset key point corresponding to the target hand in the target video frame is less than or equal to a first preset threshold, it is determined that the hand diagnosis recording state is entered, and the position of the preset key point in the target hand is obtained; The hand diagnosis result is generated according to the position of the preset key point in the target hand.

11. The massage information generation method according to claim 4, characterized by, The target hand diagnosis gesture corresponding to at least one hand in each video frame is recorded based on the preset conversion relationship to generate a hand diagnosis result, including: If it is determined that the target hand diagnosis gesture corresponding to the target hand in the target video frame is a second preset gesture, and the average motion speed of the preset key point corresponding to the target hand in the target video frame meets the preset requirement, it is determined that the hand diagnosis recording state is entered, and the position of the preset key point in the target hand is obtained; The hand diagnosis result is generated according to the position of the preset key point in the target hand.

12. The massage information generation method according to claim 11, characterized by, The average motion speed of the preset key point corresponding to the target hand in the target video frame meets the preset requirement, including: If the average motion speed of the preset key point corresponding to the target hand in the target video frame meets the preset requirement is greater than a second preset threshold in a first time period, and is less than or equal to the second preset threshold in a second time period, the average motion speed of the preset key point corresponding to the target hand in the target video frame meets the preset requirement.

13. The massage information generation method according to any one of claims 1 to 12, characterized by, The massage information is generated according to the hand diagnosis result, including: According to the hand diagnosis result, the massage parameters corresponding to the target area are determined, and the massage parameters include the target massage position and / or the target massage path; According to the massage parameters corresponding to the target area, massage information is generated, and the massage information is used to indicate the massage scheme corresponding to the target area.

14. A massage information generating apparatus characterized by comprising: Including: The identification module is used to identify the target hand diagnosis gesture corresponding to at least one hand in the hand diagnosis video stream, and the hand diagnosis video stream includes the hand diagnosis gesture acting on the target area; The recording module is used to record the target hand diagnosis gesture corresponding to the at least one hand to generate a hand diagnosis result, and the hand diagnosis result is used to indicate the target massage position and / or the target massage path of the target area; The generation module is used to generate massage information according to the hand diagnosis result.

15. An electronic device, comprising: Including: A processor, a storage medium, and a bus, the storage medium storing machine readable instructions executable by the processor, the processor communicating with the storage medium via the bus when the electronic device is running, the processor executing the machine readable instructions to perform the steps of the massage information generation method according to any one of claims 1-13.

16. A computer readable storage medium characterized by: A computer readable storage medium storing a computer program, the computer program being executed by a processor to perform the steps of the massage information generation method according to any one of claims 1-13.

Images