Joint movement data detection method, device and equipment and storage medium

Abstract

The application provides a joint activity data detection method, device and equipment and a storage medium. The method comprises the following steps: obtaining a target hand image of a to-be-tested person; determining whether the target hand image meets a preset image processing condition; if the target hand image meets the image processing condition, inputting the target hand image into a trained key point prediction model to obtain hand key point prediction information corresponding to the target hand image output by the key point prediction model; and determining wrist activity data of the to-be-tested person according to the hand key point prediction information. The application can effectively improve the efficiency of joint activity data detection and reduce the labor and time cost of joint activity data detection.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, device and storage medium for detecting joint activity data. Background Technology

[0002] In the medical field, limited active or passive range of motion of the wrist joint is a common clinical problem, often caused by injuries such as fractures. Patients frequently experience limited range of motion and functional impairment due to pain, soft tissue shortening and stiffness after immobilization, and muscle weakness. Furthermore, limb spasticity and weakness resulting from central nervous system damage can also lead to limited range of motion and functional impairment.

[0003] Currently, wrist joint function is typically assessed using measuring tools such as goniometers to measure the maximum range of motion of the wrist joint. This type of assessment requires manual intervention, which can lead to long waiting times for patients when there are many patients, as well as problems such as low testing efficiency and high labor and time costs. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for detecting joint activity data, aiming to improve the efficiency of joint activity data detection and reduce the cost required for joint activity data detection.

[0005] In a first aspect, this application provides a method for detecting joint activity data, the method comprising the following steps:

[0006] Acquire the target hand image of the subject;

[0007] Determine whether the target hand image meets the preset image processing conditions;

[0008] If the target hand image meets the image processing conditions, the target hand image is input into the trained keypoint prediction model to obtain the hand keypoint prediction information corresponding to the target hand image output by the keypoint prediction model.

[0009] The wrist movement data of the person being tested are determined based on the predicted information of key hand points.

[0010] Secondly, this application also provides a joint activity data detection device, the joint activity data detection device comprising:

[0011] The image acquisition module is used to acquire target hand images of the person being tested;

[0012] The first processing module is used to determine whether the target hand image meets preset image processing conditions;

[0013] The prediction module is used to input the target hand image into a trained keypoint prediction model if the target hand image meets the image processing conditions, and obtain the hand keypoint prediction information corresponding to the target hand image output by the keypoint prediction model.

[0014] The second processing module is used to determine the wrist movement data of the person being tested based on the predicted information of the key points of the hand.

[0015] Thirdly, this application also provides a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, it implements the steps of the joint activity data detection method as described above.

[0016] Fourthly, this application also provides a storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of the joint activity data detection method described above.

[0017] This application provides a method, apparatus, device, and storage medium for detecting joint activity data. This application acquires a target hand image of a person to be tested, and when the target hand image meets the image processing conditions, predicts the hand key points corresponding to the target hand image based on a trained key point prediction model. The wrist activity data of the person to be tested is determined by the hand key point prediction information corresponding to the target hand image. The entire process does not require human intervention, which can effectively improve the efficiency of joint activity data detection and reduce the manual and time costs of joint activity data detection. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a joint activity data detection method provided in an embodiment of this application;

[0020] Figure 2a A schematic diagram of a target hand image provided in an embodiment of this application;

[0021] Figure 2b A schematic diagram of a target hand image provided for another embodiment of this application;

[0022] Figure 2cA schematic diagram of a target hand image provided in yet another embodiment of this application;

[0023] Figure 2d A schematic diagram of a target hand image provided in yet another embodiment of this application;

[0024] Figure 3 This is a schematic diagram of a joint activity data detection device provided in an embodiment of this application;

[0025] Figure 4 This is a schematic block diagram of the structure of a computer device according to an embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0028] This application provides a method, apparatus, computer device, and computer-readable storage medium for detecting joint activity data. The method can be applied to terminal devices such as mobile phones, tablets, laptops, and desktop computers. It can also be applied to servers, which can be standalone servers or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0029] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0030] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a method for detecting joint activity data according to an embodiment of this application.

[0031] like Figure 1 As shown, the joint activity data detection method includes steps S101 to S104.

[0032] Step S101: Obtain the target hand image of the person to be tested.

[0033] For example, an image acquisition device is used to acquire an image of the target hand of the person being tested, so that the terminal or server can obtain the image of the target hand of the person being tested.

[0034] Understandably, the image acquisition device is communicatively connected to a terminal and / or a server, wherein the terminal and / or server are used to implement the joint activity data detection method provided in this application.

[0035] In other embodiments, before acquiring an image of the target hand of the person being tested, the method further includes: playing voice prompts via a voice broadcasting device and / or displaying image prompts via a display device. The voice and image prompts are used to prompt the person being tested to position their hand in the target orientation in front of the image acquisition device, enabling the image acquisition device to obtain an image of the target hand. It is understood that the voice and image prompts may also be used to prompt the person being tested to change the orientation of their hand.

[0036] Step S102: Determine whether the target hand image meets the preset image processing conditions.

[0037] For example, after obtaining the target hand image, it is determined whether the target hand image meets the preset image processing conditions. If it does, step S103 is entered to determine the wrist movement data of the person being tested corresponding to the target hand image. If it does not meet the conditions, the corresponding prompt information is output and the target hand image is deleted, so that the image acquisition device can re-acquire the image of the person being tested's hand.

[0038] By determining whether the target hand image meets the preset image processing conditions, the system decides whether to process the target hand image. Images that do not meet the image processing conditions are filtered out to avoid processing these images, thereby improving the efficiency and accuracy of joint activity data detection.

[0039] In some embodiments, determining whether a target hand image meets preset image processing conditions includes: determining the shooting direction information, shooting angle information, and occlusion information of the target hand image; if the shooting direction information, shooting angle information, and occlusion information of the target hand image all meet the corresponding image processing conditions, the target hand image is determined to meet the preset image processing conditions.

[0040] For example, the target hand image can be used to determine the shooting direction information, shooting angle information, and occlusion information. Specifically, the shooting direction information is used to indicate from which direction the image acquisition device acquires the image of the subject's hand; the shooting angle information is used to indicate whether the hand is horizontal when acquiring the image of the hand, that is, whether the palm is parallel or perpendicular to the image acquisition device; the occlusion information is used to indicate whether the subject's hand is obstructed by clothing or other items, so as to obtain a clear image of the target hand.

[0041] It is understandable that when the shooting direction information, shooting angle information, and occlusion information all meet the corresponding image processing conditions, the target hand image is determined to meet the image processing conditions.

[0042] In other embodiments, if at least one of the shooting direction information, shooting angle information, and occlusion information of the target hand image does not meet the corresponding image processing conditions, it is determined that the target hand image does not meet the image processing conditions. In this case, the target hand image is deleted and the person being tested is prompted to repeat the hand image acquisition process.

[0043] In some embodiments, if the shooting direction information, shooting angle information, and occlusion information of the target hand image all meet the corresponding image processing conditions, determining that the target hand image meets the preset image processing conditions includes: when the shooting direction information of the target hand image is determined to be a target shooting direction, determining that the shooting direction information meets the first image processing condition, wherein the target shooting direction is used to indicate that the hand is being photographed from the thumb side or the back side of the hand; when the shooting angle of the target hand image is less than or equal to a first angle threshold, or greater than or equal to a second angle threshold, determining that the shooting angle information meets the second image processing condition, wherein the shooting angle is used to indicate the angle between the horizontal line where the palm of the hand is located and the shooting horizontal line, and the first angle threshold is less than the second angle threshold; when the occlusion information indicates that the hand in the target hand image is not occluded, determining that the occlusion information of the target hand image meets the third image processing condition; when the shooting direction information meets the first image processing condition, the shooting angle information meets the second image processing condition, and the occlusion information meets the third image processing condition, determining that the target hand image meets the image processing conditions.

[0044] For example, when determining the shooting direction information as the target shooting direction based on the target hand image, the shooting direction information is determined to meet the first image processing condition, wherein the target shooting direction is used to indicate that the image of the subject's hand is acquired from the thumb side, such as the thumb or little finger side; or the image of the subject's hand is acquired from the back side of the subject's hand.

[0045] For example, determining whether the shooting angle is less than or equal to a first angle threshold or greater than or equal to a second angle threshold involves, for instance, determining the relationship between the plane containing the subject's palm and the preset platform when the image acquisition device is parallel to a preset platform. Specifically, the angle between the plane containing the subject's palm and the preset platform is the shooting angle. If the angle between the plane containing the subject's palm and the preset platform is less than or equal to the first angle threshold and the subject's hand can be imaged by the image acquisition device, the shooting angle is determined to meet the second image processing condition. It can be understood that if the angle between the plane containing the subject's palm and the preset platform is greater than or equal to the first angle threshold and the subject's hand can be imaged by the image acquisition device, the shooting angle is determined to meet the second image processing condition. The first angle threshold is, for example, 5°, and the second angle threshold is, for example, 85°.

[0046] For example, occlusion information is determined by the color information of the target hand image, such as determining the color of the fingers and the color of the arm in the target hand image. If the color difference between the fingers and the arm is less than or equal to a preset color difference, it is determined that the hand of the person being tested corresponding to the target hand image is unoccluded, and the occlusion information meets the third image processing conditions.

[0047] For example, the process of determining whether the target hand image meets the image processing conditions can be implemented based on an image analysis model, which can be, for example, the DenseNet model.

[0048] Understandably, when the above-mentioned shooting direction information, shooting angle information, and occlusion information all meet the corresponding image processing conditions, the target hand image is determined to meet the image processing conditions, and key point prediction processing is performed on the target hand image.

[0049] Step S103: If the target hand image meets the image processing conditions, input the target hand image into the trained key point prediction model to obtain the hand key point prediction information corresponding to the target hand image output by the key point prediction model.

[0050] For example, after determining that the target hand image meets the image processing conditions, the target hand image is input into the key point prediction model to predict the key points of the target hand image and obtain the hand key point prediction information.

[0051] In some embodiments, the target hand image is input into a trained keypoint prediction model to obtain hand keypoint prediction information corresponding to the target hand image output by the keypoint prediction model, including: a segmentation network layer based on the keypoint prediction model to determine the arm and palm parts in the target hand image; and a keypoint prediction network layer based on the keypoint prediction model to predict elbow and wrist keypoints in the arm part, and palm and finger keypoints in the palm part.

[0052] For example, the key point prediction model includes a segmentation network layer and a key point prediction network layer. Based on the segmentation network layer, the target hand in the target hand image is segmented to obtain a first image for indicating the arm part and a second image for indicating the palm part. The first image and the second image are then input into the key point prediction network layer.

[0053] Based on the keypoint prediction network layer, elbow keypoints and wrist keypoints are predicted in the first image, and palm keypoints and finger keypoints are predicted in the second image.

[0054] In the specific implementation process, the backbone network of the key point prediction network layer is Hr-Net.

[0055] For example, wrist movement data of the test subject can be determined based on four key points obtained from the prediction.

[0056] Step S104: Determine the wrist movement data of the person being tested based on the hand key point prediction information.

[0057] For example, the hand activity data of the person being tested can be determined based on the predicted information of key hand points, realizing automated detection of wrist joint activity data without manual detection, thus effectively improving detection efficiency.

[0058] In other implementations, automatic claims processing can be achieved by using the wrist movement data of the person being tested and their insurance information, thereby improving the efficiency of claims processing.

[0059] In some embodiments, determining the wrist movement data of the subject based on the elbow key point, the wrist key point, the palm key point, and the finger key points includes: connecting the elbow key point and the wrist key point to obtain a first key point connecting line; connecting the palm key point and the finger key point to obtain a second key point connecting line; and determining the wrist movement data of the subject based on the first key point connecting line and the second key point connecting line.

[0060] For example, connecting the elbow key point and the wrist key point yields a first key point connecting line, and connecting the palm key point and the finger key points yields a second key point connecting line. The maximum range of motion of the wrist key is then determined based on the first key point connecting line and the second key point connecting line to determine wrist activity data.

[0061] In the specific implementation process, the method also includes: determining the hand movement posture of the target hand image based on the key point prediction model, so as to determine the wrist activity data of the person being tested based on the movement posture and key point prediction information. It is understandable that the position of key point prediction is different under different movement postures. Therefore, after determining the hand movement posture in the target hand image, the corresponding key point prediction is performed to improve the accuracy of key point prediction.

[0062] In some embodiments, the method further includes: acquiring motion posture information corresponding to the target hand image; and determining wrist activity data of the subject based on hand key point prediction information and motion posture information.

[0063] Understandably, during the process of acquiring hand images of the person being tested, the movement posture of the person's hand can be determined based on the input information of the person being tested and / or the staff, so that the terminal and / or the server can obtain the movement posture information corresponding to the target hand image. This improves the accuracy of hand key point prediction based on the movement posture information, and can determine the wrist activity data of the person being tested based on the movement posture information and hand key point prediction information.

[0064] Please see Figure 2a , Figure 2b , Figure 2c and Figure 2d , Figure 2a This is a schematic diagram of a target hand image provided in an embodiment of this application. Figure 2b This is a schematic diagram of a target hand image provided for another embodiment of this application. Figure 2c This is a schematic diagram of a target hand image provided in yet another embodiment of this application. Figure 2d This is a schematic diagram of a target hand image provided in yet another embodiment of this application.

[0065] For example, movement postures include back extension ( Figure 2a (as shown) or palmar flexion ( Figure 2b As shown), and scale deviation ( Figure 2c (as shown) or radial deviation ( Figure 2d As shown in the figure, it is understandable that if there are some movement disorders in the wrist joint, the wrist may still have normal movement function in a certain movement state, but may be restricted in another movement state. By determining the hand movement posture corresponding to the target hand image, the wrist key point prediction information can be determined, thereby determining the wrist activity data of the subject.

[0066] In some embodiments, determining the wrist movement data of the subject based on the first key point connecting line and the second key point connecting line includes: determining the angle between the first key point connecting line and the second key point connecting line; and determining the wrist movement data of the subject based on the angle.

[0067] For example, the angle between the first key point connecting line and the second key point connecting line is determined, and the maximum range of motion of the person being tested in the current motion posture is determined based on the angle of the angle.

[0068] Specifically, when the target hand image corresponds to a palmar flexion posture, the key points of the target hand image are determined, and the angle between the first key point connecting line and the second key point connecting line is obtained. The angle of this angle is then determined to obtain the wrist joint range of motion of the subject in palmar flexion. It can be understood that if the subject achieves the corresponding posture using the maximum range of motion of the wrist joint during image acquisition, then the angle represents the maximum range of motion of the subject's wrist joint in the current posture.

[0069] In other embodiments, video of the wrist joint movement of the subject is acquired, wherein the subject's wrist joint slowly moves from a dorsiflexion position to a palmar flexion position. The video of the wrist joint movement is split into frames based on preset time intervals to obtain multiple target hand images. The target hand images are then processed accordingly to obtain wrist activity data such as maximum wrist range of motion and wrist mobility (characterized by wrist movement rate). The specific implementation of processing the target hand images is as described in the above embodiments and will not be repeated here.

[0070] The joint activity data detection method provided in the above embodiments acquires a target hand image of the subject and, when the target hand image meets image processing conditions, predicts the hand keypoint prediction information corresponding to the target hand image based on a trained keypoint prediction model. This prediction information is then used to determine the wrist activity data of the subject. The entire process requires no manual intervention, effectively improving the efficiency and accuracy of joint activity data detection while reducing labor and time costs. Furthermore, this method enables patients and subjects to self-detect their wrist activity data and can output corresponding information based on the data in different application scenarios, such as treatment plans for the wrist joint, achieving automated diagnosis. This allows doctors to quickly obtain information about the patient's wrist joint without requiring manual testing for each patient, improving consultation efficiency. It can also output claim information based on wrist activity data, achieving automated insurance claims and improving the efficiency of claims processing.

[0071] Please see Figure 3 , Figure 3This is a schematic diagram of a joint activity data detection device provided in an embodiment of this application. The joint activity data detection device can be configured in a server or terminal to perform the aforementioned joint activity data detection method.

[0072] like Figure 3 As shown, the joint motion data detection device includes: an image acquisition module 110, a first processing module 120, a prediction module 130, and a second processing module 140.

[0073] Image acquisition module 110 is used to acquire target hand images of the person being tested.

[0074] The first processing module 120 is used to determine whether the target hand image meets the preset image processing conditions.

[0075] The prediction module 130 is used to input the target hand image into a trained keypoint prediction model if the target hand image meets the image processing conditions, and obtain the hand keypoint prediction information corresponding to the target hand image output by the keypoint prediction model.

[0076] The second processing module 140 is used to determine the wrist movement data of the person to be tested based on the hand key point prediction information.

[0077] For example, the prediction module 130 includes an image segmentation submodule and a key point prediction submodule.

[0078] The image segmentation submodule is used to determine the arm and palm parts in the target hand image based on the segmentation network layer of the key point prediction model.

[0079] The keypoint prediction submodule is used to predict elbow and wrist keypoints in the arm region and palm and finger keypoints in the palm region based on the keypoint prediction network layer of the keypoint prediction model.

[0080] The second processing module 140 is further configured to determine the wrist movement data of the person being tested based on the elbow key point, the wrist key point, the palm key point and the finger key points.

[0081] For example, the second processing module 140 includes a first connection submodule and a second connection submodule.

[0082] The first connection submodule is used to connect the elbow key point and the wrist key point to obtain the first key point connection line.

[0083] The second connection submodule is used to connect the key points of the palm and the key points of the fingers to obtain the second key point connection line.

[0084] The second processing module 140 is further configured to determine the wrist movement data of the person under test based on the first key point connection line and the second key point connection line.

[0085] For example, the second processing module 140 also includes an angle determination submodule.

[0086] The included angle determination submodule is used to determine the included angle between the first key point connection line and the second key point connection line.

[0087] The second processing module 140 is also used to determine the wrist movement data of the person being tested based on the included angle.

[0088] For example, the joint activity data detection device also includes an information determination module.

[0089] The information determination module is used to obtain the motion posture information corresponding to the target hand image.

[0090] The second processing module 140 is further configured to determine the wrist activity data of the person to be tested based on the hand key point prediction information and the motion posture information.

[0091] For example, the first processing module 120 includes a hand-capture information determination submodule.

[0092] The hand image capture information determination submodule is used to determine the shooting direction information, shooting angle information, and occlusion information of the target hand image.

[0093] The first processing module 120 is further configured to determine that the target hand image meets preset image processing conditions if the shooting direction information, the shooting angle information, and the occlusion information of the target hand image all meet the corresponding image processing conditions.

[0094] For example, the hand shooting information determination submodule includes a first shooting information determination submodule, a second shooting information determination submodule, and a third shooting information determination submodule.

[0095] The first shooting information determination submodule is used to determine that the shooting direction information of the target hand image meets the first image processing conditions when the shooting direction information of the target hand image is determined to be the target shooting direction. The target shooting direction is used to indicate that the hand is shot from the thumb side or from the back side of the hand.

[0096] The second shooting information determination submodule is used to determine that the shooting angle information meets the second image processing conditions when the shooting angle of the target hand image is less than or equal to a first angle threshold, or greater than or equal to a second angle threshold. The shooting angle is used to indicate the angle between the horizontal line where the palm of the hand is located and the shooting horizontal line, and the first angle threshold is less than the second angle threshold.

[0097] The third shooting information determination submodule is used to determine that the occlusion information of the target hand image meets the third image processing condition when the occlusion information indicates that the hand in the target hand image is unoccluded.

[0098] The first processing module 120 is further configured to determine that the target hand image meets the image processing conditions when the shooting direction information meets the first image processing conditions, the shooting angle information meets the second image processing conditions, and the occlusion information meets the third image processing conditions.

[0099] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the joint activity data detection device described above can be referred to the corresponding process in the aforementioned embodiments of the joint activity data detection method, and will not be repeated here.

[0100] Please see Figure 4 , Figure 4 This is a schematic block diagram illustrating the structure of a computer device provided in an embodiment of this application. The computer device may be a server or a terminal.

[0101] like Figure 4 As shown, the computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.

[0102] Non-volatile storage media can store operating systems and computer programs. These computer programs include program instructions that, when executed, cause the processor to perform any joint motion data detection method.

[0103] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0104] Internal memory provides an environment for the execution of computer programs stored in non-volatile storage media. When executed by a processor, the computer program enables the processor to perform any joint motion data detection method.

[0105] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 4The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0106] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0107] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps:

[0108] Acquire the target hand image of the subject;

[0109] Determine whether the target hand image meets the preset image processing conditions;

[0110] If the target hand image meets the image processing conditions, the target hand image is input into the trained keypoint prediction model to obtain the hand keypoint prediction information corresponding to the target hand image output by the keypoint prediction model.

[0111] The wrist movement data of the person being tested are determined based on the predicted information of key hand points.

[0112] In one embodiment, when the processor inputs the target hand image into a trained keypoint prediction model to obtain hand keypoint prediction information corresponding to the target hand image output by the keypoint prediction model, it is configured to:

[0113] Based on the segmentation network layer of the key point prediction model, the arm and palm parts are determined in the target hand image;

[0114] Based on the key point prediction network layer of the key point prediction model, elbow key points and wrist key points are predicted in the arm area, and palm key points and finger key points are predicted in the palm area.

[0115] When the processor determines the wrist movement data of the person being tested based on the arm key point prediction information, it is also used to:

[0116] The wrist movement data of the test subject are determined based on the elbow key point, the wrist key point, the palm key point and the finger key point.

[0117] In one embodiment, when the processor determines the wrist movement data of the subject based on the elbow key point, the wrist key point, the palm key point, and the finger key points, it is configured to:

[0118] The key points of the elbow and the key points of the wrist are connected to obtain the first key point connection line.

[0119] The key points of the palm and the key points of the fingers are connected to obtain the second key point connection line.

[0120] The wrist movement data of the person being tested are determined based on the first key point connection line and the second key point connection line.

[0121] In one embodiment, when the processor determines the wrist movement data of the subject based on the first keypoint connection line and the second keypoint connection line, it is configured to:

[0122] Determine the angle between the first key point connecting line and the second key point connecting line;

[0123] The wrist movement data of the person being tested are determined based on the included angle.

[0124] In one embodiment, the processor, when implementing the joint motion data detection method, is used to:

[0125] Obtain the motion posture information corresponding to the target hand image;

[0126] In determining the wrist movement data of the person being tested based on the hand key point prediction information, the processor is also used to:

[0127] The wrist movement data of the person being tested are determined based on the predicted information of key hand points and the information of movement posture.

[0128] In one embodiment, when determining whether the target hand image meets preset image processing conditions, the processor is configured to:

[0129] The shooting direction information, shooting angle information, and occlusion information of the target hand image are determined.

[0130] If the shooting direction information, shooting angle information, and occlusion information of the target hand image all meet the corresponding image processing conditions, then the target hand image is determined to meet the preset image processing conditions.

[0131] In one embodiment, when the processor determines that the target hand image meets preset image processing conditions if the shooting direction information, shooting angle information, and occlusion information of the target hand image all meet the corresponding image processing conditions, it is configured to:

[0132] When the shooting direction information of the target hand image is determined to be the target shooting direction, the shooting direction information is determined to meet the first image processing condition, wherein the target shooting direction is used to indicate that the hand is shot from the thumb side or from the back side of the hand;

[0133] When the shooting angle of the target hand image is less than or equal to a first angle threshold, or greater than or equal to a second angle threshold, the shooting angle information is determined to meet the second image processing condition. The shooting angle is used to indicate the angle between the horizontal line where the palm of the hand is located and the shooting horizontal line. The first angle threshold is less than the second angle threshold.

[0134] When the occlusion information indicates that the hand in the target hand image is unoccluded, it is determined that the occlusion information of the target hand image meets the third image processing condition;

[0135] When the shooting direction information meets the first image processing condition, the shooting angle information meets the second image processing condition, and the occlusion information meets the third image processing condition, the target hand image is determined to meet the image processing conditions.

[0136] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, and the method implemented when the program instructions are executed can refer to various embodiments of the joint activity data detection method of this application.

[0137] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0138] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0139] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0140] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above descriptions are merely specific implementations of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting joint motion data, characterized in that, include: Acquire the target hand image of the subject; Determine whether the target hand image meets the preset image processing conditions; If the target hand image meets the image processing conditions, the target hand image is input into the trained keypoint prediction model to obtain the hand keypoint prediction information corresponding to the target hand image output by the keypoint prediction model. The wrist movement data of the person being tested are determined based on the predicted information of key hand points. The step of determining whether the target hand image meets the preset image processing conditions includes: The shooting direction information, shooting angle information, and occlusion information of the target hand image are determined. When the shooting direction information of the target hand image is determined to be the target shooting direction, the shooting direction information is determined to meet the first image processing condition, wherein the target shooting direction is used to indicate that the hand is shot from the thumb side or from the back side of the hand; When the shooting angle of the target hand image is less than or equal to a first angle threshold, or greater than or equal to a second angle threshold, the shooting angle information is determined to meet the second image processing condition. The shooting angle is used to indicate the angle between the horizontal line where the palm of the hand is located and the shooting horizontal line. The first angle threshold is less than the second angle threshold. When the occlusion information indicates that the hand in the target hand image is unoccluded, it is determined that the occlusion information of the target hand image meets the third image processing condition; When the shooting direction information meets the first image processing condition, the shooting angle information meets the second image processing condition, and the occlusion information meets the third image processing condition, the target hand image is determined to meet the image processing conditions.

2. The joint motion data detection method as described in claim 1, characterized in that, The step of inputting the target hand image into a trained keypoint prediction model to obtain the hand keypoint prediction information corresponding to the target hand image output by the keypoint prediction model includes: Based on the segmentation network layer of the key point prediction model, the arm and palm parts are determined in the target hand image; Based on the key point prediction network layer of the key point prediction model, elbow key points and wrist key points are predicted in the arm area, and palm key points and finger key points are predicted in the palm area. The step of determining the wrist movement data of the person being tested based on the arm key point prediction information includes: The wrist movement data of the test subject are determined based on the elbow key point, the wrist key point, the palm key point and the finger key point.

3. The joint motion data detection method as described in claim 2, characterized in that, The process of determining the wrist movement data of the subject based on the elbow key point, the wrist key point, the palm key point, and the finger key points includes: The key points of the elbow and the key points of the wrist are connected to obtain the first key point connection line. The key points of the palm and the key points of the fingers are connected to obtain the second key point connection line. The wrist movement data of the person being tested are determined based on the first key point connection line and the second key point connection line.

4. The joint motion data detection method as described in claim 3, characterized in that, The step of determining the wrist movement data of the subject based on the first key point connection line and the second key point connection line includes: Determine the angle between the first key point connecting line and the second key point connecting line; The wrist movement data of the person being tested are determined based on the included angle.

5. The joint motion data detection method according to any one of claims 1-4, characterized in that, The method further includes: Obtain the motion posture information corresponding to the target hand image; The step of determining the wrist movement data of the subject based on the hand key point prediction information includes: The wrist movement data of the person being tested are determined based on the predicted information of key hand points and the information of movement posture.

6. A joint motion data detection device, characterized in that, The joint movement data detection device includes: The image acquisition module is used to acquire target hand images of the person being tested; The first processing module is used to determine whether the target hand image meets preset image processing conditions; The prediction module is used to input the target hand image into a trained keypoint prediction model if the target hand image meets the image processing conditions, and obtain the hand keypoint prediction information corresponding to the target hand image output by the keypoint prediction model. The second processing module is used to determine the wrist movement data of the person being tested based on the predicted information of the key points of the hand.

7. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, it implements the steps of the joint motion data detection method as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the joint activity data detection method as described in any one of claims 1 to 5.

Images