Key point detection method, device and storage medium

Abstract

The application relates to the technical field of image processing, and discloses a key point detection method, equipment and a storage medium, the method comprising the following steps: acquiring a current frame image and previous frame key point data, performing key point detection on the current frame image and the previous frame key point data based on a preset detection model, and the preset detection model is obtained by training based on key point data samples after adding random disturbance noise; since the key point detection is performed by using the preset detection model, the preset detection model is trained by adding random disturbance noise to simulate time sequence actions with complex postures, so that the robustness of the preset detection model is improved, the problem of key point jumping in model detection is solved, and the accuracy of key point detection is ensured.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a key point detection method, device, and storage medium. Background Technology

[0002] Currently, keypoint detection typically employs deep learning networks. This involves collecting and annotating a large number of images with different poses to train the model and achieve keypoint detection. However, in practical applications, this method suffers from insufficient stability in keypoint detection, exhibiting keypoint jitter and jumps when the pose is complex or the movements change rapidly.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide a key point detection method, device, and storage medium, which aims to solve the technical problem of key point jitter and jump when the human body posture is complex and the posture movement changes rapidly in the prior art.

[0005] To achieve the above objectives, the present invention provides a key point detection method, which includes the following steps:

[0006] Get the current frame image and the key point data of the previous frame;

[0007] Keypoint detection is performed based on the current frame image and the keypoint data of the previous frame using a preset detection model, which is trained on keypoint data samples with added random perturbation noise.

[0008] Optionally, before the step of acquiring the current frame image and the key point data of the previous frame, the method further includes:

[0009] Image samples and key point data samples are acquired, and the key point data samples are grouped to obtain a first data sample and a second data sample.

[0010] The first data sample is set to zero to obtain the first training sample;

[0011] Add random perturbation noise to the second data sample to obtain the second training sample;

[0012] The initial detection model is trained based on the image samples, the first training samples, and the second training samples to obtain a preset detection model.

[0013] Optionally, the step of adding random perturbation noise to the second data sample to obtain the second training sample includes:

[0014] Obtain the coordinate information of the second data sample and the image information corresponding to the second data sample;

[0015] Random perturbation noise is added based on the coordinate information and the image information to obtain a second training sample.

[0016] Optionally, the step of adding random perturbation noise based on the coordinate information and the image information to obtain the second training sample includes:

[0017] The disturbance range is determined based on the image information, and an interference value is selected from the disturbance range;

[0018] Random perturbation noise is added based on the coordinate information, the image information, and the interference value to obtain a second training sample.

[0019] Optionally, the step of determining the perturbation range based on the image information and selecting an interference value from the perturbation range includes:

[0020] Obtain the image sampling ratio when the initial detection model performs downsampling;

[0021] The downsampled image height and the downsampled image height are determined based on the image information and the image sampling ratio.

[0022] Compare the downsampled image height with the downsampled image height;

[0023] The disturbance range is determined based on the comparison results, and the disturbance value is selected from the disturbance range.

[0024] Optionally, the step of adding random perturbation noise based on the coordinate information, the image information, and the interference value to obtain the second training sample includes:

[0025] Random perturbation noise is added based on the coordinate information, the image information, and the interference value using a preset perturbation noise formula to obtain a second training sample;

[0026] The preset disturbance noise formula is:

[0027]

[0028]

[0029] In the formula, x i Let y be the x-coordinate of the second training sample. i u is the ordinate of the second training sample. i v i Based on the coordinate information, u is determined. i v is the x-coordinate of the second data sample.i The vertical coordinate of the second data sample is given. The width and height are determined based on the image information. The width is the image width, and the height is the image height. The random value is rand(-offset,offset).

[0030] Optionally, the step of detecting key points based on the current frame image and the key point data of the previous frame using a preset detection model includes:

[0031] The key point data of the previous frame is preprocessed to obtain a normalized coordinate vector;

[0032] Keypoint detection is performed based on the current frame image and normalized coordinate vector using a preset detection model.

[0033] Optionally, the step of detecting key points based on the current frame image and the normalized coordinate vector using a preset detection model includes:

[0034] Based on the current frame image and normalized coordinate vector, key point detection is performed using a preset detection model to obtain a two-dimensional heat map and a depth map;

[0035] The coordinates of the three-dimensional key points are determined based on the two-dimensional heat map and the depth map.

[0036] Furthermore, to achieve the above objectives, the present invention also proposes a key point detection device, which includes a memory, a processor, and a key point detection program stored in the memory and executable on the processor. The key point detection program is configured to implement the key point detection method described above.

[0037] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a key point detection program, which, when executed by a processor, implements the key point detection method as described above.

[0038] Furthermore, to achieve the above objectives, the present invention also proposes a key point detection device, which includes: an acquisition module and a detection module;

[0039] The acquisition module is used to acquire the current frame image and the key point data of the previous frame;

[0040] The detection module is used to detect key points based on the current frame image and the key point data of the previous frame using a preset detection model. The preset detection model is obtained by training on key point data samples with added random perturbation noise.

[0041] This invention discloses the acquisition of the current frame image and the key point data of the previous frame, and the key point detection based on the current frame image and the key point data of the previous frame using a preset detection model. The preset detection model is trained based on key point data samples with added random perturbation noise. Since this invention performs key point detection using a preset detection model, the preset detection model improves the robustness of the preset detection model by adding random perturbation noise during training to simulate complex temporal actions, thereby solving the problem of key point jumps in model detection and ensuring the accuracy of key point detection. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the structure of the key point detection device for the hardware operating environment involved in the embodiments of the present invention;

[0043] Figure 2 This is a flowchart illustrating the first embodiment of the key point detection method of the present invention;

[0044] Figure 3 This is a flowchart illustrating the second embodiment of the key point detection method of the present invention;

[0045] Figure 4 This is a schematic diagram of key point data sample processing in an embodiment of the key point detection method of the present invention;

[0046] Figure 5 This is a flowchart illustrating the third embodiment of the key point detection method of the present invention;

[0047] Figure 6 This is a flowchart illustrating the fourth embodiment of the key point detection method of the present invention;

[0048] Figure 7 This is a schematic diagram of a preset detection model in an embodiment of the key point detection method of the present invention;

[0049] Figure 8 This is a structural block diagram of the first embodiment of the key point detection device of the present invention.

[0050] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0051] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0052] Reference Figure 1 , Figure 1 This is a schematic diagram of the key point detection device structure of the hardware operating environment involved in the embodiment of the present invention.

[0053] like Figure 1As shown, the key point detection device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen, and optionally, it may also include a standard wired interface or a wireless interface. In this invention, the wired interface of the user interface 1003 may be a USB interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0054] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the key point detection device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0055] like Figure 1 As shown, the memory 1005, which is identified as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a key point detection program.

[0056] exist Figure 1 In the key point detection device shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the user equipment; the key point detection device calls the key point detection program stored in the memory 1005 through the processor 1001 and executes the key point detection method provided in the embodiment of the present invention.

[0057] Based on the above hardware structure, an embodiment of the key point detection method of the present invention is proposed.

[0058] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the key point detection method of the present invention, which presents the first embodiment of the key point detection method of the present invention.

[0059] Understandably, the following methods are commonly used for keypoint detection:

[0060] 1. Traditional human skeleton key point detection algorithms are based on template matching on the basis of geometric priors. They use templates to represent the entire human body structure, including the representation of key points, limb structures, and the relationships between different limb structures.

[0061] 2. Currently, most methods use deep learning networks for prediction. By collecting a large number of images of different human poses, they are labeled and trained to achieve the effect of predicting 2D and 3D key points of human skeleton.

[0062] However, the above method has the following drawbacks:

[0063] 1. Traditional methods and techniques cannot achieve real-time efficiency, have slow prediction efficiency, and are inaccurate in predicting complex human postures and movements.

[0064] 2. Existing deep learning solutions can basically meet the requirements of real-time prediction, but there is a problem of insufficient stability in key point prediction. When the human posture is complex and the posture changes rapidly, key point jitter and jump problems will occur.

[0065] To overcome the aforementioned shortcomings, this embodiment uses a preset detection model for key point detection. During training, the preset detection model incorporates random perturbation noise to simulate complex temporal movements, thereby improving the robustness of the preset detection model and solving the problem of key point jumps in model detection, thus ensuring the accuracy of key point detection.

[0066] In the first embodiment, the key point detection method includes the following steps:

[0067] Step S10: Obtain the current frame image and the key point data of the previous frame.

[0068] It should be understood that the execution subject of the method in this embodiment may be a key point detection device with data processing, network communication and program running functions, such as a server, or other electronic devices that can achieve the same or similar functions. This embodiment does not limit this.

[0069] It should be noted that key point data can be data of key locations in an image. For example, in a human image, key point data can be 2D key point data of the human skeleton.

[0070] Step S20: Based on the current frame image and the key point data of the previous frame, key point detection is performed using a preset detection model. The preset detection model is obtained by training on key point data samples after adding random perturbation noise.

[0071] It is understandable that key point detection based on the current frame image and the key point data of the previous frame can be achieved by inputting the current frame image and the key point data of the previous frame into the preset detection model to obtain the key point detection results.

[0072] It should be understood that, in order to ensure that the preset detection model can recognize temporal actions with complex postures, the preset detection model can be trained in advance based on key point data samples with added random perturbation noise.

[0073] In the first embodiment, the acquisition of the current frame image and the key point data of the previous frame is disclosed. Key point detection is performed based on the current frame image and the key point data of the previous frame using a preset detection model. The preset detection model is trained based on key point data samples with added random perturbation noise. Since this embodiment performs key point detection using a preset detection model, the preset detection model improves the robustness of the preset detection model by adding random perturbation noise during training to simulate complex temporal actions, thereby solving the problem of key point jumps in model detection and ensuring the accuracy of key point detection.

[0074] Reference Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the key point detection method of the present invention, based on the above. Figure 2 The first embodiment shown is followed by a second embodiment of the key point detection method of the present invention.

[0075] In the second embodiment, before step S10, the method further includes:

[0076] Step S01: Obtain image samples and key point data samples, and group the key point data samples to obtain a first data sample and a second data sample.

[0077] It should be understood that in practical applications, there may be situations where training occurs on the first frame and training occurs on other frames. Therefore, in order to ensure that there is data input to the detection model during training on the first frame, in this embodiment, the key point data samples are also zeroed out and noise is added.

[0078] It should be noted that the key point data sample can be a 2D key point data sample of the human skeleton, which can be input in advance.

[0079] It is understandable that grouping keypoint data samples to obtain the first and second data samples can be done based on a preset ratio. This preset ratio can be pre-set; for example, it could be 3:7.

[0080] In the specific implementation, 30% of the 2D key point data samples of the human skeleton are randomly divided into the first data sample, and 70% of the 2D key point data samples of the human skeleton are divided into the second data sample.

[0081] Step S02: Set the first data sample to zero to obtain the first training sample.

[0082] In the specific implementation, 30% of the 2D keypoint data of the human skeleton is randomly selected and set to 0, such as [0,0,...,0]. The original 2D keypoint ground truth remains unchanged. The data set to 0 is regarded as the first frame and used for subsequent model training. This step can be expressed by the formula x. i =0, y i =0i=1,2,3,…14, the keypoint array set to 0 can be represented as kps=[x1,y1,...,x i ,y i ].

[0083] Step S03: Add random perturbation noise to the second data sample to obtain the second training sample.

[0084] It should be understood that, in order to simulate the complex shaking motion of human posture, in this embodiment, random perturbation noise is added to the second data sample to obtain the second training sample, so as to improve the robustness of the data.

[0085] Step S04: Train the initial detection model based on the image samples, the first training samples, and the second training samples to obtain a preset detection model.

[0086] Understandably, during the first frame training, the image sample and the first training sample are input into the initial detection model for training to obtain the preset detection model; during training in subsequent frames, the image sample and the second training sample are input into the initial detection model for training to obtain the preset detection model.

[0087] For ease of understanding, please refer to Figure 4 This explanation does not limit the scope of this solution. Figure 4 This diagram illustrates the processing of keypoint data samples. In the diagram, 30% of the 2D keypoint data samples of the human skeleton are randomly divided into the first data sample, and 70% of the 2D keypoint data samples of the human skeleton are divided into the second data sample. 30% of the randomly selected 2D keypoint data of the human skeleton is set to 0, such as [0,0,...,0]. The original 2D keypoint ground truth remains unchanged. The data set to 0 is regarded as the first frame and used for subsequent model training. Random perturbation noise is added to 70% of the randomly selected 2D keypoint data of the human skeleton. The human pose image is the image sample.

[0088] In the second embodiment, the acquisition of image samples and key point data samples is disclosed. The key point data samples are grouped to obtain a first data sample and a second data sample. The first data sample is zeroed out to obtain a first training sample. Random perturbation noise is added to the second data sample to obtain a second training sample. The initial detection model is trained based on the image samples, the first training sample, and the second training sample to obtain a preset detection model. Since this embodiment performs zeroing out and adding noise to the key point data samples respectively, it can ensure that there is data input to the detection model, thereby improving the reliability of model training.

[0089] Reference Figure 5 , Figure 5 This is a flowchart illustrating the third embodiment of the key point detection method of the present invention. Based on the above embodiments, a third embodiment of the key point detection method of the present invention is proposed.

[0090] In the third embodiment, step S03 includes:

[0091] Step S031: Obtain the coordinate information of the second data sample and obtain the image information corresponding to the second data sample.

[0092] It should be understood that, in order to ensure that the added random perturbation noise conforms to the actual shaking of the human body posture, in this embodiment, random perturbation noise is also added based on the coordinate information and image information corresponding to the second data sample.

[0093] It should be noted that image information may include image height, image width, and other information.

[0094] Step S032: Add random perturbation noise based on the coordinate information and the image information to obtain the second training sample.

[0095] It is understandable that obtaining a second training sample by adding random perturbation noise based on coordinate and image information can be achieved by adding random perturbation noise to coordinate and image information using a pre-defined noise model. The pre-defined noise model can be used to simulate the actual shaking of human posture.

[0096] In the third embodiment, the method of obtaining the coordinate information of the second data sample and the image information corresponding to the second data sample is disclosed. Random perturbation noise is added according to the coordinate information and the image information to obtain the second training sample. Since this embodiment adds random perturbation noise according to the coordinate information and image information corresponding to the second data sample, it can ensure that the added random perturbation noise conforms to the actual shaking of the human body posture.

[0097] Furthermore, in order to simulate a smoother transition of the human pose key point data from the previous frame, step S032 includes:

[0098] The disturbance range is determined based on the image information, and an interference value is selected from the disturbance range;

[0099] Random perturbation noise is added based on the coordinate information, the image information, and the interference value to obtain a second training sample.

[0100] It should be understood that, in order to reduce the amount of computation and improve data processing efficiency, in this embodiment, the interference value is determined first, and then random perturbation noise is added based on the interference value.

[0101] It is understandable that determining the perturbation range based on image information can involve determining the image width and height, using the shorter side length of the image width and height as a reference value, and scaling the reference value by a preset ratio to obtain the perturbation range. The preset ratio can be set in advance.

[0102] Furthermore, to improve the accuracy of the perturbation range, the step of determining the perturbation range based on the image information and selecting an interference value from the perturbation range includes:

[0103] Obtain the image sampling ratio when the initial detection model performs downsampling;

[0104] The downsampled image height and the downsampled image height are determined based on the image information and the image sampling ratio.

[0105] Compare the downsampled image height with the downsampled image height;

[0106] The disturbance range is determined based on the comparison results, and the disturbance value is selected from the disturbance range.

[0107] In the specific implementation, the selection steps for the perturbation interval (-offset, offset) mainly include: 1. Calculating the size of the preset detection model downsampled to 1 / 4. At this time, the size of the downsampled image is (width / 4, height / 4), where width is the image width and height is the image height; 2. Using 10% of the length of the smallest side as the upper and lower limits of the perturbation interval, i.e., offset = min(width / 4, height / 4) * 10%. Assuming the image size is 300x300, the perturbation interval is (-7.5, 7.5). Human keypoint data is converted into a two-dimensional feature map through a fully connected layer. The size of this feature map is consistent with the size of the original image downsampled to 1 / 4. By limiting the upper and lower limits of the perturbation interval to 10% of the smallest side length of the image downsampled to 1 / 4, it helps to simulate a smoother transition of human pose keypoint data from the previous frame.

[0108] It is understandable that selecting an interference value from the disturbance range can be done randomly. For example, a value can be randomly selected from the disturbance range (-7.5, 7.5) as the interference value.

[0109] Furthermore, to ensure the reliability of adding random perturbation noise, the step of adding random perturbation noise based on the coordinate information, the image information, and the interference value to obtain the second training sample includes:

[0110] Random perturbation noise is added based on the coordinate information, the image information, and the interference value using a preset perturbation noise formula to obtain a second training sample;

[0111] The preset disturbance noise formula is:

[0112]

[0113]

[0114] In the formula, x i Let y be the x-coordinate of the second training sample. i u is the ordinate of the second training sample. i v i Based on the coordinate information, u is determined. i v is the x-coordinate of the second data sample. i The vertical coordinate of the second data sample is denoted by , and width and height are determined based on the image information. Width is the image width and height is the image height. rand(-offset,offset) is the interference value, which means taking a random number in the interference interval (-offset,offset).

[0115] It should be understood that, in order to ensure the reliability of adding random perturbation noise, a preset perturbation noise formula is introduced in this embodiment to add random perturbation noise.

[0116] Reference Figure 6 , Figure 6 This is a flowchart illustrating the fourth embodiment of the key point detection method of the present invention. Based on the above embodiments, the fourth embodiment of the key point detection method of the present invention is proposed.

[0117] In the fourth embodiment, step S10 includes:

[0118] Step S101: Preprocess the key point data of the previous frame to obtain a normalized coordinate vector.

[0119] It should be understood that, in order to improve the efficiency of key point detection, in this embodiment, the key point data of the previous frame can be preprocessed before key point detection.

[0120] It should be noted that preprocessing can include normalization, etc.

[0121] Step S102: Based on the current frame image and the normalized coordinate vector, key point detection is performed using a preset detection model.

[0122] Understandably, keypoint detection based on the current frame image and normalized coordinate vectors using a preset detection model involves inputting the current frame image and normalized coordinate vectors into the preset detection model to perform keypoint detection and obtain the detection results.

[0123] In the fourth embodiment, the key point data of the previous frame is preprocessed to obtain a normalized coordinate vector, and key point detection is performed based on the current frame image and the normalized coordinate vector using a preset detection model. Since this embodiment preprocesses the key point data of the previous frame before performing key point detection, the efficiency of key point detection can be improved.

[0124] In the fourth embodiment, step S20 includes:

[0125] Step S201: Based on the current frame image and normalized coordinate vector, key point detection is performed using a preset detection model to obtain a two-dimensional heat map and a depth map.

[0126] It should be understood that, in order to detect the coordinates of three-dimensional key points, in this embodiment, a two-dimensional heat map and a depth map can be obtained first through a preset detection model, and then the coordinates of three-dimensional key points can be determined.

[0127] It should be noted that the preset detection model can consist of a Mobilenetv2 backbone network, a fully connected network, a deconvolutional network, an upsampling network, and a convolutional network.

[0128] Step S202: Determine the coordinates of the three-dimensional key points based on the two-dimensional heat map and the depth map.

[0129] For ease of understanding, please refer to Figure 7 This explanation does not limit the scope of this solution. Figure 7 The diagram illustrates the detection process using a preset detection model. In the diagram, after inputting the current frame image and the normalized coordinate vector into the preset detection model, a two-dimensional heatmap and a depth map can be obtained. Since the depth map contains depth information, combining the two-dimensional heatmap and the depth map can determine the coordinates of the three-dimensional key points.

[0130] In the fourth embodiment, a key point detection method is disclosed based on the current frame image and normalized coordinate vectors using a preset detection model to obtain a two-dimensional heat map and a depth map, and then the coordinates of the three-dimensional key points are determined based on the two-dimensional heat map and the depth map. Since this embodiment first obtains the two-dimensional heat map and the depth map through the preset detection model and then determines the coordinates of the three-dimensional key points, it can accurately detect the coordinates of the three-dimensional key points.

[0131] Furthermore, this embodiment of the invention also proposes a storage medium storing a key point detection program, which, when executed by a processor, implements the key point detection method as described above.

[0132] In addition, refer to Figure 8 The present invention also proposes a key point detection device, which includes: an acquisition module 10 and a detection module 20;

[0133] Understandably, the following methods are commonly used for keypoint detection:

[0134] 1. Traditional human skeleton key point detection algorithms are based on template matching on the basis of geometric priors. They use templates to represent the entire human body structure, including the representation of key points, limb structures, and the relationships between different limb structures.

[0135] 2. Currently, most methods use deep learning networks for prediction. By collecting a large number of images of different human poses, they are labeled and trained to achieve the effect of predicting 2D and 3D key points of human skeleton.

[0136] However, the above method has the following drawbacks:

[0137] 1. Traditional methods and techniques cannot achieve real-time efficiency, have slow prediction efficiency, and are inaccurate in predicting complex human postures and movements.

[0138] 2. Existing deep learning solutions can basically meet the requirements of real-time prediction, but there is a problem of insufficient stability in key point prediction. When the human posture is complex and the posture changes rapidly, key point jitter and jump problems will occur.

[0139] To overcome the aforementioned shortcomings, this embodiment uses a preset detection model for key point detection. During training, the preset detection model incorporates random perturbation noise to simulate complex temporal movements, thereby improving the robustness of the preset detection model and solving the problem of key point jumps in model detection, thus ensuring the accuracy of key point detection.

[0140] The acquisition module 10 is used to acquire the current frame image and the key point data of the previous frame.

[0141] It should be noted that key point data can be data of key locations in an image. For example, in a human image, key point data can be 2D key point data of the human skeleton.

[0142] The detection module 20 is used to detect key points based on the current frame image and the key point data of the previous frame using a preset detection model. The preset detection model is obtained by training on key point data samples after adding random perturbation noise.

[0143] It is understandable that key point detection based on the current frame image and the key point data of the previous frame can be achieved by inputting the current frame image and the key point data of the previous frame into the preset detection model to obtain the key point detection results.

[0144] It should be understood that, in order to ensure that the preset detection model can recognize temporal actions with complex postures, the preset detection model can be trained in advance based on key point data samples with added random perturbation noise.

[0145] In this embodiment, the acquisition of the current frame image and the key point data of the previous frame is disclosed. Key point detection is performed based on the current frame image and the key point data of the previous frame using a preset detection model. The preset detection model is trained on key point data samples with added random perturbation noise. Since this embodiment performs key point detection using a preset detection model, the preset detection model improves the robustness of the preset detection model by adding random perturbation noise during training to simulate complex temporal actions, thereby solving the problem of key point jumps in model detection and ensuring the accuracy of key point detection.

[0146] Other embodiments or specific implementations of the key point detection device of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0147] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0148] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0149] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0150] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A key point detection method, characterized in that, The key point detection method includes the following steps: Acquire the current frame image and the key point data of the previous frame, where the key point data is the 2D key point data of the human skeleton; Based on the current frame image and the key point data of the previous frame, key points are detected by a preset detection model. The preset detection model is trained based on key point data samples with added random perturbation noise, wherein the random perturbation noise is used to simulate the complex shaking motion of human posture. Before acquiring the current frame image and the key point data of the previous frame, the process also includes: Image samples and key point data samples are acquired, and the key point data samples are grouped to obtain a first data sample and a second data sample. The first data sample is set to zero to simulate the boundary situation where there are no valid key points from the previous frame in the first frame, and the first training sample is obtained. Obtain the coordinate information of the second data sample and the image information corresponding to the second data sample; Obtain the image sampling ratio when the initial detection model performs downsampling, and determine the image width and image height of the downsampled image based on the image information and the image sampling ratio; The minimum side length between the image width and image height is used as a reference value. The reference value is scaled by a preset ratio to obtain the perturbation range. An interference value is selected from the perturbation interval, and random perturbation noise is added according to the coordinate information, the image information, and the interference value to obtain a second training sample; The initial detection model is trained based on the image samples, the first training samples, and the second training samples to obtain a preset detection model.

2. The key point detection method as described in claim 1, characterized in that, The step of adding random perturbation noise based on the coordinate information, the image information, and the interference value to obtain the second training sample includes: Random perturbation noise is added based on the coordinate information, the image information, and the interference value using a preset perturbation noise formula to obtain a second training sample; The preset disturbance noise formula is: In the formula, The x-coordinate of the second training sample. The ordinate of the second training sample is denoted as . , Determined based on coordinate information, The x-axis of the second data sample. The ordinate of the second data sample. , Determined based on image information, Image width, Image height, This is an interference value.

3. The key point detection method as described in claim 1 or 2, characterized in that, The step of detecting key points based on the current frame image and the key point data of the previous frame using a preset detection model includes: The key point data of the previous frame is preprocessed to obtain a normalized coordinate vector; Keypoint detection is performed based on the current frame image and normalized coordinate vector using a preset detection model.

4. The key point detection method as described in claim 3, characterized in that, The step of detecting key points based on the current frame image and normalized coordinate vector using a preset detection model includes: Based on the current frame image and normalized coordinate vector, key point detection is performed using a preset detection model to obtain a two-dimensional heat map and a depth map; The coordinates of the three-dimensional key points are determined based on the two-dimensional heat map and the depth map.

5. A key point detection device, characterized in that, The key point detection device includes: a memory, a processor, and a key point detection program stored in the memory and executable on the processor. When the key point detection program is executed by the processor, it implements the key point detection method as described in any one of claims 1 to 4.

6. A storage medium, characterized in that, The storage medium stores a key point detection program, which, when executed by a processor, implements the key point detection method as described in any one of claims 1 to 4.

Images