Method for establishing human body posture recognition model, human body posture recognition method and device

Abstract

The present invention relates to the field of image recognition technology, and in particular to a method for establishing a human posture recognition model, a human posture recognition method and a device, the method comprising: obtaining image training data, wherein the image training data comprises a current frame image and a previous frame image, the previous frame image is located in the previous frame of the current frame image, and the current frame image has a determined human posture; performing dimension-upgrading processing on the coordinates of human key points in the previous frame image to obtain a dimension-upgrading image corresponding to the previous frame image; splicing the current frame image with the dimension-upgrading image to obtain a spliced ​​image; using the coordinates of human key points of the human posture corresponding to the current frame image as training labels, training the spliced ​​image using a deep neural network, and obtaining a human posture recognition model. The present invention improves the accuracy of human posture recognition results.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method for establishing a human body posture recognition model, a human body posture recognition method and a human body posture recognition device. Background Art

[0002] Human posture recognition has become a very meaningful research hotspot in the fields of computer vision, pattern recognition, artificial intelligence, etc. It has broad application areas, including virtual reality, biomechanics, games, medical health and other human-computer interaction fields.

[0003] In the prior art, for a video, when recognizing the human posture of a person in the video, the human posture recognition is usually performed based on each frame image itself. For example, if a video contains a first frame and a second frame in chronological order, then the human posture of the person in the first frame image is recognized based on the first frame image to obtain a first human posture corresponding to the first frame image, and the human posture of the person in the second frame image is recognized based on the second frame image to obtain a second human posture corresponding to the second frame image. However, the above method of directly using the current frame image to perform human posture recognition has a low accuracy rate in the recognition result, and it is difficult to accurately recognize the human posture of each frame image. Summary of the invention

[0004] In view of the above problems, the present invention is proposed to provide a method for establishing a human body posture recognition model, a human body posture recognition method and a device that overcome the above problems or at least partially solve the above problems.

[0005] According to a first aspect of the present invention, the present invention provides a method for establishing a human posture recognition model, the method comprising:

[0006] Acquire image training data, wherein the image training data includes a current frame image and a previous frame image, the previous frame image is located in a frame before the current frame image, and the current frame image has a determined human body posture;

[0007] Performing dimension-upgrading processing on the coordinates of the key points of the human body in the previous frame image to obtain a dimension-upgraded image corresponding to the previous frame image;

[0008] Splicing the current frame image with the upgraded dimensional image to obtain a spliced ​​image;

[0009] The coordinates of the human body key points of the human body posture corresponding to the current frame image are used as training labels, and the spliced ​​image is trained using a deep neural network to obtain a human body posture recognition model.

[0010] Preferably, when the previous frame image contains n human body key point coordinates, the dimensionality-upgrading processing of the human body key point coordinates in the previous frame image includes:

[0011] The coordinates of the n human body key points are converted into an n-dimensional image based on Gaussian distribution.

[0012] Preferably, the step of splicing the current frame image with the upgraded dimensional image comprises:

[0013] The current frame image and the upgraded dimensional image are spliced ​​in a stacking manner.

[0014] Preferably, the deep neural network includes a neural network convolution layer and a target training tail layer corresponding to preset training conditions.

[0015] Preferably, the image training data is a color image or a grayscale image.

[0016] According to a second aspect of the present invention, the present invention provides a method for human body posture recognition, the method comprising:

[0017] Acquire an image to be identified and a target previous frame image located in a previous frame of the image to be identified;

[0018] The human body key point coordinates of the previous frame image of the target and the image to be recognized are input as input data to the human body posture recognition model obtained by the method for establishing a human body posture recognition model according to the first aspect, and the human body posture recognition model is used to perform human body posture recognition on the image to be recognized to obtain the human body key point coordinates of the image to be recognized;

[0019] Based on the coordinates of the human body key points of the image to be recognized, a human body posture corresponding to the image to be recognized is determined.

[0020] Preferably, after obtaining the coordinates of the human body key points of the image to be identified and before determining the human body posture corresponding to the image to be identified, the method further includes:

[0021] Performing out-of-bounds processing on the coordinates of the key points of the human body of the image to be identified;

[0022] The step of determining the human body posture corresponding to the image to be identified based on the coordinates of the human body key points of the image to be identified is specifically as follows:

[0023] Based on the coordinates of the human body key points of the image to be identified after the cross-border processing, the human body posture corresponding to the image to be identified is determined.

[0024] According to a third aspect of the present invention, the present invention provides a device for establishing a human posture recognition model, characterized in that the device comprises:

[0025] A data acquisition module, used to acquire image training data, wherein the image training data includes a current frame image and a previous frame image, the previous frame image is located in the previous frame of the current frame image, and the current frame image has a determined human body posture;

[0026] A dimension-upgrading module, used for performing dimension-upgrading processing on the coordinates of the key points of the human body in the previous frame image to obtain an upgraded dimension image corresponding to the previous frame image;

[0027] A splicing module, used for splicing the current frame image with the upgraded dimensional image to obtain a spliced ​​image;

[0028] The training module is used to use the coordinates of the human body key points of the human body posture corresponding to the current frame image as training labels, and use a deep neural network to train the spliced ​​image to obtain a human body posture recognition model.

[0029] Preferably, when the previous frame image contains n human key point coordinates, the dimension upgrading module is specifically used to:

[0030] The coordinates of the n human body key points are converted into an n-dimensional image based on Gaussian distribution.

[0031] Preferably, the splicing module is specifically used for:

[0032] The current frame image and the upgraded dimensional image are spliced ​​in a stacking manner.

[0033] Preferably, the deep neural network includes a neural network convolution layer and a target training tail layer corresponding to preset training conditions.

[0034] Preferably, the image training data is a color image or a grayscale image.

[0035] According to a fourth aspect of the present invention, the present invention provides a human body posture recognition device, characterized in that the device comprises:

[0036] An image acquisition module is used to acquire an image to be recognized and a target previous frame image located in a previous frame of the image to be recognized;

[0037] A recognition module, used to input the coordinates of the human body key points of the previous frame image of the target and the image to be recognized as input data into the human body posture recognition model obtained by the method for establishing a human body posture recognition model according to the first aspect, and perform human body posture recognition on the image to be recognized using the human body posture recognition model to obtain the coordinates of the human body key points of the image to be recognized;

[0038] The determination module is used to determine the human body posture corresponding to the image to be identified based on the coordinates of the human body key points of the image to be identified.

[0039] Preferably, the device further comprises:

[0040] An out-of-bounds processing module, used for performing out-of-bounds processing on the coordinates of the key points of the human body in the image to be identified;

[0041] Wherein, the determination module is specifically used for:

[0042] Based on the coordinates of the human body key points of the image to be identified after the cross-border processing, the human body posture corresponding to the image to be identified is determined.

[0043] According to a fifth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method steps in the first aspect or the second aspect described above.

[0044] According to a sixth aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method steps in the first aspect or the second aspect are implemented.

[0045] According to the method for establishing a human posture recognition model, the human posture recognition method and device of the present invention, image training data is first obtained, and the image training data includes a current frame image and a previous frame image, the previous frame image is located in the previous frame of the current frame image, and the current frame image has a determined human posture. Then, the coordinates of the human key points in the previous frame image are upgraded to obtain an upgraded image corresponding to the previous frame image. Then, the current frame image and the upgraded image are spliced ​​to obtain a spliced ​​image. Finally, the coordinates of the human key points of the human posture corresponding to the current frame image are used as training labels, and the spliced ​​image is trained using a deep neural network to obtain a human posture recognition model. Since not only the human body posture of the current frame image is considered during training, but also the human body key points of the previous frame image located in the previous frame of the current frame image are considered, the coordinates of the human body key points in the previous frame image are processed by dimensionality upgrading, and the current frame image is spliced ​​in the dimensionality upgraded image obtained by the dimensionality upgrading to obtain a spliced ​​image. The spliced ​​image reflects the relationship between the current frame image and the previous frame image. Therefore, the spliced ​​image is used for training, so that the relationship between the current frame image and the previous frame image is fully considered during training. Finally, a human body posture recognition model is established, which can recognize the human body posture of the current frame image based on the previous frame image and the current frame image. The model can be used to accurately recognize the human body posture of the image to be recognized based on the image to be recognized and the target previous frame image located in the previous frame of the image to be recognized, thereby improving the accuracy of the human body posture recognition results.

[0046] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Also, throughout the accompanying drawings, the same reference figures are used to represent the same components. In the drawings:

[0048] Figure 1 A flow chart of a method for establishing a human posture recognition model in a first embodiment of the present invention is shown;

[0049] Figure 2 A flow chart of a method for human body posture recognition in a second embodiment of the present invention is shown;

[0050] Figure 3 A structural diagram of a device for establishing a human posture recognition model in a third embodiment of the present invention is shown;

[0051] Figure 4 A structural diagram of a human body posture recognition device in a fourth embodiment of the present invention is shown;

[0052] Figure 5 A structural diagram of a computer device in a fifth embodiment of the present invention is shown. DETAILED DESCRIPTION

[0053] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0054] The first embodiment of the present invention provides a method for establishing a human posture recognition model, the method is used to establish a human posture recognition model, the human posture recognition model can recognize the posture of a human body in an image, the human posture includes but is not limited to standing, sitting, both hands raised sideways, etc. The execution subject of the method can be any electronic device with processing capabilities, such as a computer, a tablet computer, a mobile phone, etc.

[0055] See also Figure 1 As shown, the method for establishing a human posture recognition model according to an embodiment of the present invention comprises the following steps:

[0056] Step 101: Acquire image training data, wherein the image training data includes a current frame image and a previous frame image, the previous frame image is located in the previous frame of the current frame image, and the current frame image has a determined human body posture.

[0057] Step 102: Perform dimensionality upscaling processing on the coordinates of the key points of the human body in the previous frame image to obtain an upscaling image corresponding to the previous frame image.

[0058] Step 103: splice the current frame image with the upgraded dimensional image to obtain a spliced ​​image.

[0059] Step 104: using the coordinates of the human body key points of the human body posture corresponding to the current frame image as training labels, and using a deep neural network to train the spliced ​​image to obtain a human body posture recognition model.

[0060] Specifically, in step 101, firstly, a sample for modeling is obtained, and the sample is image training data. The source of the image training data includes a website and a large database, etc., and the embodiment of the present invention does not limit the source of the image training data. In step 101, a set of image training data can be obtained. However, in order to improve the accuracy of the model, in step 101, multiple sets of image training data can be obtained. Among them, a set of image training data includes a current frame image and a previous frame image, the previous frame image is located in the previous frame of the current frame image, and the current frame image and the previous frame image belong to the same video. For example, for the first video, the first frame image and the second frame image are two adjacent frames in the first video, then, the first frame image is the previous frame image, and the second frame image is the current frame image. Furthermore, the current frame image has a determined human body posture, that is, the human body posture of the current frame image is known.

[0061] It should be noted that, for the multiple sets of image training data obtained in step 101, each set of image training data can be derived from the same video or from different videos. For example, for the case of the same video, the adjacent first frame image and the second frame image are obtained in the first video, and the first frame image and the second frame image are the first set of image training data, and the adjacent third frame image and the fourth frame image are obtained in the first video, and the third frame image and the fourth frame image are the second set of image training data. For the case of different videos, the adjacent first frame image and the second frame image are obtained in the first video, and the first frame image and the second frame image are the first set of image training data, and the adjacent third frame image and the fourth frame image are obtained in the second video, and the third frame image and the fourth frame image are the second set of image training data. When the multiple sets of image training data are derived from different videos, the diversity of the data can be used to further improve the recognition accuracy of the established human posture recognition model.

[0062] Furthermore, the image training data may be a color image, such as an RGB three-channel color image, for example, an image of size 256*256*3. Using the color image as the image training data, the human posture recognition model finally established can accurately recognize the human posture of the color image. In addition, the image training data may also be a grayscale image, such as a Y single-channel grayscale image, for example, an image of size 256*256*1. Using the grayscale image as the image training data, the human posture recognition model finally established can accurately recognize the human posture of the grayscale image.

[0063] After acquiring the image training data, step 102 is performed. Specifically, the recognition of human posture depends on the recognition of human key points, that is, for an image containing a human body, after identifying the coordinates of each human key point in the image, the posture of the human body in the image can be determined by analyzing the coordinates of the human key points. Among them, the human key points are also called human skeleton key points, which can reflect human skeleton information. Usually, the human key points include parts such as the head, neck, shoulders, elbows, hands, buttocks, knees and feet. In an embodiment of the present invention, the previous frame image with the determined human key point coordinates can be obtained in advance, or the human key point coordinates in the previous frame image can be determined after acquiring the image training data, and this application does not limit this. At the same time, this application does not limit the method used to determine the human key point coordinates in the previous frame image.

[0064] Furthermore, in step 102, the coordinates of the key points of the human body are upgraded based on the Gaussian distribution. If the previous frame image contains n key points of the human body, then the previous frame image contains the coordinates of n key points of the human body, and then, the coordinates of the n key points of the human body are converted into an n-dimensional image based on the Gaussian distribution. For example, if the image training data is a 256*256*3 image, and the previous frame image contains the coordinates of n key points of the human body, for the previous frame image, the coordinates of the n key points of the human body are converted into a 256*256*n image through step 102. Specifically, the coordinates of each key point of the human body in the previous frame image are (x pi ,y pi ), where i represents the i-th human key point, 0≤i≤n. First, the coordinates of each human key point are converted into a 256*256*1 image. The value of each element in the 256*256*1 image is calculated using the following formula using Gaussian distribution:

[0065]

[0066] Among them, Y (x,y) is the value of the element in the 256*256*1 image, σ is the standard deviation, and σ is 1.

[0067] For n human key point coordinates, the above formula can be used to obtain n 256*256*1 images. Then, by stacking n 256*256*1 images, a 256*256*n image can be obtained. Among them, the 256*256*n image is an n-dimensional image, that is, a dimension-upgraded image.

[0068] After obtaining the upgraded dimension image, execute step 103. Specifically, in step 103, the current frame image and the upgraded dimension image are spliced ​​in a stacking manner to obtain a spliced ​​image. Among them, the stacking manner is a concat manner, and the current frame image and the upgraded dimension image can be spliced ​​into a spliced ​​image by using the concat manner. If the current frame image is a 256*256*3 image, the final spliced ​​image is 256*256*(3+n). If the current frame image is a 256*256*1 image, the final spliced ​​image is 256*256*(1+n). Splicing images in a concat manner can ensure the continuity of the images used as training data, thereby improving the recognition accuracy of the trained human posture recognition model.

[0069] After the spliced ​​image is obtained, step 104 is executed. Specifically, in step 104, since the current frame image has a determined human body posture, the coordinates of each human body key point in the human body posture of the current frame image are also known. Then, the coordinates of each human body key point in the human body posture of the current frame image are used as training labels. For example, if the coordinates of n human body key points in the human body posture of the current frame image are (x c1 ,y c1 ), (x c2 ,y c2 )……(x cn ,y cn ), then, (x c1 ,y c1 ), (x c2 ,y c2 )……(x cn ,y cn ) as training labels.

[0070] Further, for the deep neural network in the embodiment of the present invention, on the one hand, an existing basic deep neural network (backbone), such as VGG Net or Resnet, can be selected. In practice, different basic deep neural networks can be selected according to actual needs. For example, if the model accuracy is required to be high but the relatively slow speed can be accepted, then a basic deep neural network with a deeper network and a larger number of parameters such as Resnet101 or DenseNet169 can be selected. If the speed of the model is required to be high, that is, a faster model is required, and a small loss in model accuracy is acceptable, then a basic deep neural network with a shallower network and a smaller number of parameters such as Resnet18 can be selected.

[0071] For the deep neural network in the embodiment of the present invention, on the other hand, the deep neural network can also be customized. That is, the deep neural network includes a neural network convolution layer and a target training tail layer corresponding to the preset training conditions. Specifically, only the convolution layer of the basic deep neural network is retained, and the convolution layer of the basic deep neural network is used as the neural network convolution layer of the deep neural network in the embodiment of the present invention. Further, the training tail layer of the deep neural network is customized, and the target training tail layer corresponding to the preset training conditions is used as the training tail layer of the deep neural network in the embodiment of the present invention. Thus, the neural network convolution layer of the deep neural network in the embodiment of the present invention includes a neural network convolution layer and a target training tail layer.

[0072] For the target training tail layer, the target training tail layer is determined by the preset training conditions according to different actual needs. The target training tail layer includes a global average pooling layer and m fully connected layers, where m is an integer greater than or equal to 1. If the preset training condition representation requires that the processing speed of the human posture recognition model is faster, the value of m is smaller. If the preset training condition representation requires that the accuracy of the human posture recognition model is higher, the value of m is larger. That is, the more fully connected layers there are in the target training tail layer, the higher the accuracy of the model but the slower the speed. Conversely, the fewer fully connected layers there are in the target training tail layer, the lower the accuracy of the model but the faster the speed.

[0073] Furthermore, after the deep neural network is determined according to actual needs, the spliced ​​image is trained using the deep neural network, and L2loss is used as the loss function during the training process. Finally, a human posture recognition model is established. Furthermore, the human posture recognition model is used to recognize the image to be recognized, and the human posture corresponding to the image to be recognized is determined.

[0074] Based on the same inventive concept, the second embodiment of the present invention provides a human body posture recognition method, the implementation of which relies on the human body posture recognition model established in the first embodiment. Figure 2 The human body posture recognition method of the embodiment of the present invention comprises the following steps:

[0075] Step 201: Acquire an image to be recognized and a target previous frame image located in a previous frame of the image to be recognized.

[0076] Step 202: The coordinates of the human body key points of the target previous frame image and the image to be recognized are input into a human posture recognition model, and the human posture recognition model is used to perform human posture recognition on the image to be recognized to obtain the coordinates of the human body key points of the image to be recognized.

[0077] Step 203: Based on the coordinates of the human body key points of the image to be identified, determine the human body posture corresponding to the image to be identified.

[0078] Specifically, in the prediction stage of the human posture recognition model in the embodiment of the present invention, the input data depends on the image to be recognized and the target previous frame image located in the previous frame of the image to be recognized. Specifically, the input data is the coordinates of the key points of the human body of the image to be recognized and the target previous frame image. This application does not limit the method for determining the coordinates of the key points of the human body of the target previous frame image. After the input data is input into the human posture recognition model, the human posture recognition model automatically performs human posture recognition and finally outputs the coordinates of the key points of the human body of the image to be recognized as the output result. The corresponding human posture can be determined according to the coordinates of the key points of the human body of the image to be recognized. However, this application does not limit the method for determining the human posture according to the key points of the human body.

[0079] Furthermore, in the process of model prediction, the coordinates in the prediction result often exceed the image range, which leads to errors and reduces the accuracy of the recognition result. To overcome the above problems, in an embodiment of the present invention, after obtaining the human body key point coordinates of the image to be recognized, and before determining the human body posture corresponding to the image to be recognized, the method further includes: performing cross-border processing on the human body key point coordinates of the image to be recognized. The cross-border processing of the human body key point coordinates of the image to be recognized is specifically: adjusting the coordinates of the human body key point coordinates of the image to be recognized that exceed the range of the image to be recognized to the boundary value of the image to be recognized. For example, if the image to be recognized is 256*256, when the obtained human body key point coordinates are (270, 250), then the human body key point coordinates are adjusted to (256, 250) by cross-border processing. Further, in step 203, based on the human body key point coordinates of the image to be recognized after the cross-border processing, the human body posture corresponding to the image to be recognized is determined. Thereby, the error can be reduced and the accuracy of the recognition result can be improved.

[0080] According to the method for establishing a human posture recognition model, the human posture recognition method and device of the present invention, image training data is first obtained, and the image training data includes a current frame image and a previous frame image, the previous frame image is located in the previous frame of the current frame image, and the current frame image has a determined human posture. Then, the coordinates of the human key points in the previous frame image are upgraded to obtain an upgraded image corresponding to the previous frame image. Then, the current frame image and the upgraded image are spliced ​​to obtain a spliced ​​image. Finally, the coordinates of the human key points of the human posture corresponding to the current frame image are used as training labels, and the spliced ​​image is trained using a deep neural network to obtain a human posture recognition model. Since not only the human body posture of the current frame image is considered during training, but also the human body key points of the previous frame image located in the frame before the current frame image are considered, the coordinates of the human body key points in the previous frame image are processed by dimensionality upgrading, and the current frame image is spliced ​​in the upgraded image obtained by the dimensionality upgrading process to obtain a spliced ​​image. The spliced ​​image reflects the relationship between the current frame image and the previous frame image. Therefore, the spliced ​​image is used for training, so that the relationship between the current frame image and the previous frame image is fully considered during training. Finally, a human body posture recognition model is established, which can recognize the human body posture of the current frame image based on the previous frame image and the current frame image. The model can be used to accurately recognize the human body posture of the image to be recognized based on the image to be recognized and the target previous frame image located in the frame before the image to be recognized.

[0081] Based on the same inventive concept, the third embodiment of the present invention also provides a device for establishing a human posture recognition model, such as Figure 3 As shown, the device comprises:

[0082] The data acquisition module 301 is used to acquire image training data, wherein the image training data includes a current frame image and a previous frame image, the previous frame image is located in the previous frame of the current frame image, and the current frame image has a determined human body posture;

[0083] A dimension-upgrading module 302 is used to perform dimension-upgrading processing on the coordinates of the key points of the human body in the previous frame image to obtain a dimension-upgrading image corresponding to the previous frame image;

[0084] A stitching module 303 is used to stitch the current frame image with the upgraded dimensional image to obtain a stitched image;

[0085] The training module 304 is used to use the coordinates of the human body key points of the human body posture corresponding to the current frame image as training labels, and use a deep neural network to train the spliced ​​image to obtain a human body posture recognition model.

[0086] Preferably, when the previous frame image contains n human key point coordinates, the dimension upgrading module is specifically used to:

[0087] The coordinates of the n human body key points are converted into an n-dimensional image based on Gaussian distribution.

[0088] Preferably, the splicing module is specifically used for:

[0089] The current frame image and the upgraded dimensional image are spliced ​​in a stacking manner.

[0090] Preferably, the deep neural network includes a neural network convolution layer and a target training tail layer corresponding to preset training conditions.

[0091] Preferably, the image training data is a color image or a grayscale image.

[0092] Based on the same inventive concept, the fourth embodiment of the present invention also provides a human posture recognition device, such as Figure 4 As shown, the device comprises:

[0093] An image acquisition module 401 is used to acquire an image to be recognized and a target previous frame image located in a previous frame of the image to be recognized;

[0094] The recognition module 402 is used to input the coordinates of the human body key points of the previous frame image of the target and the image to be recognized as input data into the human body posture recognition model obtained by the method for establishing a human body posture recognition model according to the first aspect, and perform human body posture recognition on the image to be recognized using the human body posture recognition model to obtain the coordinates of the human body key points of the image to be recognized;

[0095] The determination module 403 is used to determine the human body posture corresponding to the image to be identified based on the coordinates of the human body key points of the image to be identified.

[0096] Preferably, the device further comprises:

[0097] An out-of-bounds processing module, used for performing out-of-bounds processing on the coordinates of the key points of the human body in the image to be identified;

[0098] Wherein, the determination module is specifically used for:

[0099] Based on the coordinates of the human body key points of the image to be identified after the cross-border processing, the human body posture corresponding to the image to be identified is determined.

[0100] Based on the same inventive concept, the fifth embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method steps described in the first and second embodiments are implemented.

[0101] Based on the same inventive concept, the sixth embodiment of the present invention further provides a computer device, such as Figure 5 For the sake of convenience, only the parts related to the embodiments of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present invention. The computer device can be any terminal device including a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), a car computer, etc., and the computer device is taken as a mobile phone as an example:

[0102] Figure 5 FIG. 1 is a block diagram of a portion of the structure related to the computer device provided by an embodiment of the present invention. Figure 5 , the computer device includes: a memory 501 and a processor 502. Those skilled in the art can understand that Figure 5 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0103] Combine the following Figure 5 A detailed introduction to the various components of computer equipment:

[0104] The memory 501 can be used to store software programs and modules. The processor 502 executes various functional applications and data processing by running the software programs and modules stored in the memory 501. The memory 501 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data (such as audio data, a phone book, etc.), etc. In addition, the memory 501 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0105] The processor 502 is the control center of the computer device, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 501, and calling data stored in the memory 501. Optionally, the processor 502 may include one or more processing units; preferably, the processor 502 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications.

[0106] In the embodiment of the present invention, the processor 502 included in the computer device may have functions corresponding to any method step in the first embodiment and the second embodiment described above.

[0107] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific languages ​​is for disclosing the best mode of the present invention.

[0108] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0109] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0110] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0111] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0112] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0113] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0114] A1. A method for establishing a human posture recognition model, characterized in that the method comprises:

[0115] Acquire image training data, wherein the image training data includes a current frame image and a previous frame image, the previous frame image is located in a frame before the current frame image, and the current frame image has a determined human body posture;

[0116] Performing dimension-upgrading processing on the coordinates of the key points of the human body in the previous frame image to obtain a dimension-upgraded image corresponding to the previous frame image;

[0117] Splicing the current frame image with the upgraded dimensional image to obtain a spliced ​​image;

[0118] The coordinates of the human body key points of the human body posture corresponding to the current frame image are used as training labels, and the spliced ​​image is trained using a deep neural network to obtain a human body posture recognition model.

[0119] A2. The method according to A1, characterized in that when the previous frame image contains n human key point coordinates, the dimensionality-upgrading processing of the human key point coordinates in the previous frame image comprises:

[0120] The coordinates of the n human body key points are converted into an n-dimensional image based on Gaussian distribution.

[0121] A3. The method according to A1, characterized in that the step of splicing the current frame image with the upgraded dimensional image comprises:

[0122] The current frame image and the upgraded dimensional image are spliced ​​in a stacking manner.

[0123] A4. The method according to A1 is characterized in that the deep neural network includes a neural network convolution layer and a target training tail layer corresponding to preset training conditions.

[0124] A5. The method according to A1 is characterized in that the image training data is a color image or a grayscale image.

[0125] B6. A method for human body posture recognition, characterized in that the method comprises:

[0126] Acquire an image to be identified and a target previous frame image located in a previous frame of the image to be identified;

[0127] The human body key point coordinates of the previous frame image of the target and the image to be recognized are input as input data to the human body posture recognition model obtained by any method of establishing a human body posture recognition model in A1-A5, and the human body posture recognition model is used to perform human body posture recognition on the image to be recognized to obtain the human body key point coordinates of the image to be recognized;

[0128] Based on the coordinates of the human body key points of the image to be recognized, a human body posture corresponding to the image to be recognized is determined.

[0129] B7. The method according to B6, characterized in that after obtaining the coordinates of the human body key points of the image to be identified and before determining the human body posture corresponding to the image to be identified, the method further comprises:

[0130] Performing out-of-bounds processing on the coordinates of the key points of the human body of the image to be identified;

[0131] The step of determining the human body posture corresponding to the image to be identified based on the coordinates of the human body key points of the image to be identified is specifically as follows:

[0132] Based on the coordinates of the human body key points of the image to be identified after the cross-border processing, the human body posture corresponding to the image to be identified is determined.

[0133] C8. A device for establishing a human posture recognition model, characterized in that the device comprises:

[0134] A data acquisition module, used to acquire image training data, wherein the image training data includes a current frame image and a previous frame image, the previous frame image is located in the previous frame of the current frame image, and the current frame image has a determined human body posture;

[0135] A dimension-upgrading module, used for performing dimension-upgrading processing on the coordinates of the key points of the human body in the previous frame image to obtain an upgraded dimension image corresponding to the previous frame image;

[0136] A splicing module, used for splicing the current frame image with the upgraded dimensional image to obtain a spliced ​​image;

[0137] The training module is used to use the coordinates of the human body key points of the human body posture corresponding to the current frame image as training labels, and use a deep neural network to train the spliced ​​image to obtain a human body posture recognition model.

[0138] C9. The device according to C8, characterized in that when the previous frame image contains n human key point coordinates, the dimension upgrading module is specifically used to:

[0139] The coordinates of the n human body key points are converted into an n-dimensional image based on Gaussian distribution.

[0140] C10. The device according to C8, characterized in that the splicing module is specifically used for:

[0141] The current frame image and the upgraded dimensional image are spliced ​​in a stacking manner.

[0142] C11. The device according to C8 is characterized in that the deep neural network includes a neural network convolution layer and a target training tail layer corresponding to preset training conditions.

[0143] C12. The device according to C8, characterized in that the image training data is a color image or a grayscale image.

[0144] D13. A human body posture recognition device, characterized in that the device comprises:

[0145] An image acquisition module is used to acquire an image to be recognized and a target previous frame image located in a previous frame of the image to be recognized;

[0146] A recognition module, used for inputting the coordinates of the key points of the human body of the previous frame image of the target and the image to be recognized as input data into the human body posture recognition model obtained by the method for establishing a human body posture recognition model described in any one of A1 to A5, performing human body posture recognition on the image to be recognized using the human body posture recognition model, and obtaining the coordinates of the key points of the human body of the image to be recognized;

[0147] The determination module is used to determine the human body posture corresponding to the image to be identified based on the coordinates of the human body key points of the image to be identified.

[0148] D14. The device according to D13, characterized in that the device further comprises:

[0149] An out-of-bounds processing module, used for performing out-of-bounds processing on the coordinates of the key points of the human body in the image to be identified;

[0150] Wherein, the determination module is specifically used for:

[0151] Based on the coordinates of the human body key points of the image to be identified after the cross-border processing, the human body posture corresponding to the image to be identified is determined.

[0152] E15. A computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the method steps described in any one of A1-A5 are implemented.

[0153] F16. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method steps described in any one of A1-A5 when executing the program.

Claims

1. A method for establishing a human posture recognition model, characterized in that: The method comprises: Acquire image training data, wherein the image training data includes a current frame image and a previous frame image, the previous frame image is located in a frame before the current frame image, and the current frame image has a determined human body posture; Performing dimension-upgrading processing on the coordinates of the key points of the human body in the previous frame image to obtain a dimension-upgraded image corresponding to the previous frame image; Splicing the current frame image with the upgraded dimensional image to obtain a spliced ​​image; Using the coordinates of the human body key points of the human body posture corresponding to the current frame image as training labels, and using a deep neural network to train the spliced ​​image to obtain a human body posture recognition model; When the previous frame image contains n human key point coordinates, the dimensionality-upgrading processing of the human key point coordinates in the previous frame image includes: Convert the coordinates of the n human body key points into an n-dimensional image based on Gaussian distribution; The step of splicing the current frame image with the upgraded dimension image comprises: The current frame image and the upgraded dimensional image are spliced ​​in a stacking manner.

2. The method according to claim 1, characterized in that The deep neural network includes a neural network convolution layer and a target training tail layer corresponding to preset training conditions.

3. The method according to claim 1, characterized in that The image training data is a color image or a grayscale image.

4. A human body posture recognition method, characterized in that: The method comprises: Acquire an image to be identified and a target previous frame image located in a previous frame of the image to be identified; The human body key point coordinates of the previous frame image of the target and the image to be recognized are input as input data to the human body posture recognition model obtained by the method for establishing a human body posture recognition model according to any one of claims 1 to 3, and the human body posture recognition model is used to perform human body posture recognition on the image to be recognized to obtain the human body key point coordinates of the image to be recognized; Based on the coordinates of the human body key points of the image to be recognized, a human body posture corresponding to the image to be recognized is determined.

5. The method according to claim 4, characterized in that After obtaining the coordinates of the human body key points of the image to be identified and before determining the human body posture corresponding to the image to be identified, the method further includes: Performing out-of-bounds processing on the coordinates of the key points of the human body of the image to be identified; The step of determining the human body posture corresponding to the image to be identified based on the coordinates of the human body key points of the image to be identified is specifically as follows: Based on the coordinates of the human body key points of the image to be identified after the cross-border processing, the human body posture corresponding to the image to be identified is determined.

6. A device for establishing a human posture recognition model, characterized in that: The device comprises: A data acquisition module, used to acquire image training data, wherein the image training data includes a current frame image and a previous frame image, the previous frame image is located in the previous frame of the current frame image, and the current frame image has a determined human body posture; A dimension-upgrading module, used for performing dimension-upgrading processing on the coordinates of the key points of the human body in the previous frame image to obtain an upgraded dimension image corresponding to the previous frame image; A splicing module, used for splicing the current frame image with the upgraded dimensional image to obtain a spliced ​​image; A training module, used to use the coordinates of the human body key points of the human body posture corresponding to the current frame image as training labels, and use a deep neural network to train the spliced ​​image to obtain a human body posture recognition model; When the previous frame image contains n human key point coordinates, the dimension-increasing module is specifically used to: Convert the coordinates of the n human body key points into an n-dimensional image based on Gaussian distribution; Splicing module, specifically used for: The current frame image and the upgraded dimensional image are spliced ​​in a stacking manner.

7. The device according to claim 6, characterized in that The deep neural network includes a neural network convolution layer and a target training tail layer corresponding to preset training conditions.

8. The device according to claim 6, characterized in that The image training data is a color image or a grayscale image.

9. A human body posture recognition device, characterized in that: The device comprises: An image acquisition module is used to acquire an image to be recognized and a target previous frame image located in a previous frame of the image to be recognized; A recognition module, used for inputting the coordinates of the key points of the human body of the previous frame image of the target and the image to be recognized as input data into the human body posture recognition model obtained by the method for establishing a human body posture recognition model according to any one of claims 1 to 3, performing human body posture recognition on the image to be recognized using the human body posture recognition model, and obtaining the coordinates of the key points of the human body of the image to be recognized; The determination module is used to determine the human body posture corresponding to the image to be identified based on the coordinates of the human body key points of the image to be identified.

10. The device according to claim 9, characterized in that The device also includes: An out-of-bounds processing module, used for performing out-of-bounds processing on the coordinates of the key points of the human body in the image to be identified; Wherein, the determination module is specifically used for: Based on the coordinates of the human body key points of the image to be identified after the cross-border processing, the human body posture corresponding to the image to be identified is determined.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method steps described in any one of claims 1 to 3 are implemented.

12. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method steps according to any one of claims 1 to 3 are implemented.

Images