Human body posture assessment method, device, computer equipment and storage medium

Abstract

The present invention discloses a human body posture evaluation method, device, computer equipment and storage medium, which include obtaining an image to be evaluated, which is an image including a user's posture; inputting the image to be evaluated into a preset human body posture estimation network to obtain human body key point data, which includes key point coordinates and a human body surrounding frame; scaling the image to be evaluated through the human body surrounding frame, and performing coordinate transformation on the key point coordinates according to the scaled image to be evaluated to obtain the coordinates to be evaluated; obtaining standard comparison data, which includes standard coordinates and confidence levels corresponding to each standard coordinate; calculating the similarity between the coordinates to be evaluated and the standard coordinates through the confidence levels to obtain evaluation information of the image to be evaluated; and realizing rapid and accurate evaluation of the user's human body posture by evaluating and analyzing the similarity between each coordinate to be evaluated and the standard coordinates in the image to be evaluated.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method, device, computer equipment and storage medium for evaluating human body posture. Background Art

[0002] Analyzing the posture of the human body through images is an important issue in computer vision research. At present, human posture assessment is widely used in many fields such as human-computer interaction, movie special effects, and intelligent monitoring systems. For example, in the field of sports, more and more people are doing fitness exercises through fitness guidance apps. However, when users practice sports according to video instructions, they may suffer reduced exercise effects or even get injured due to non-standard movements. Therefore, how to quickly and accurately evaluate the correctness of the user's posture and provide users with more scientific fitness guidance has become a problem that needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present invention provide a human body posture assessment method, apparatus, computer equipment and storage medium to solve the problem that human body posture cannot be assessed quickly and accurately.

[0004] A human body posture assessment method, comprising:

[0005] Acquire an image to be evaluated, wherein the image to be evaluated is an image including a user's posture;

[0006] Inputting the image to be evaluated into a preset human posture estimation network to obtain human key point data, wherein the human key point data includes key point coordinates and a human body surrounding frame;

[0007] Scaling the image to be evaluated by using the human body surrounding frame, and performing coordinate transformation on the key point coordinates according to the scaled image to be evaluated to obtain coordinates to be evaluated;

[0008] Acquire standard comparison data, wherein the standard comparison data includes standard coordinates and a confidence level corresponding to each of the standard coordinates;

[0009] The similarity between the coordinates to be evaluated and the standard coordinates is calculated by using the confidence level to obtain evaluation information of the image to be evaluated.

[0010] A human body posture assessment device, comprising:

[0011] An image acquisition module to be evaluated, used to acquire an image to be evaluated, wherein the image to be evaluated is an image including a user's posture;

[0012] An input module, used for inputting the image to be evaluated into a preset human posture estimation network to obtain human key point data, wherein the human key point data includes key point coordinates and a human body surrounding frame;

[0013] A scaling processing module, used to scale the image to be evaluated by using the human body surrounding frame, and to perform coordinate transformation on the key point coordinates according to the scaled image to be evaluated to obtain coordinates to be evaluated;

[0014] A standard comparison data acquisition module, used to acquire standard comparison data, wherein the standard comparison data includes standard coordinates and a confidence level corresponding to each standard coordinate;

[0015] The similarity calculation module is used to calculate the similarity between the coordinates to be evaluated and the standard coordinates through the confidence level to obtain evaluation information of the image to be evaluated.

[0016] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned human body posture assessment method when executing the computer program.

[0017] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the human body posture assessment method is implemented.

[0018] The human body posture evaluation method, device, computer equipment and storage medium obtain an image to be evaluated, which is an image including a user's posture; input the image to be evaluated into a preset human body posture estimation network to obtain human body key point data, which includes key point coordinates and a human body enclosing frame; scale the image to be evaluated through the human body enclosing frame, and perform coordinate transformation on the key point coordinates according to the scaled image to be evaluated to obtain the coordinates to be evaluated; obtain standard comparison data, which includes standard coordinates and the confidence corresponding to each standard coordinate; calculate the similarity between the coordinates to be evaluated and the standard coordinates through the confidence to obtain evaluation information of the image to be evaluated; and by evaluating and analyzing the similarity between each coordinate to be evaluated and the standard coordinate in the image to be evaluated, a rapid and accurate evaluation of the user's human body posture is achieved, thereby facilitating the provision of accurate movement guidance and targeted feedback to the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0020] Figure 1is a schematic diagram of an application environment of a human body posture assessment method according to an embodiment of the present invention;

[0021] Figure 2 is an example diagram of a method for evaluating human posture in one embodiment of the present invention;

[0022] Figure 3 is another example diagram of a method for assessing human posture in one embodiment of the present invention;

[0023] Figure 4 is another example diagram of a method for assessing human posture in one embodiment of the present invention;

[0024] Figure 5 is another example diagram of a method for assessing human posture in one embodiment of the present invention;

[0025] Figure 6 is another example diagram of a method for assessing human posture in one embodiment of the present invention;

[0026] Figure 7 is a principle block diagram of a human body posture assessment device in one embodiment of the present invention;

[0027] Figure 8 is another principle block diagram of a human body posture assessment device in one embodiment of the present invention;

[0028] Fig. 9 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] The human body posture evaluation method provided by the embodiment of the present invention can be applied as follows: Figure 1 Specifically, the human body posture evaluation method is applied in a human body posture evaluation system, and the human body posture evaluation system includes: Figure 1 The client and server shown in the figure communicate with each other through the network to solve the problem that human posture cannot be quickly and accurately evaluated. The client, also known as the user end, refers to the program corresponding to the server that provides local services to customers. The client can be installed on, but not limited to, various personal computers, laptops, smart phones, tablets, and portable wearable devices. The server can be implemented with an independent server or a server cluster consisting of multiple servers.

[0031] In one embodiment, if Figure 2 As shown, a human body posture assessment method is provided, and the method is applied in Figure 1 The server in the example is used as an example to illustrate the following steps:

[0032] S11: Acquire an image to be evaluated, where the image to be evaluated is an image including a user's posture.

[0033] Among them, the image to be evaluated refers to the image to be evaluated for human posture. The image to be evaluated includes an image of the user's posture, that is, the image to be evaluated includes the user's standing posture, sitting posture, kneeling posture or any other posture. Optionally, the image to be evaluated can be obtained by real-time acquisition of an image containing the user's posture by a camera as the image to be evaluated, or pre-acquiring an image containing the user's posture as the image to be evaluated, or directly acquiring the user's posture image from a user posture library as the image to be evaluated, and the user's posture image can also be obtained from a data set disclosed by the Internet or a third-party organization / platform as the image to be evaluated, such as: a fitness guidance app.

[0034] S12: Input the image to be evaluated into a preset human posture estimation network to obtain human key point data, which includes key point coordinates and a human body bounding box.

[0035] Among them, the human posture estimation network refers to a pre-built network framework that can recognize the user posture in the image to be evaluated and output a recognition result, that is, the human key point data. In this embodiment, the human posture estimation network uses the OpenPose framework. Openpose is an open source library based on convolutional neural networks and supervised learning and Caffe as the framework of Ctrip. It can track human facial expressions, torsos, limbs and even fingers, and can output the positioning of key points of the face, the positioning of key points of the hands and the positioning of each joint of the human body; the OpenPose framework is not only suitable for single people but also for multiple people, and has good robustness.

[0036] Specifically, the image to be evaluated is input into a preset human posture estimation network. By connecting the joints of the human body (neck, shoulders, elbows, etc.), the relative positions of the key points of the human body in three-dimensional space are calculated, and the position changes of the key points of the human body are observed, so as to estimate the human posture and obtain the key point data of the human body.

[0037] Among them, the human body key point data includes key point coordinates and human body bounding box. Key point coordinates refer to the coordinate positions of the key points of human body posture identified from the image to be evaluated. The key points of human body posture can be 14 key points, including head, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle. The key point coordinates can be represented by a two-dimensional coordinate plane. For example: the key point coordinates corresponding to the head key point are (x 1 ,y 1 ), the key point coordinates corresponding to the neck key point are (x 2 ,y 2 ) and the key point coordinates corresponding to the left shoulder key point are (x 3, y 3 ) etc. It can be understood that each image to be evaluated includes a plurality of key point coordinates. The human body enclosing frame refers to an external enclosing boundary frame enclosing the key points of the human body. Preferably, the human body enclosing frame can be represented by the coordinate values ​​of four points on the enclosing frame.

[0038] S13: scaling the image to be evaluated using the human body surrounding frame, and performing coordinate transformation on the key point coordinates according to the scaled image to be evaluated to obtain the coordinates to be evaluated.

[0039] Among them, the coordinates to be evaluated refer to the coordinate information obtained after the coordinates of the key points in the image to be evaluated are transformed. Specifically, scaling the image to be evaluated by the human body enclosing frame refers to the process of cropping the image to be evaluated according to the human body enclosing frame, retaining only the image portion within the human body enclosing frame; and then scaling the cropped image to be evaluated according to a preset standard size. It can be understood that the image to be evaluated after the scaling process only includes the image portion within the human body enclosing frame. The size of the image to be evaluated after the scaling process may be the same as or different from the size of the image to be evaluated before the scaling process. For example: first crop the image to be evaluated according to the human body enclosing frame, crop the portion outside the human body enclosing frame in the image to be evaluated, and then scale the cropped image to be evaluated to an image of the same size as the image to be evaluated before cropping.

[0040] Furthermore, the coordinates of the key points in the image to be evaluated are transformed in the same coordinates according to the image to be evaluated after scaling, that is, the corresponding key point coordinates are transformed in the same coordinates according to the same scaling ratio as the image to be evaluated to obtain the coordinates to be evaluated. For example, the key point coordinates of the key point A of the human body in the image to be evaluated are (x 1 ,y 1 ), if the cropped image to be evaluated is enlarged by 2 times, the coordinates of the key point A of the human body in the image to be evaluated are (2x 1 ,2y 1 ).

[0041] S14: Acquire standard comparison data, where the standard comparison data includes standard coordinates and the confidence level corresponding to each standard coordinate.

[0042] Wherein, standard comparison data refers to standard data collected in advance for evaluating whether the human posture in the image to be evaluated meets the requirements. For example: the standard comparison data can be a human posture reference diagram of a fitness coach, or a human posture reference diagram that meets the requirements after being evaluated and screened by a human posture estimation network. It should be noted that the standard comparison data obtained in this step belongs to the same posture type as the human posture in the image to be evaluated. Specifically, the fitness video of the fitness coach can be input into the preset human posture estimation network in advance, and then the output multiple human posture reference diagrams and the standard coordinates included in each human posture reference diagram, and the confidence corresponding to each standard coordinate are stored in the database of the server, and after the coordinates to be evaluated of the image to be evaluated are obtained, the standard comparison data corresponding to the image to be evaluated is directly obtained from the database.

[0043] Among them, the standard comparison data includes standard coordinates and the confidence level corresponding to each standard coordinate. The standard coordinate refers to the position information corresponding to each key point of the human body in the standard comparison data. Similarly, the standard coordinate can also be represented by a two-dimensional plane coordinate. The confidence level is used to indicate the probability value of each standard coordinate position being correct. It can be understood that the importance of each standard coordinate can be judged according to the confidence level corresponding to each standard coordinate. For example: the standard comparison data includes the standard coordinate (x 1 ,y 1 ) and the corresponding confidence PA, standard coordinates (x 2 ,y 2 ) and the corresponding confidence level PB, etc. If PA>PB, it means that the standard coordinate (x 1 ,y 1 ) is more important than the standard coordinate (x 2 ,y 2 ).

[0044] S15: Calculate the similarity between the coordinates to be evaluated and the standard coordinates through the confidence level to obtain evaluation information of the image to be evaluated.

[0045] Among them, the evaluation information refers to information used to evaluate the accuracy of the user posture in the image to be evaluated. Preferably, the evaluation information can be a specific evaluation score, and the higher the evaluation score, the more accurate the user posture in the image to be evaluated. Specifically, each standard coordinate can be weighted according to the confidence corresponding to each standard coordinate, and a higher weight is set for the standard coordinate with a high confidence, and a lower weight is set for the standard coordinate with a low confidence; then a similarity algorithm, such as a cosine similarity algorithm, is used to calculate the similarity between each coordinate to be evaluated and the corresponding standard coordinate to obtain an initial similarity value, and finally all the initial similarity values ​​are weighted and calculated according to the weight value corresponding to each standard coordinate, so as to obtain the evaluation information of the image to be evaluated.

[0046] Preferably, an evaluation calculation formula may be defined in advance according to the confidence and similarity algorithms, and then the evaluation calculation formula is directly used to calculate the similarity between each coordinate to be evaluated and the corresponding standard coordinate, thereby obtaining evaluation information of the image to be evaluated.

[0047] In this embodiment, an image to be evaluated is obtained, which is an image including a user's posture; the image to be evaluated is input into a preset human posture estimation network to obtain human key point data, which includes key point coordinates and a human body bounding box; the image to be evaluated is scaled by the human body bounding box, and the key point coordinates are transformed according to the scaled image to be evaluated to obtain the coordinates to be evaluated; standard comparison data is obtained, which includes standard coordinates and confidence levels corresponding to each standard coordinate; the similarity between the coordinates to be evaluated and the standard coordinates is calculated by the confidence level to obtain evaluation information of the image to be evaluated; and by evaluating and analyzing the similarity between each coordinate to be evaluated and the standard coordinates in the image to be evaluated, a rapid and accurate evaluation of the user's human posture is achieved, thereby facilitating the provision of accurate movement guidance and targeted feedback to the user.

[0048] In one embodiment, if Figure 3 As shown, the similarity between the coordinates to be evaluated and the standard coordinates is calculated by the confidence level to obtain the evaluation information of the image to be evaluated, which specifically includes the following steps:

[0049] S151: Calculate the similarity between the coordinate to be evaluated and the standard coordinate using the following formula:

[0050]

[0051] Wherein, D(F,G) is the similarity between the coordinates to be evaluated and the standard coordinates, is the confidence level corresponding to the kth standard coordinate, is the kth coordinate to be evaluated among the coordinates to be evaluated, is the kth standard coordinate in the standard coordinates, and K is the number of standard coordinates.

[0052] Specifically, is the distance between the kth coordinate to be evaluated and the kth standard coordinate, then, the distance between the kth coordinate to be evaluated and the kth standard coordinate is multiplied by the confidence corresponding to the kth standard coordinate, and the similarity between the kth coordinate to be evaluated and the kth standard coordinate can be obtained. It can be understood that the above formula set in advance can be directly used to obtain the similarity between all the coordinates to be evaluated and the corresponding standard coordinates in the image to be evaluated. It can be understood that the similarity between the coordinates to be evaluated and the standard coordinates can be represented by a specific numerical value, and a similarity of 1 means that the coordinates to be evaluated and the standard coordinates are exactly the same. For example: the similarity between the coordinates to be evaluated and the standard coordinates can be 0.8, 0.85 or 0.9, etc.

[0053] S152: Convert the similarity to obtain evaluation information of the image to be evaluated.

[0054] Specifically, after determining the similarity between the coordinates to be evaluated and the standard coordinates, the similarity can be converted according to a preset conversion rule to obtain the evaluation information of the image to be evaluated. Preferably, the conversion rule can be to first convert the similarity between the coordinates to be evaluated and the standard coordinates into a specific evaluation score, and then give specific evaluation suggestions based on the evaluation score, so as to obtain the evaluation information of the image to be evaluated. For example: if the similarity between the coordinates to be evaluated and the standard coordinates is 0.9, the corresponding evaluation score can be 90 points.

[0055] In this embodiment, the similarity between the coordinates to be evaluated and the standard coordinates is calculated by the following formula:

[0056]

[0057] Wherein, D(F,G) is the similarity between the coordinates to be evaluated and the standard coordinates, is the confidence level corresponding to the kth standard coordinate, is the kth coordinate to be evaluated among the coordinates to be evaluated, is the kth standard coordinate in the standard coordinates, and K is the number of standard coordinates; the similarity is transformed to obtain the evaluation information of the image to be evaluated; thereby improving the accuracy of the generated evaluation information of the image to be evaluated.

[0058] In one embodiment, if Figure 4 As shown, the image to be evaluated is scaled by a human body bounding box, and the coordinates of the key points are transformed according to the scaled image to be evaluated to obtain the coordinates to be evaluated, which specifically includes the following steps:

[0059] S131: Cropping the image to be evaluated using the human body bounding box, and scaling the cropped image to be evaluated according to a preset standard size.

[0060] Specifically, cropping the image to be evaluated by the human body surrounding frame refers to the process of cropping the outer part of the human body surrounding frame in the image to be evaluated and retaining only the inner part of the human body surrounding frame. Specifically, an image cropping tool can be used to implement the cropping process of the image to be evaluated. Optionally, the image cropping tool can be a jQuery Jcrop image cropping tool or a FOTOE image cropping tool. Preferably, the image segmentation algorithm of opencv can also be used to automatically implement the cropping of the image to be evaluated.

[0061] Among them, the preset standard size refers to a preset standard image size. For example: the preset standard size can be 600*600, 750*750 or 800*800, etc. Preferably, in this embodiment, in order to improve the subsequent evaluation accuracy, the standard size is set to the same size as the image size of the image to be evaluated before cropping. Specifically, an image scaling algorithm can be used to scale the cropped image to be evaluated; or an image scaling tool can be used to scale the cropped image to be evaluated to obtain an image to be evaluated that is the same size as the preset standard size. Optionally, the image scaling algorithm can be a bilinear interpolation algorithm or a trilinear convolution interpolation algorithm. The image scaling tool can be photoshop, iResizer or FastStone Photo Resizer.

[0062] S132: Transform the coordinates of the key points by using scaling parameters.

[0063] Among them, the scaling parameter refers to the parameter obtained by proportionally converting the standard size to the image size of the cropped image to be evaluated. For example: if the image size of the cropped image to be evaluated is 600*600, and the standard size is 800*1000; the scaling parameter obtained is (4 / 3,5 / 3); wherein 4 / 3 is the scaling ratio in the x-axis direction, and 5 / 3 is the scaling ratio in the y-axis direction. Specifically, the coordinates of the key points are transformed in the same proportion by the obtained scaling parameters. For example: if the key point coordinates are (12,15), and the scaling parameter is (4 / 3,5 / 3), the key point coordinates after the coordinate transformation are (16,25), that is, the value 12 in the x-axis direction of the key point coordinates is scaled by 4 / 3 times, and the value 15 in the y-axis direction of the key point coordinates is scaled by 5 / 3 times.

[0064] S133: performing L1 normalization or L2 normalization processing on the coordinates of the key points after the coordinate transformation to obtain coordinates to be evaluated.

[0065] Specifically, the coordinates of the key points after the coordinate transformation are regarded as a vector array, and then the vectors in the vector array are normalized by L1 or L2 to obtain the coordinates to be evaluated. Performing L1 normalization on the vectors in the vector array means scaling the vectors in the vector array to unit norm, and performing L2 normalization on the vectors in the vector array means unifying each vector in the vector array and summing them, and the result will be 1. The user can choose any normalization method according to the actual situation, and this solution does not make specific restrictions. In this step, the accuracy of the generated coordinates to be evaluated is guaranteed by performing L1 or L2 normalization on the key point coordinates after the coordinate transformation.

[0066] In this embodiment, the image to be evaluated is cropped by a human body surrounding frame, and the cropped image to be evaluated is scaled according to a preset standard size to generate scaling parameters; the coordinates of the key points are transformed by the scaling parameters; the coordinates of the key points after the coordinate transformation are normalized by L1 or L2 to obtain the coordinates to be evaluated; thereby ensuring the accuracy of the generated coordinates to be evaluated, and further improving the accuracy of subsequent similarity calculations using the coordinates to be evaluated and the standard data.

[0067] In one embodiment, if Figure 5 As shown, a human body posture assessment method is provided, and the method is applied in Figure 1 The server in the example is used as an example to illustrate the following steps:

[0068] S21: Obtaining video data to be processed, where the video data to be processed is video data including user gestures recorded by a video acquisition device.

[0069] The video data to be processed is the original video data to be processed. Specifically, the video data to be processed is the video data including the user's posture recorded by the video acquisition device. The video acquisition device sends the recorded video data to be processed to the server, and the server obtains the video data to be processed.

[0070] S22: extracting a to-be-evaluated image set from the to-be-processed video data according to a preset time node, where the to-be-evaluated image set includes N to-be-evaluated images.

[0071] Among them, the preset time node refers to the time point preset for extracting the image to be evaluated from the video data to be processed. For example, the time node can be 1 minute 23 seconds, 2 minutes 23 seconds, and 2 minutes 23 seconds, etc. It can be understood that the preset time node can be one or more. Preferably, a time node can be set in the starting stage, the middle stage, and the final stage of the video data to be processed, respectively. Specifically, according to the preset time node, the image to be evaluated corresponding to each time node is extracted from the video data to be processed, and then, the image to be evaluated corresponding to each time node is composed of an image set to be evaluated, and the image set to be evaluated includes N images to be evaluated.

[0072] Specifically, the filter function in FFmpeg can be used to realize the image extraction of the video data to be processed. Among them, FFmpeg is a set of open source computer programs that can be used to record, convert digital audio and video, and convert them into streams. The crop function in the filter is used to realize the image extraction of the video data to be processed. Specifically, the image capture of the video data to be processed is realized by crop=width:height:x:y. Preferably, in order to avoid the distortion or blurring of the image to be evaluated corresponding to a certain time node extracted, in this embodiment, there are at least two images to be evaluated corresponding to each time node extracted from the video data to be processed.

[0073] S23: using a human body posture evaluation method to evaluate each image to be evaluated in the image set to be evaluated, and obtaining evaluation information of each image to be evaluated.

[0074] Specifically, by adopting the human body posture evaluation method in the above embodiment, each image to be evaluated in the image set to be evaluated is evaluated, and evaluation information of each image to be evaluated can be obtained. No redundant description is given here.

[0075] S24: Calculate the evaluation score of the video data to be processed according to the evaluation information of each image to be evaluated.

[0076] Specifically, after determining the evaluation information of each image to be evaluated, the evaluation information of each image to be evaluated is integrated and processed to obtain the evaluation score of the video data to be processed. Since the evaluation information of each image to be evaluated includes a corresponding evaluation score, in this step, the evaluation scores in the evaluation information of each image to be evaluated are statistically summed and then averaged to obtain the evaluation score of the video data to be processed. Preferably, each image to be evaluated can also be weighted in advance, and then according to the weight value of each image to be evaluated, the evaluation scores in the evaluation information of each image to be evaluated are weighted and statistically summed and then averaged to obtain the evaluation score of the video data to be processed.

[0077] In this embodiment, by acquiring video data to be processed, the video data to be processed is video data including user posture recorded by a video acquisition device; extracting an image set to be evaluated from the video data to be processed according to a preset time node, the image set to be evaluated includes N images to be evaluated; adopting a human body posture evaluation method, evaluating each image to be evaluated in the image set to be evaluated, and obtaining evaluation information of each image to be evaluated; calculating an evaluation score of the video data to be processed according to the evaluation information of each image to be evaluated; by extracting the image to be evaluated in the video data to be processed, and then calculating the evaluation score of the video data to be processed according to the evaluation information of the image to be evaluated, thereby achieving rapid and accurate evaluation of the user's human body posture in the video data to be processed.

[0078] In one embodiment, the time node includes M sub-time nodes, each sub-time node corresponds to at least one image to be evaluated;

[0079] Extracting the image set to be evaluated from the video data to be processed according to the preset time node specifically includes the following steps:

[0080] A preset number of images to be evaluated are extracted from the video data to be processed according to each sub-time node, and the images to be evaluated corresponding to each sub-time node are combined into an image set to be evaluated.

[0081] The preset number refers to the number of images to be evaluated corresponding to each sub-time node extracted from the video data to be processed. In this embodiment, the preset number is greater than or equal to 2. It can be understood that the more preset numbers, that is, the more images to be evaluated, the higher the accuracy of the subsequent evaluation of the video data to be processed, but the computational complexity of the server will also be higher. The specific number can be set according to the needs of different application scenarios. If the focus is on recognition accuracy, the preset number can be increased, and if the focus is on recognition efficiency, the preset number can be appropriately reduced.

[0082] Specifically, the time node includes M sub-time nodes, each of which corresponds to at least one image to be evaluated. According to each sub-time node, a preset number of images to be evaluated are extracted from the video data to be processed, and then the images to be evaluated corresponding to each sub-time node are combined into an image set to be evaluated.

[0083] In one embodiment, if Figure 6 As shown, according to the evaluation information of each image to be evaluated, the evaluation score of the video data to be processed is calculated, which specifically includes the following steps:

[0084] S241: Determine target evaluation information from a preset number of to-be-evaluated images corresponding to each sub-time node, wherein the target evaluation information is evaluation information indicating the highest similarity with the corresponding standard coordinates.

[0085] Specifically, since there are at least two images to be evaluated corresponding to each sub-time node, the most representative image to be evaluated can be selected from the preset number of images to be evaluated corresponding to each sub-time node as the target evaluation image, and the evaluation information corresponding to each target evaluation image can be determined as the target evaluation information. The target evaluation information is the evaluation information indicating the highest similarity with the corresponding standard coordinates. Specifically, the selection of the target evaluation image from the preset number of images to be evaluated corresponding to each sub-time node can be achieved by pre-training the corresponding neural network model to obtain a posture recognition model. That is, a large amount of image data representing different postures is annotated and then input into a neural network model for training, so as to obtain a posture recognition model.

[0086] S242: Calculate the evaluation score of the video data to be processed according to the target evaluation information corresponding to each sub-time node.

[0087] Specifically, after determining the target evaluation information corresponding to each sub-time node, the target evaluation information corresponding to each sub-time node is integrated and processed to obtain the evaluation score of the video data to be processed. It can be understood that since each target evaluation information includes a corresponding evaluation score, in this step, the evaluation score in each target evaluation information is statistically summed and then averaged to obtain the evaluation score of the video data to be processed.

[0088] In this embodiment, target evaluation information is determined from a preset number of images to be evaluated corresponding to each sub-time node, wherein the target evaluation information is evaluation information indicating the highest similarity with the corresponding standard coordinates; based on the target evaluation information corresponding to each sub-time node, an evaluation score of the video data to be processed is calculated; thereby further improving the accuracy of the calculated evaluation score of the video data to be processed.

[0089] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0090] In one embodiment, a human body posture assessment device is provided, which corresponds one-to-one to the human body posture assessment method in the above embodiment. Figure 7 As shown, the human posture evaluation device includes an image acquisition module 11 to be evaluated, an input module 12, a scaling processing module 13, a standard comparison data acquisition module 14 and a similarity calculation module 15. The functional modules are described in detail as follows:

[0091] An image acquisition module 11 for acquiring an image to be evaluated, wherein the image to be evaluated is an image including a user's posture;

[0092] An input module 12 is used to input the image to be evaluated into a preset human posture estimation network to obtain human key point data, which includes key point coordinates and a human body surrounding frame;

[0093] A scaling processing module 13 is used to scale the image to be evaluated through the human body surrounding frame, and to perform coordinate transformation on the key point coordinates according to the scaled image to be evaluated to obtain the coordinates to be evaluated;

[0094] A standard comparison data acquisition module 14 is used to acquire standard comparison data, where the standard comparison data includes standard coordinates and the confidence level corresponding to each standard coordinate;

[0095] The similarity calculation module 15 is used to calculate the similarity between the coordinates to be evaluated and the standard coordinates through confidence, so as to obtain evaluation information of the image to be evaluated.

[0096] Preferably, if Figure 8 As shown, the similarity calculation module 15 includes:

[0097] The similarity calculation unit 151 is used to calculate the similarity between the coordinates to be evaluated and the standard coordinates using the following formula:

[0098]

[0099] Among them, D(F,G) is the similarity between the coordinates to be evaluated and the standard coordinates, is the confidence level corresponding to the kth standard coordinate, is the kth coordinate to be evaluated among the coordinates to be evaluated, is the kth standard coordinate in the standard coordinates, K is the number of standard coordinates;

[0100] The conversion unit 152 is used to convert the similarity to obtain evaluation information of the image to be evaluated.

[0101] Preferably, the scaling processing module 13 includes:

[0102] A cropping and scaling unit, used to crop the image to be evaluated by using the human body surrounding frame, and scale the cropped image to be evaluated according to a preset standard size;

[0103] A coordinate transformation unit, used for transforming the coordinates of key points by scaling parameters;

[0104] The normalization processing unit is used to perform L1 normalization or L2 normalization processing on the coordinates of the key points after the coordinate transformation to obtain the coordinates to be evaluated.

[0105] Preferably, the human body posture assessment device further comprises:

[0106] A module for acquiring video data to be processed, used to acquire video data to be processed, where the video data to be processed is video data including user gestures recorded by a video acquisition device;

[0107] An extraction module is used to extract a set of images to be evaluated from the video data to be processed according to a preset time node, where the set of images to be evaluated includes N images to be evaluated;

[0108] An evaluation module is used to evaluate each image to be evaluated in the image set to be evaluated by using a human posture evaluation method to obtain evaluation information of each image to be evaluated;

[0109] The evaluation score calculation module is used to calculate the evaluation score of the video data to be processed according to the evaluation information of each image to be evaluated.

[0110] Preferably, the extraction module comprises:

[0111] The extraction unit is used to extract a preset number of images to be evaluated from the video data to be processed according to each sub-time node, and form an image set to be evaluated by combining the images to be evaluated corresponding to each sub-time node.

[0112] Preferably, the evaluation score calculation module includes:

[0113] A target evaluation information determination unit, configured to determine target evaluation information from a preset number of to-be-evaluated images corresponding to each sub-time node, wherein the target evaluation information is evaluation information indicating the highest similarity with the corresponding standard coordinates;

[0114] The evaluation score calculation unit is used to calculate the evaluation score of the video data to be processed according to the target evaluation information corresponding to each sub-time node.

[0115] The specific definition of the human posture assessment device can be found in the definition of the human posture assessment method above, which will not be repeated here. Each module in the above-mentioned human posture assessment device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0116] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data used in the human body posture assessment method of the above embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a human body posture assessment method is implemented.

[0117] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the human body posture assessment method in the above embodiment is implemented.

[0118] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the human body posture assessment method in the above embodiment is implemented.

[0119] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0120] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0121] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A human body posture assessment method, characterized in that: include: Acquire an image to be evaluated, wherein the image to be evaluated is an image including a user's posture; Input the image to be evaluated into a preset human posture estimation network to obtain human key point data, wherein the human key point data includes key point coordinates and a human body surrounding frame, and the key point coordinates are coordinate positions of human posture key points identified from the image to be evaluated; Scaling the image to be evaluated by using the human body surrounding frame, and performing coordinate transformation on the key point coordinates according to the scaled image to be evaluated to obtain coordinates to be evaluated; Acquire standard comparison data, wherein the standard comparison data includes standard coordinates and a confidence level corresponding to each of the standard coordinates; The similarity between the coordinates to be evaluated and the standard coordinates is calculated by using the confidence level to obtain evaluation information of the image to be evaluated.

2. The human body posture assessment method according to claim 1, wherein: The calculating the similarity between the coordinates to be evaluated and the standard coordinates by using the confidence level to obtain evaluation information of the image to be evaluated includes: The similarity between the coordinates to be evaluated and the standard coordinates is calculated by the following formula: ； in, is the similarity between the coordinate to be evaluated and the standard coordinate, is the confidence level corresponding to the kth standard coordinate, is the kth coordinate to be evaluated among the coordinates to be evaluated, is the kth standard coordinate in the standard coordinates, K is the number of standard coordinates; The similarity is transformed to obtain evaluation information of the image to be evaluated.

3. The human body posture assessment method according to claim 1, wherein: The step of scaling the image to be evaluated by using the human body surrounding frame, and performing coordinate transformation on the key point coordinates according to the scaled image to be evaluated to obtain the coordinates to be evaluated includes: Cropping the image to be evaluated using the human body surrounding frame, and scaling the cropped image to be evaluated according to a preset standard size; Performing coordinate transformation on the key point coordinates by scaling parameters; The coordinates of the key points after coordinate transformation are subjected to L1 normalization or L2 normalization processing to obtain coordinates to be evaluated.

4. A human body posture assessment method, characterized in that: include: Acquire video data to be processed, where the video data to be processed is video data including user gestures recorded by a video acquisition device; Extracting a to-be-evaluated image set from the to-be-processed video data according to a preset time node, wherein the to-be-evaluated image set includes N to-be-evaluated images; Using the human body posture evaluation method according to any one of claims 1 to 3, each image to be evaluated in the image set to be evaluated is evaluated to obtain evaluation information of each image to be evaluated; An evaluation score of the video data to be processed is calculated according to the evaluation information of each of the images to be evaluated.

5. The human body posture assessment method according to claim 4, characterized in that: The time node includes M sub-time nodes, each of which corresponds to at least one image to be evaluated; The step of extracting a set of images to be evaluated from the video data to be processed according to a preset time node includes: A preset number of images to be evaluated are extracted from the video data to be processed according to each sub-time node, and the images to be evaluated corresponding to each sub-time node are combined into an image set to be evaluated.

6. The human body posture assessment method according to claim 5, characterized in that: The step of calculating the evaluation score of the video data to be processed according to the evaluation information of each of the images to be evaluated includes: Determine target evaluation information from a preset number of to-be-evaluated images corresponding to each sub-time node, wherein the target evaluation information is evaluation information indicating the highest similarity with the corresponding standard coordinates; The evaluation score of the video data to be processed is calculated according to the target evaluation information corresponding to each sub-time node.

7. A human body posture assessment device, characterized in that: include: An image acquisition module to be evaluated, used to acquire an image to be evaluated, wherein the image to be evaluated is an image including a user's posture; An input module, used to input the image to be evaluated into a preset human posture estimation network to obtain human key point data, wherein the human key point data includes key point coordinates and a human body surrounding frame, and the key point coordinates are coordinate positions of human posture key points identified from the image to be evaluated; A scaling processing module, used to scale the image to be evaluated by using the human body surrounding frame, and to perform coordinate transformation on the key point coordinates according to the scaled image to be evaluated to obtain coordinates to be evaluated; A standard comparison data acquisition module, used to acquire standard comparison data, wherein the standard comparison data includes standard coordinates and a confidence level corresponding to each standard coordinate; The similarity calculation module is used to calculate the similarity between the coordinates to be evaluated and the standard coordinates through the confidence level to obtain evaluation information of the image to be evaluated.

8. The human body posture assessment device according to claim 7, characterized in that: The similarity calculation module comprises: The similarity calculation unit is used to calculate the similarity between the coordinate to be evaluated and the standard coordinate by the following formula: ； in, is the similarity between the coordinate to be evaluated and the standard coordinate, is the confidence level corresponding to the kth standard coordinate, is the kth coordinate to be evaluated among the coordinates to be evaluated, is the kth standard coordinate in the standard coordinates, K is the number of standard coordinates; The conversion unit is used to convert the similarity to obtain evaluation information of the image to be evaluated.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the human body posture assessment method as described in any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the human body posture assessment method as claimed in any one of claims 1 to 6 is implemented.

Images