Video processing method, device, equipment and storage medium

Abstract

The present invention discloses a video processing method, device, equipment and storage medium. The method comprises: obtaining a detection frame image containing a target object in a running video, and running posture data associated with the detection frame image; wherein the running posture data comprises: a regression position of a posture point of the target object, a posture point heat map and a leg orientation relationship; according to the detection frame image and the running posture data, correcting the leg orientation relationship in the running posture data; and according to the corrected running posture data, determining the key moment of the running process of the target object. The scheme provided by the present invention can ensure the accuracy of the finally determined running posture data by correcting the acquired running posture data, so as to facilitate the subsequent running posture state analysis based on the running posture data.

Description

Technical Field

[0001] The embodiments of the present invention relate to computer technology, and in particular to a video processing method, apparatus, device and storage medium. Background Art

[0002] With the continuous development of video processing technology, related target detection algorithms and posture recognition algorithms have also emerged in an endless stream. In particular, the recognition of the running posture of the target object in the running video has been widely used. However, the existing posture recognition algorithm is prone to recognition errors when obtaining running posture data, such as left and right recognition errors, direction of travel errors, and position recognition errors. Therefore, how to more effectively ensure the accuracy of running posture data is a problem that needs to be solved urgently. Summary of the invention

[0003] The present invention provides a video processing method, device, equipment and storage medium. By correcting the acquired running posture data, the accuracy of the finally determined running posture data can be guaranteed, which facilitates the subsequent running posture state analysis based on the running posture data.

[0004] In a first aspect, an embodiment of the present invention provides a video processing method, the method comprising:

[0005] Acquire a detection frame image containing a target object in a running video, and running posture data associated with the detection frame image; wherein the running posture data includes: a posture point regression position of the target object, a posture point heat map, and a leg orientation relationship;

[0006] Correcting the leg orientation relationship in the running posture data according to the detection frame image and the running posture data;

[0007] According to the corrected running posture data, key moments of the target object's running process are determined.

[0008] In a second aspect, an embodiment of the present invention further provides a video processing device, the device comprising:

[0009] An acquisition module is used to acquire a detection frame image containing a target object in a running video, and running posture data associated with the detection frame image; wherein the running posture data includes: a posture point regression position of the target object, a posture point heat map, and a leg orientation relationship;

[0010] A correction module, used for correcting the leg orientation relationship in the running posture data according to the detection frame image and the running posture data;

[0011] The determination module is used to determine the key moments of the target object's running process according to the corrected running posture data.

[0012] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0013] one or more processors;

[0014] A memory for storing one or more programs;

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the video processing method provided by any embodiment of the present invention.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, wherein when the program is executed by a processor, the video processing method provided by any embodiment of the present invention is implemented.

[0017] After obtaining the detection frame image containing the target object in the running video, and the running posture data such as the posture point regression position, posture point heat map, and leg orientation relationship of the target object associated with the detection frame image, the embodiment of the present invention further corrects the leg orientation relationship in the running posture data according to the detection frame image and the running posture data, and finally determines the key moment of the target object's running process according to the corrected running posture data. By correcting the acquired running posture data, the accuracy of the finally determined running posture data can be guaranteed, which facilitates the subsequent running posture state analysis based on the running posture data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1A A flowchart of a video processing method provided in Embodiment 1 of the present invention;

[0019] Figure 1B A schematic diagram of the running phase provided in the first embodiment of the present invention;

[0020] Figure 2 A flowchart of a video processing method provided in Embodiment 2 of the present invention;

[0021] Figure 3 A structural block diagram of a video processing device provided in Embodiment 3 of the present invention;

[0022] Figure 4 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0023] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0024] Embodiment 1

[0025] Figure 1A A flowchart of a video processing method provided in Embodiment 1 of the present invention is provided. Figure 1B This is a schematic diagram of the running stage provided in the first embodiment of the present invention. This embodiment is applicable to the case where running posture data of a running video is obtained and corrected. The method can be executed by a video processing device, which can be implemented in software and / or hardware and can be integrated into an electronic device with a video processing function. Figure 1A As shown, the video processing method provided in this embodiment specifically includes:

[0026] S101, obtaining a detection frame image containing a target object in a running video, and running posture data associated with the detection frame image.

[0027] Among them, the running video refers to a video of a runner running, for example, it can be a video of a runner running shot in slow motion. The target object refers to the target running object in the running video. The detection frame image refers to the detection frame image of the area where the target object is located determined from each frame in the running video. The running posture data refers to the data used to characterize the running posture of the target object. Specifically, the running posture data includes: the regression position of the posture point of the target object, the posture point heat map and the leg orientation relationship. The regression position of the posture point of the target object refers to the regression position of each joint posture point of the target object in the detection frame image. The posture point can at least include joint posture points such as left and right hips, left and right knees, left and right ankles, left and right heels, and left and right toes. The posture point heat map refers to a heat map that characterizes the probability that each joint posture point is located at its corresponding regression position, that is, each posture point has a corresponding heat map. The leg orientation relationship refers to the leg orientation relationship information that characterizes the front and back and up and down of the left and right legs of the target object. For example, the leg orientation relationship may include relevant information about which leg of the target object is in front, which leg is behind, which leg is on top, and which leg is on the bottom in each frame of the running video.

[0028] Optionally, a common target detection algorithm, such as the Yolox target detection algorithm, can be used to perform target detection on the running video to obtain all target frame images containing the target objects in the running video detected by the target detection algorithm, and the obtained target frame images are further integrated in chronological order to determine the detection frame images containing the target objects in the running video, that is, the detection frame images containing the target objects in the running video are obtained.

[0029] It should be noted that, through the target detection algorithm, a target frame can be detected in each frame image of the running video containing the target object. At the next moment, the position of the target frame in the video image will change. The same target frame will continue to change in different video image frames, and a set of time-series motion trajectory frames can be formed, that is, the detection frame image containing the target object.

[0030] Optionally, the acquired target frame images are integrated in chronological order to determine a detection frame image containing the target object. One possible implementation method is to screen a group of acquired target frame images to ensure that the target runner is completely within the image. Specifically, each target frame image can be subjected to edge judgment, and the distance between the position and the edge of the image is judged based on the position of each target frame in the image determined by the target detection algorithm. If the distance is less than a certain threshold, it is considered that the target runner is not completely within the image, and the target frame image of that frame can be discarded. The screened group of target frame images can be integrated in chronological order to determine the detection frame image containing the target object.

[0031] Optionally, after obtaining the detection frame image containing the target object in the running video, a commonly used posture recognition algorithm, such as the pose algorithm provided in the Mediapipe open source project, can be used to estimate the human posture for each detection frame image, and the posture point regression position, posture point heat map and leg orientation relationship of the target object can be obtained, that is, the running posture data associated with the detection frame image can be obtained.

[0032] S102: Correct the leg orientation relationship in the running posture data according to the detection frame image and the running posture data.

[0033] Optionally, after determining the detection frame image and the running posture data, the detection frame image and the running posture data can be input into a pre-trained correction model to output the corrected running posture data for each frame of the detection frame image, and the corrected running posture data can be further used to replace the original running posture data, that is, the leg orientation relationship in the running posture data is corrected. Alternatively, based on the acquired detection frame image and running posture data, according to preset rules, it can be determined whether the leg orientation relationship in the running posture data associated with each detection frame image is incorrect. If so, the regression positions of the posture points contained in the two legs and the posture point heat maps are swapped, that is, the leg orientation relationship in the running posture data is corrected.

[0034] S103: Determine the key moments of the target object's running process according to the corrected running posture data.

[0035] The key moment refers to a key moment of contact, buffering, pushing off or taking off during the target object's running process. The target object's running process may include at least one running phase.

[0036] Optionally, the key moment may be when the running posture of the target object in the running video is touching the ground, buffering, pushing off or taking off, and the corresponding video frame is the frame sequence of all video frames in the entire running video. For example, the running video has a total of 1000 frames, and the key moment of touching the ground is the 25th video frame; it may also be when the running posture of the target object in the running video is touching the ground, buffering, pushing off or taking off, and the corresponding video frame is at the moment in the entire running video. For example, the running video is 5 minutes long, and the key moment of touching the ground is at the 3rd minute and 10 seconds.

[0037] See also Figure 1B The key moment of touchdown can be the moment of the detection frame image corresponding to the time when the y-axis value (position coordinate value on the y-axis) of the landing foot is the largest. The y-axis value of the landing foot can refer to the y-axis value of the toe posture point, heel posture point, or the midpoint of the line connecting the heel and the toe of the landing foot. The key moment of take-off refers to the moment of the detection frame image corresponding to the time when the y-axis value of the midpoint of the line connecting the left and right hip joints (or the two-point interval of the hip) is the smallest between every two touchdown moments. The key moment of buffering refers to the moment of the detection frame image corresponding to the time when the midpoint of the heel and toe of the landing foot of the target object and the midpoint of the left and right hip joints have the same x-axis value. The foot that leaves the ground refers to the foot whose toe has a tendency to move upward on the ground. The key moment of push-off refers to the moment when the foot that leaves the ground as a whole just leaves the ground.

[0038] Optionally, there are many ways to determine the critical moment of push-off. For example, when the y-axis speed of the toe posture point of the lifted foot (the landing foot at the previous moment is the lifted foot at the next moment) is not 0 and the maximum, the moment of the corresponding detection frame image can be taken as the critical moment of push-off; the critical moment of push-off can also be determined according to the speed of the toe posture point. The specific determination method is: 1. Calculate the toe speed at each moment between the buffer critical moment and the next take-off moment (sampling may or may not be performed at this time); 2. Traverse each frame of the detection frame image to determine whether the toe speed corresponding to the frame image exceeds a preset threshold. If so, the moment of the detection frame image of the frame is the critical moment of push-off.

[0039] It should be noted that see Figure 1B , from the side, the target object's two legs are separated at the critical moment of flight. At the critical moment of touchdown and push-off, the target object's two legs are in a non-overlapping state with one leg in front and the other leg behind. At the critical moment of cushioning, the target object's two legs may overlap or not overlap with a certain probability.

[0040] Optionally, the detection frame image corresponding to the time when the y-axis values ​​of all the posture points included in the landing foot are the maximum can be determined, and the moment corresponding to the detection frame image of this frame can be used as the touchdown moment. The touchdown moment can also be reversed based on the principle that the touchdown moment is between the buffering moment and the airborne moment. Specifically, the pixel speed can be calculated by discrete differential based on the pixel positions of the ankle, heel and toe of the landing foot, and the time point when the speed is not 0 can be reversely inferred from the support point (when the speeds of the three posture points all tend to 0), and this time point can be used as the touchdown moment. It should be noted that since we are looking for the moment corresponding to the landing foot just touching the ground, we need to determine the moment corresponding to the last posture point whose speed is not 0 when reversed from the buffering moment.

[0041] Optionally, if the running video is a slow-motion video (120-240 frames per second), the variation between frames may not be large, so the speed range can be enlarged by using interval sampling. Although the speed range is larger, it will not change when the speed approaches 0, so a more accurate speed change can be obtained.

[0042] Optionally, after determining the touchdown moment, the accuracy of the touchdown moment determined by S101-S103 can be verified by deep learning. Specifically, the verification steps can be: 1. Determine the touchdown moment through S101-S103. 2. Use the automatic cutout method of deep learning to determine the touchdown moment and the buffering moment, as well as the y-axis value of the posture point of the corresponding landing foot. 3. Determine the difference between the y-axis value of the landing foot at the touchdown moment and the buffering moment. If the difference is less than a preset threshold, the determined touchdown moment is considered reasonable. Otherwise, the determined touchdown moment is considered unreasonable and the sampling method can be changed to find it.

[0043] It should be noted that the key moments of the target object's running process determined by this implementation can facilitate the subsequent calculation of a series of angles formed between important posture points, and can also facilitate the study of the target object's running posture state in combination with kinematic analysis.

[0044] For example, see Figure 1B A running phase of the target object's running process may include a ground contact critical moment, a buffering critical moment, a push-off critical moment, and a take-off critical moment.

[0045] Optionally, the orientation relationship of the posture points in each frame of the detection frame image can be determined according to the corrected running posture data, directly based on the regression positions of the posture points in the corrected running posture data and the posture point heat map, and the orientation relationship of the posture points in each frame of the detection frame image can be compared with the orientation relationship of the posture points at key moments of touching the ground, buffering, pushing off and taking off defined in this embodiment, to determine the detection frame image corresponding to each key moment in the detection frame image, and finally to determine the key moments of the target object's running process.

[0046] It should be noted that this embodiment does not consider dividing the entire running process corresponding to the running video into multiple running stages. Therefore, for each key moment, at least one detection frame image can be determined, and each key moment can be further determined as the moment corresponding to at least one detection frame image, that is, each key moment of the target object's running process can be determined in multiple stages.

[0047] After obtaining the detection frame image containing the target object in the running video, and the running posture data such as the posture point regression position, posture point heat map, and leg orientation relationship of the target object associated with the detection frame image, the embodiment of the present invention further corrects the leg orientation relationship in the running posture data according to the detection frame image and the running posture data, and finally determines the key moment of the target object's running process according to the corrected running posture data. By correcting the acquired running posture data, the accuracy of the finally determined running posture data can be guaranteed, which facilitates the subsequent running posture state analysis based on the running posture data.

[0048] Optionally, based on the detection frame image and the running posture data, the leg orientation relationship in the running posture data is corrected, including: determining the movement direction of the target object based on the detection frame image and / or the running posture data; grouping the detection frame images according to the running posture data; and correcting the leg orientation relationship in the running posture data based on the movement direction, the grouping results and the left-right alternation principle.

[0049] The moving direction of the target object refers to the direction in which the target object runs in the running video, and specifically, may include running from the left side of the video image to the right side, and from the right side of the video image to the left side. The left-right alternation principle refers to the principle that the left and right legs of the target object run forward alternately during the running process.

[0050] Optionally, a feasible implementation method for determining the movement direction of the target object based on the detection frame image and / or the running posture data is as follows: determining the orientation of the nose posture point of the target object based on the running posture data, and further determining the movement direction of the target object based on the determined orientation of the nose posture point. Specifically, the movement direction of the target object can be determined based on the azimuth relationship between the nose position and the trunk position (i.e., the line connecting the midpoint of the hip joint and the midpoint of the shoulder), that is, based on whether the nose position is located on the left or right side of the trunk line. If the nose is located on the left side of the trunk line, it indicates that the target object is running from the right side of the video image to the left side. Otherwise, it indicates that the target object is running from the left side of the video image to the right side. It is also possible to determine the movement direction of the target object based on preset calculation rules, such as determining two vectors formed by the nose position to the midpoint of the hip joint and the midpoint of the shoulder, and determining the azimuth relationship between the nose position and the trunk position based on the cross product calculation method.

[0051] Optionally, another possible implementation method for determining the movement direction of the target object based on the detection frame image and / or running posture data is: based on the detection frame image, determine the position interval of the relevant posture points of the first frame detection frame image in the time sequence (such as the position interval of the posture points included in the left leg), and the position interval of the relevant posture points of the subsequent frame detection frame image in the time sequence (such as the second frame), and further compare the changes in the position intervals of the same relevant posture points in the two frames of detection frame images in the time sequence to determine the movement direction of the target object. For example, if the position interval of the relevant posture points of the first frame detection frame image is greater than the position interval value of the second frame, it can be determined that the movement direction of the target object is running from the right side of the video image to the left side.

[0052] Optionally, another possible implementation method for determining the movement direction of the target object based on the detection frame image and / or running posture data is as follows: based on the detection frame image and the running posture data, based on the above two methods of determining the movement direction of the target object based on the detection frame image or the running posture data respectively, the corresponding movement directions are determined respectively; if the movement directions determined by the above two methods are the same, then the movement direction of the final target object is determined to be the corresponding determined movement direction; if the movement directions determined by the above two methods are different, then the above two methods can be used again to determine the movement directions respectively until the same movement direction is determined as the movement direction of the final target object.

[0053] Optionally, the detection frame images are grouped according to the running posture data, including: determining the double-leg overlapping state corresponding to the detection frame images according to the running posture data; and taking the detection frame images with the same double-leg overlapping state as a group of detection frame sequences.

[0054] The double leg overlapping state refers to whether the left and right legs of the target object overlap. The double leg overlapping state includes overlapping and non-overlapping.

[0055] Specifically, for each frame of the detection frame image, the position interval of the posture points contained in each leg can be determined according to the regression position of the posture points of the target object in the associated running posture data, and the overlapping state of the two legs corresponding to the detection frame image can be further determined according to the relationship between the position intervals of the two legs. Specifically, for the left and right legs, the positions of the knee, toe, heel and ankle joint posture points contained in the leg can be taken as the union, and the position intervals of the left and right legs can be determined respectively. If the position intervals of the left and right legs do not overlap at all, the overlapping state of the two legs of the detection frame image is non-overlapping, otherwise, it can be considered that the overlapping state of the two legs of the detection frame image is overlapping. After determining the overlapping state of the two legs of each frame of the detection frame image, the detection frame images with the non-overlapping overlapping state of the two legs can be used as a group of detection frame sequences, and the detection frame images with the overlapping overlapping state of the two legs can be used as a group of detection frame sequences, that is, the detection frame images with the same overlapping state of the two legs can be used as a group of detection frame sequences.

[0056] Optionally, the leg orientation relationship in the running posture data is corrected according to the movement direction, the grouping result and the left-right alternating principle, including: performing a first correction processing on the leg orientation relationship in the running posture data associated with the first group of detection frame sequences according to the movement direction and the left-right alternating principle; performing a second correction processing on the leg orientation relationship in the running posture data associated with the second group of detection frame sequences according to the movement direction, the left-right alternating principle and the corrected running posture data associated with the first group of detection frame sequences.

[0057] Among them, the overlapping state of the two legs corresponding to the first group of detection frame sequences is non-overlapping; the overlapping state of the two legs corresponding to the second group of detection frame sequences is overlapping. The first correction process refers to the correction process of the left-right relationship of the two legs in the leg orientation relationship. The second correction process can be the same as the first correction process, or it can be a correction process of correcting the up-down relationship of the two legs in the leg orientation relationship on the basis of the left-right relationship correction process. The left-right alternation principle refers to the running principle followed by the target object during the running process. Specifically, the running process of the target object can be divided into several running stages, and each running stage can include a state a, b state, A state and B state of the time sequence, wherein the a state is: the state of the left foot overlapping on the top, the b state is: the state of the left foot in front (referring to the front of the body) without overlapping, the A state is: the state of the right foot overlapping on the top, and the B state is: the state of the right foot in front without overlapping.

[0058] Specifically, according to the direction of movement and the left-right alternation principle, a possible implementation method for performing the first correction processing on the leg orientation relationship in the running posture data associated with the first group of detection frame sequences is: for the first group of detection frame images and the second group of detection frame images, first analyze the running posture data associated with the detection frame images according to the direction of movement and the left-right alternation principle to determine at least one running stage of the time sequence, each running stage includes at least two frames of the first group of detection frame images and at least two frames of the second group of detection frame images, and for each running stage, it should be: first, a second group of detection frame images in state a, then a first group of detection frame images in state b, then a second group of detection frame images in state A, and then a first group of detection frame images in state B. That is, the left and right legs in front of two adjacent first-group detection frame images should be interchanged. If it is detected that the left and right legs in front of two adjacent first-group detection frame images do not conform to the rules of each running stage mentioned above, it is considered that the leg orientation relationship of the second first-group detection frame image is incorrectly determined (the left and right legs are recognized oppositely), and the posture point regression positions and posture point heat maps of the left and right legs in the second first-group detection frame image are further swapped, that is, the leg orientation relationship in the running posture data associated with the first-group detection frame sequence is subjected to the first correction processing.

[0059] It should be noted that the above-mentioned correction of the leg orientation relationship may include the correction of the left-right orientation relationship of the knees, toes, heels and ankles of the left and right legs. After determining the accurate left-right orientation relationship of the knees, toes, heels and ankles of the left and right legs, the correct left-right orientation relationship of the hip joint posture point can be further determined based on the orientation angle relationship between the various posture points of the human body.

[0060] Optionally, after the first correction processing is performed on the leg orientation relationship in the running posture data associated with the first group of detection frame sequences, the second correction processing can be further performed on the leg orientation relationship in the running posture data associated with the second group of detection frame sequences according to the motion direction, the left-right alternation principle and the corrected running posture data associated with the first group of detection frame sequences. Specifically, only the upper and lower relationship (the numerical value relationship in the y-axis direction) of the position intervals of the ankle, heel and toe joint posture points in the running posture data associated with the second group of detection frame sequences can be analyzed to determine whether the corrected first group of detection frame sequences and the second group of detection frame sequences meet: the running phase timing sequence determined according to the motion direction and the left-right alternation principle, for example, the sequence of state a, state b, state A and state B. If it meets, the second correction processing is not performed. If it does not meet, the regression positions of the posture points of the left and right legs in the erroneous second group of detection frame images and the posture point heat map are swapped, that is, the second correction processing is performed on the leg orientation relationship in the running posture data associated with the second group of detection frame sequences.

[0061] Optionally, the left-right relationship between the left and right legs of the running posture data associated with the second group of detection frame sequences can also be corrected. Specifically, the left-right leg orientation relationship of each second group of detection frames is determined based on the left-right relationship between the left-right leg orientation relationships of its two adjacent first group of detection frames.

[0062] For example, if it is found that the first, second and third images in the time sequence are: the first group of detection frames with the left leg in front, the second group of detection frames with the left and right legs overlapping, and the first group of detection frames with the right foot in front, it means that the state of the target object at this time is a running state with the left foot on the ground and the right foot stretched forward.

[0063] Embodiment 2

[0064] Figure 2 This is a flow chart of a video processing method provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment further explains in detail how to correct the regression position of the posture point of the target object in the running posture data before determining the key moment of the target object's running process based on the corrected running posture data, such as Figure 2 As shown, the video processing method provided in this embodiment specifically includes:

[0065] S201, obtaining a detection frame image containing a target object in a running video, and running posture data associated with the detection frame image.

[0066] S202: Correct the leg orientation relationship in the running posture data according to the detection frame image and the running posture data.

[0067] S203, correcting the regression position of the posture point of the target object in the running posture data according to the regression position of the posture point of the target object in the running posture data and the posture point heat map.

[0068] Optionally, the left and right hips, left and right knees, left and right ankles, left and right heels, and left and right toes joint posture points can be used as important posture points for regression position correction, and the regression positions of the remaining posture points are based on the regression positions of the posture points in the running posture data associated with the acquired detection frame image; the regression positions of all posture points corresponding to the acquired running posture data can also be corrected, and this embodiment does not impose any restrictions on this.

[0069] Optionally, the regression positions of important posture points in each frame detection frame image may be iteratively updated, that is, the regression positions of the posture points of the target object in the running posture data may be corrected. The specific iterative updating method is: for the regression positions of important posture points in the first frame detection frame image, the regression positions of the posture points determined in the running posture data may be directly used as the standard; for the regression positions of important posture points in the second frame detection frame image, they may be determined based on the regression positions of corresponding posture points in the first frame image and the heat map of posture points in the second frame image. For example, based on the relationship between the regression positions of the corresponding posture points finally determined in the first frame image and multiple candidate positions with different probabilities determined by the heat map of posture points in the second frame image, the candidate regression position closest to the final regression position determined in the first frame image among the candidate positions is used as the final regression position of the corresponding posture point in the second frame image.

[0070] It should be noted that the regression position finally determined for each posture point in each frame of the image may be the regression position given by the running posture data (the regression position with the highest probability in the heat map), or it may be the regression position selected from at least one candidate regression position of the heat map according to the above-mentioned iterative update method.

[0071] It should be noted that for each important posture point of each detection frame image in each frame, it is necessary to determine the final regression position according to the above process, that is, to complete the correction of the regression position of the posture point of the target object in the running posture data.

[0072] It should be noted that the probability of the left and right knees finally being at the regression position determined by the running posture data is very high. Therefore, the regression position determined solely by the posture recognition algorithm may be inaccurate. After obtaining the running posture data determined by posture recognition, this embodiment further determines the final regression position from the determined regression position and the candidate position determined by the heat map, which can further ensure the accuracy of the determined running posture data.

[0073] S204: Determine the key moments of the target object's running process according to the corrected running posture data.

[0074] Optionally, the running process may include at least one step, each step being a running phase, including four key moments of ground contact, cushioning, push-off and take-off.

[0075] Optionally, the key moments of the target object's running process are determined based on the corrected running posture data, including: determining the key data from the running posture data based on the regression position of the posture points in the corrected running posture data and the relationship between the leg orientations; and determining the key moments of the target object's running process based on the timestamps corresponding to the detection box images associated with the key data in the running video.

[0076] Among them, the key data refers to the data of the four key moments of each step of the target object during running. The timestamp refers to each moment in the video time period.

[0077] Optionally, the corrected posture point regression position and the leg orientation relationship can be input into a pre-trained determination model to output key data determined from the running posture data, or the running posture data can be sorted according to preset rules to determine the key data in the running posture data. Specifically, according to the posture point regression position and the leg orientation relationship in the corrected running posture data, the key data is determined from the running posture data, including: grouping the corrected running posture data according to the left-right alternation principle to obtain the running posture data corresponding to each step of the target object during the running process; and determining the key moment of each step of the target object during the running process according to the running posture data corresponding to each step.

[0078] Optionally, the corrected running posture data can be divided into two groups of detection frame sequences according to the overlapping state of both legs in each frame of the detection frame. The running process can be further divided into several running stages, that is, several steps, based on the left-right alternation principle, and the running posture data of each running stage is obtained, that is, the running posture data corresponding to each step of the target object's running process is obtained.

[0079] Optionally, after obtaining the running posture data corresponding to each step of the target object's running, the orientation relationship of the posture points in each frame of the detection frame image contained in each step can be further determined based on the running posture data corresponding to each step, and the orientation relationship of the posture points in each frame of the detection frame image can be compared with the orientation relationship of the posture points at preset key moments of touching the ground, buffering, pushing off and taking off, so as to determine the detection frame image corresponding to each key moment in the detection frame image of each step, thereby determining the four key moments of each step.

[0080] According to the detection frame image and the running posture data, the embodiment of the present invention corrects the leg orientation relationship in the running posture data, further corrects the regression position of the posture point of the target object in the running posture data, and finally determines the key moment of the target object's running process according to the corrected running posture data. By correcting the acquired running posture data, the accuracy of the finally determined running posture data can be guaranteed, which facilitates the subsequent running posture state analysis based on the running posture data.

[0081] Embodiment 3

[0082] Figure 3 This is a structural block diagram of a video processing device provided in Embodiment 3 of the present invention. The video processing device provided in Embodiment 3 of the present invention can execute the video processing method provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0083] The video processing device may include: an acquisition module 301 , a correction module 302 , and a determination module 303 .

[0084] The acquisition module 301 is used to acquire a detection frame image containing a target object in a running video, and running posture data associated with the detection frame image; wherein the running posture data includes: a posture point regression position of the target object, a posture point heat map, and a leg orientation relationship;

[0085] A correction module 302, configured to correct the leg orientation relationship in the running posture data according to the detection frame image and the running posture data;

[0086] The determination module 303 is used to determine the key moments of the target object's running process according to the corrected running posture data.

[0087] After obtaining the detection frame image containing the target object in the running video, and the running posture data such as the posture point regression position, posture point heat map, and leg orientation relationship of the target object associated with the detection frame image, the embodiment of the present invention further corrects the leg orientation relationship in the running posture data according to the detection frame image and the running posture data, and finally determines the key moment of the target object's running process according to the corrected running posture data. By correcting the acquired running posture data, the accuracy of the finally determined running posture data can be guaranteed, which facilitates the subsequent running posture state analysis based on the running posture data.

[0088] Furthermore, the correction module 302 may include:

[0089] a determination unit, configured to determine a moving direction of a target object according to the detection frame image and / or the running posture data;

[0090] A grouping unit, used for grouping the detection frame images according to the running posture data;

[0091] The correction unit is used to correct the leg orientation relationship in the running posture data according to the movement direction, the grouping result and the left-right alternation principle.

[0092] Furthermore, the grouping unit is specifically used for:

[0093] Determine, according to the running posture data, a double-leg overlap state corresponding to the detection frame image; wherein the double-leg overlap state includes overlap and non-overlap;

[0094] The detection frame images with the same overlapping state of the two legs are taken as a set of detection frame sequences.

[0095] Furthermore, the correction unit is specifically used for:

[0096] According to the movement direction and the left-right alternation principle, a first correction process is performed on the leg orientation relationship in the running posture data associated with the first group of detection frame sequences;

[0097] According to the movement direction, the left-right alternation principle and the corrected running posture data associated with the first set of detection frame sequences, a second correction process is performed on the leg orientation relationship in the running posture data associated with the second set of detection frame sequences;

[0098] Among them, the double-leg overlapping state corresponding to the first group of detection frame sequences is non-overlapping; and the double-leg overlapping state corresponding to the second group of detection frame sequences is overlapping.

[0099] Furthermore, the above device is also used for:

[0100] According to the corrected running posture data, before determining the critical moment of the target object's running process, the regression position of the target object's posture points in the running posture data is corrected according to the regression position of the target object's posture points in the running posture data and the posture point heat map.

[0101] Further, the determination module 303 may include:

[0102] A data determination unit, for determining key data from the running posture data according to the relationship between the posture point regression position and the leg orientation in the corrected running posture data;

[0103] The moment determination unit is used to determine the key moment of the target object's running process according to the timestamp corresponding to the detection frame image associated with the key data in the running video.

[0104] Furthermore, the data determination unit is specifically used for:

[0105] According to the left-right alternation principle, the corrected running posture data are grouped to obtain the running posture data corresponding to each step of the target object during the running process;

[0106] According to the running posture data corresponding to each step, the key moment of each step in the running process of the target object is determined.

[0107] Embodiment 4

[0108] Figure 4 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Figure 4 A block diagram is shown of an exemplary apparatus suitable for implementing an exemplary embodiment of the present invention. Figure 4 The device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0109] like Figure 4 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0110] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0111] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0112] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory (cache 32). The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4 Not shown in the figure, the present embodiment may provide a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media). In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of each embodiment of the present invention.

[0113] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described in the embodiments of the present invention.

[0114] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via the bus 18. It should be understood that although Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0115] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the video processing method provided in the embodiment of the present invention.

[0116] Embodiment 5

[0117] Embodiment 5 of the present invention further provides a computer-readable storage medium on which a computer program (or computer-executable instructions) is stored. When the program is executed by a processor, it is used to execute the video processing method provided in the embodiment of the present invention.

[0118] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0119] Computer-readable signal media may include digital signals in baseband or transmitted by carrier signals, which carry computer-readable program codes. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0120] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0121] The computer program code for performing the operation of the embodiments of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0122] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the embodiments of the present invention are described in more detail through the above embodiments, the embodiments of the present invention are not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A video processing method, characterized in that: The method comprises: Acquire a detection frame image containing a target object in a running video, and running posture data associated with the detection frame image; wherein the running posture data includes: a posture point regression position of the target object, a posture point heat map, and a leg orientation relationship; Correcting the leg orientation relationship in the running posture data according to the detection frame image and the running posture data; Determining the key moments of the target object's running process according to the corrected running posture data; Wherein, the step of correcting the leg orientation relationship in the running posture data according to the detection frame image and the running posture data includes: Determine a moving direction of the target object according to the detection frame image and / or the running posture data; grouping the detection frame images according to the running posture data; According to the movement direction, the grouping result and the left-right alternating principle, the leg orientation relationship in the running posture data is corrected, including: according to the movement direction and the left-right alternating principle, a first correction processing is performed on the leg orientation relationship in the running posture data associated with the first group of detection frame sequences; according to the movement direction, the left-right alternating principle and the corrected running posture data associated with the first group of detection frame sequences, a second correction processing is performed on the leg orientation relationship in the running posture data associated with the second group of detection frame sequences; wherein the overlapping state of both legs corresponding to the first group of detection frame sequences is non-overlapping; and the overlapping state of both legs corresponding to the second group of detection frame sequences is overlapping.

2. The method according to claim 1, characterized in that The detection frame images are grouped according to the running posture data, including: Determine, according to the running posture data, a double-leg overlap state corresponding to the detection frame image; wherein the double-leg overlap state includes overlap and non-overlap; The detection frame images with the same overlapping state of the two legs are taken as a set of detection frame sequences.

3. The method according to claim 1, characterized in that Before determining the key moment of the target object's running process according to the corrected running posture data, the method further includes: According to the regression position of the posture points of the target object in the running posture data and the posture point heat map, the regression position of the posture points of the target object in the running posture data is corrected.

4. The method according to claim 1 or 3, characterized in that: Determine the key moments of the target object's running process according to the corrected running posture data, including: Determining key data from the running posture data according to the relationship between the regression positions of the posture points in the corrected running posture data and the leg orientations; The key moment of the target object's running process is determined according to the timestamp corresponding to the detection frame image associated with the key data in the running video.

5. The method according to claim 4, characterized in that According to the relationship between the regression position of the posture point and the leg orientation in the corrected running posture data, key data is determined from the running posture data, including: According to the left-right alternation principle, the corrected running posture data are grouped to obtain the running posture data corresponding to each step of the target object during the running process; According to the running posture data corresponding to each step, the key moment of each step in the running process of the target object is determined.

6. A video processing device, characterized in that: The device comprises: An acquisition module is used to acquire a detection frame image containing a target object in a running video, and running posture data associated with the detection frame image; wherein the running posture data includes: a posture point regression position of the target object, a posture point heat map, and a leg orientation relationship; A correction module, used for correcting the leg orientation relationship in the running posture data according to the detection frame image and the running posture data; A determination module, used to determine the key moments of the target object's running process according to the corrected running posture data; The correction module includes: a determination unit, configured to determine a moving direction of a target object according to the detection frame image and / or the running posture data; A grouping unit, used for grouping the detection frame images according to the running posture data; A correction unit, used for correcting the leg orientation relationship in the running posture data according to the movement direction, the grouping result and the left-right alternation principle; Among them, the correction unit is specifically used to: perform a first correction processing on the leg orientation relationship in the running posture data associated with the first group of detection frame sequences according to the movement direction and the left-right alternation principle; perform a second correction processing on the leg orientation relationship in the running posture data associated with the second group of detection frame sequences according to the movement direction, the left-right alternation principle and the corrected running posture data associated with the first group of detection frame sequences; wherein the overlapping state of both legs corresponding to the first group of detection frame sequences is non-overlapping; and the overlapping state of both legs corresponding to the second group of detection frame sequences is overlapping.

7. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the video processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the video processing method according to any one of claims 1 to 5 is implemented.

Images