Human body posture estimation result generation method and device based on generative model

Abstract

The invention discloses a human body posture estimation result generation method and device based on a generative model, relates to the technical field of image processing, effectively recovers a complete skeleton structure, realizes accurate recognition of human body postures in high-shielding and high-dynamic change scenes such as an electric power production field, and improves the adaptability to shielding scenes. The method comprises the following steps: acquiring a to-be-identified video stream, and performing initial attitude estimation on each frame of video image in the to-be-identified video stream by using a pre-trained attitude detection network model to obtain an initial skeleton point sequence; fitting the initial skeleton point sequence by adopting a Gaussian mixture modeling method to obtain a defect skeleton point sequence; inputting the defective skeleton point sequence into a pre-trained inverse diffusion network model, and optimizing the defective skeleton point sequence in combination with the adjacent frame skeleton point coding information of each frame of video image in the inverse diffusion network model to obtain a complete skeleton point sequence; and generating and outputting a human body posture estimation result based on the complete skeleton point sequence.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a human pose estimation result generation method and device based on a generative model. BACKGROUND

[0002] At present, the safety and standardization of the work site are the core problems that the power management system at all levels focuses on in the continuous promotion of the intelligent construction of the power system. The power station, substation and power transmission line and other areas belong to high-risk work environment, and the work scene is complex and changeable. Whether the standard execution of personnel behavior is directly related to the safety of equipment and personal safety. Therefore, using technical means to realize accurate identification and real-time supervision of work behavior has become a key issue in the construction of an intelligent power system. In recent years, human pose estimation has attracted widespread attention. Human pose estimation aims to identify the main key points of the human body from images or videos and construct a human skeleton structure to describe the spatial state and motion pattern of individuals. Skeleton information can not only be used for static action recognition, but also can be applied to dynamic behavior modeling, dangerous behavior detection and labor standard analysis and other downstream tasks, and has been widely used in security monitoring, industrial production, medical rehabilitation, sports analysis and other scenes.

[0003] In related technologies, the pose estimation method is a discriminative method. The discriminative method learns the mapping relationship between the input image and the human key points with the help of a deep neural network. Specifically, a convolutional neural network can be used to extract image features, and the key point position can be predicted through stepwise regression or heat map. Among them, HRNet (High-Resolution Network) is a convolutional neural network model widely used in human pose estimation tasks. Its core is to maintain the continuity of high-resolution feature representation, and to improve the accuracy of key point prediction through multi-scale feature fusion. Unlike the traditional down-sampling and up-sampling structure, it preserves a high-resolution branch from the initial stage of the network and introduces new low-resolution paths at each stage, so that the network always maintains multiple scale feature maps in parallel throughout the forward propagation process. Through a specific fusion unit, the fusion of different resolution information is realized, and a better balance between spatial precision and semantic understanding is achieved.

[0004] However, the applicant realizes that, as a pure two-dimensional image convolution network, HRNet is extremely dependent on visible information in the image. In the power production site, workers usually wear overalls and safety helmets, have flexible postures, and have frequent occlusions and limited view angles. The camera installation angle is fixed, the personnel motion range is large and is not restricted, which leads to incomplete or incorrect skeleton key point prediction, making HRNet prone to prediction failure, prone to key point confusion or misattribution, seriously affecting the accuracy of posture estimation, making it difficult to ensure accurate segmentation and recognition of multiple human postures, and unable to perform human posture modeling tasks in high-occlusion and high-dynamic change scenes such as power production sites. SUMMARY

[0005] Therefore, the present application provides a human posture estimation result generation method and device based on a generative model, which mainly aims to solve the problem that the current method is prone to key point confusion or misattribution, seriously affects the accuracy of posture estimation, makes it difficult to ensure accurate segmentation and recognition of multiple human postures, and is unable to perform human posture modeling tasks in high-occlusion and high-dynamic change scenes such as power production sites.

[0006] According to a first aspect of the present application, a human posture estimation result generation method based on a generative model is provided, which comprises: Obtaining a to-be-identified video stream, using a pre-trained posture detection network model to perform initial posture estimation on each frame of video image in the to-be-identified video stream, and obtaining an initial skeleton point sequence; Using a Gaussian mixture modeling method to fit the initial skeleton point sequence, and obtaining a missing skeleton point sequence; Inputting the missing skeleton point sequence into a pre-trained inverse diffusion network model, combining the adjacent frame skeleton point encoding information of each frame of video image in the inverse diffusion network model, optimizing the missing skeleton point sequence, and obtaining a complete skeleton point sequence; Based on the complete skeleton point sequence, generating a human posture estimation result and outputting.

[0007] According to a second aspect of the present application, a human posture estimation result generation device based on a generative model is provided, which comprises: A front posture estimation module is configured to obtain a to-be-identified video stream, use a pre-trained posture detection network model to perform initial posture estimation on each frame of video image in the to-be-identified video stream, and obtain an initial skeleton point sequence; A fitting module is configured to use a Gaussian mixture modeling method to fit the initial skeleton point sequence, and obtain a missing skeleton point sequence; an inverse diffusion module, configured to input the missing skeletal point sequence into a pre-trained inverse diffusion network model, and combine adjacent frame skeletal point encoding information of each frame of video image in the inverse diffusion network model to optimize the missing skeletal point sequence, and obtain a complete skeletal point sequence; an output module, configured to generate and output a human pose estimation result based on the complete skeletal point sequence.

[0008] By the above technical solution, the human pose estimation result generation method and device based on a generative model are provided. The video stream to be recognized is obtained, an initial pose detection network model is used to perform initial pose estimation on each frame of video image in the video stream to be recognized to obtain an initial skeletal point sequence, a Gaussian mixture modeling method is used to fit the initial skeletal point sequence to obtain a missing skeletal point sequence, the missing skeletal point sequence is input into a pre-trained inverse diffusion network model, adjacent frame skeletal point encoding information of each frame of video image is combined in the inverse diffusion network model to optimize the missing skeletal point sequence, and a complete skeletal point sequence is obtained. Based on the complete skeletal point sequence, a human pose estimation result is generated and output. The visible key points of the human pose in the video stream are quickly located by the pose detection network model, and the generation capability of the inverse diffusion network model is used to refer to the adjacent frame skeletal point encoding information of each frame of video image to reasonably complete the occluded part of the human pose, effectively restore the complete skeleton structure, and accurately recognize the human pose in a high-occlusion and high-dynamic change scene such as an electric power production site, thereby improving the adaptability and robustness to the occlusion scene.

[0009] The above description is only a summary of the technical solutions of the present application. In order to enable the technical means of the present application to be more clearly understood, and to be implemented according to the content of the description, and in order to enable the above and other purposes, features and advantages of the present application to be more apparent and easy to understand, the following detailed description of the specific embodiments of the present application is provided. BRIEF DESCRIPTION OF DRAWINGS

[0010] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not meant to limit the present application. Moreover, the same reference numerals in the attached drawings indicate the same or similar components. In the drawings: Figure 1 A flowchart of a human pose estimation result generation method based on a generative model is shown; Figure 2 An architecture diagram of a human pose estimation result generation method based on a generative model is shown; Figure 3A schematic diagram of a model training process in a human pose estimation result generation method based on a generative model is shown. Figure 4 A schematic diagram of another human pose estimation result generation method based on a generative model is shown. Figure 5 A structural schematic diagram of a human pose estimation result generation device based on a generative model is shown. DETAILED DESCRIPTION

[0011] Exemplary embodiments of the present application will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it is understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present application can be more thoroughly understood and the scope of the present application can be accurately conveyed to those skilled in the art.

[0012] The present application provides a human pose estimation result generation method based on a generative model, as shown in the method comprises: Figure 1 S10, obtaining a to-be-identified video stream, performing initial pose estimation on each frame of video image in the to-be-identified video stream by using a pre-trained pose detection network model, and obtaining an initial skeleton point sequence. S10, obtaining a to-be-identified video stream, performing initial pose estimation on each frame of video image in the to-be-identified video stream by using a pre-trained pose detection network model, and obtaining an initial skeleton point sequence.

[0013] The technical solution of the present application can be applied to a monitoring system, and a large number of monitoring devices are connected to the monitoring system to identify and estimate the video collected by the monitoring devices in real time. In the present application, the monitoring system first obtains a to-be-identified video stream, which can be a real-time monitoring video stream of a substation scene, capturing the picture of the operating personnel at a rate of 25 frames per second, which can clearly record the dynamic of the operating personnel in the substation.

[0014] Then, the monitoring system uses a pre-trained pose detection network model to perform initial pose estimation on each frame of video image in the to-be-identified video stream. The pre-trained pose detection network model in the present application can be a lightweight pose detection network (HRNet architecture), which is a network structure with high efficiency, capable of reducing the amount of calculation while ensuring a certain accuracy and improving the processing speed. After processing each frame of image by the pose detection network model, an initial skeleton point sequence can be obtained. However, due to various equipment, object occlusions, and the action occlusions of the operating personnel in the substation scene, the actual obtained initial skeleton point sequence contains seriously missing or incorrectly estimated pose key points caused by occlusions.

[0015] In this way, the posture detection network model can quickly perform initial posture estimation on each frame of image in the video stream, quickly locate the visible key points of the human body posture in the video stream, and provide basic data for subsequent processing. Meanwhile, the lightweight feature of the posture detection network model ensures the efficiency of the processing, and adapts to the processing requirements of the real-time monitoring video stream. For example, during a certain equipment maintenance operation in a substation, the operator stands beside a large transformer to operate, and part of the body is blocked by the transformer. At this time, the real-time monitoring video stream captures this scene, and the posture detection network model based on the HRNet architecture performs initial posture estimation on each frame of image to obtain the initial skeleton point sequence. However, the skeleton points corresponding to the blocked part of the body are missing or estimated incorrectly, for example, the originally continuous leg skeleton point sequence is broken or deviated in position due to the blocking.

[0016] In step S10, that is, obtaining the to-be-identified video stream, the pre-trained posture detection network model is used to perform initial posture estimation on each frame of video image in the to-be-identified video stream to obtain an initial skeleton point sequence, including the following steps: S11, collecting the real-time monitoring video stream of the target scene as the to-be-identified video stream, and performing frame processing on the to-be-identified video stream to obtain a plurality of video images.

[0017] In the embodiments of the present application, it is assumed that the scene currently requiring video stream collection is a substation scene in power production, and the scene is the target scene. The monitoring system collects the real-time monitoring video stream through the monitoring camera installed in the substation and takes it as the to-be-identified video stream. There are many devices in the substation, the layout is complex, and the activity range of the operator is large. The collected real-time monitoring video stream can comprehensively record various operations and postures of the operator in the substation.

[0018] Then, the monitoring system performs frame processing on the to-be-identified video stream, splits the continuous video stream into independent images one by one according to a certain time interval or frame rate, and specifically, the frame processing can be performed according to the original frame rate of the video stream, for example, at a rate of 25 frames per second. Finally, a plurality of video images are obtained, each of which records the picture information of the operator at a certain moment, so that subsequent detailed posture analysis can be performed on each frame of image, and necessary image data support is provided for accurate human posture estimation.

[0019] For example, during a device inspection operation in a substation, the operator holds a detection instrument to inspect between devices, and the real-time monitoring video stream is collected through a monitoring camera. The video stream records the whole process from the operator entering the substation to inspecting each device. Then, frame processing is performed to obtain a series of video images containing different postures and positions of the operator, such as the picture of the operator standing to view the device, the picture of the operator bending to detect the bottom of the device, and the like.

[0020] S12, each frame of video image in the multi-frame video image is subjected to illumination normalization and resolution enhancement processing, and the processed multi-frame video image is input into a posture detection network model, and the posture detection network model is used to perform initial posture estimation on each frame of the processed video image to obtain an initial skeleton point sequence.

[0021] In the embodiments of the present application, considering that the video acquisition of the power site is often interfered by strong light, shadow, blur, equipment vibration and the like, therefore, the monitoring system will perform illumination normalization and resolution enhancement processing on each frame of video image in the multi-frame video image. The illumination normalization is an image processing technology. Since the illumination conditions of the power production site are complex and changeable, the illumination intensity and color at different times and different positions may have great differences, which easily affects the extraction and recognition of image features by the posture detection network model. Therefore, the brightness, contrast and color balance and the like of the image are adjusted through the illumination normalization processing, so that the images under different illumination conditions have similar illumination features, thereby eliminating the interference of the illumination factor on the posture estimation. Further, the resolution enhancement processing is to improve the definition and detail performance of the image. In the power production scene, the image captured by the monitoring camera may have low resolution due to the distance, device performance and the like, therefore, the number of pixels of the image is increased through the resolution enhancement processing, and the quality of the image is improved, so that the human posture features in the image are more obvious.

[0022] Next, the monitoring system will input the processed multi-frame video image into the pre-trained posture detection network model. The details of the posture detection network model can be referred to the description thereof above, and will not be described herein again. The posture detection network model will perform initial posture estimation on each frame of the processed video image. By analyzing the contour, texture and the like of the human body in the image, each key part of the human body is recognized, and the key point positions corresponding to these key parts are determined, and finally an initial skeleton point sequence is obtained. The initial skeleton point sequence can be represented by the following formula 1. The initial skeleton point sequence contains the preliminary information of the human posture, but due to the equipment occlusion, personnel self-occlusion and the like in the power production site, the initial skeleton point sequence may contain seriously missing or erroneously estimated posture key points caused by occlusion: Formula 1:

[0023] In formula 1, represents the initial skeleton point sequence, represents the posture detection network model, represents a frame of video image input into the posture detection network model.

[0024] In this way, by illumination normalization and resolution enhancement processing, the quality of the video image is improved, the illumination interference is eliminated, and the image clarity is improved, so that the pose detection network model can more accurately extract the human pose features in the image, and the accuracy of the initial pose estimation is improved. In addition, the lightweight pose detection network (HRNet architecture) can ensure processing efficiency while performing initial pose estimation on the processed image to quickly obtain the initial skeleton point sequence, providing basic data for subsequent processing. For example, in the above substation equipment inspection operation scene, among the multiple frames of video images obtained through frame processing, one frame of image shows an operator standing beside a large switch cabinet for detection. Due to the shielding of the switch cabinet, part of the operator's body is blocked, and the on-site illumination is dark and uneven. After illumination normalization processing is performed on this frame of image, the brightness of the image is adjusted, and the overall illumination is more uniform, eliminating the problem of local overdarkness or overbrightness of the image caused by illumination differences. After resolution enhancement processing, the clarity of the image is significantly improved, and the contours and clothing textures of the operator are more clear. After the processed image is input to the pose detection network model of the HRNet architecture, the model can more accurately identify the key points of the visible parts of the operator, such as the key point positions of the head, shoulders, arms, etc. However, due to the shielding of the switch cabinet, the key points of the legs and other parts are missing or estimated incorrectly, and finally an initial skeleton point sequence containing these problems is obtained.

[0025] S20, using a Gaussian mixture modeling method, fitting the initial skeleton point sequence to obtain a missing skeleton point sequence.

[0026] In the embodiments of the present application, the monitoring system continues to use the Gaussian mixture modeling method to fit the initial skeleton point sequence. Gaussian mixture modeling is a modeling method based on probability statistics, which assumes that the data is mixed by multiple Gaussian distributions. By estimating and adjusting the parameters of these Gaussian distributions, the distribution characteristics of the data are fitted. In the embodiments of the present application, the monitoring system uses this method to re-model the human pose, analyzes the position, motion trend and other data of each skeleton point in the initial skeleton point sequence, constructs multiple Gaussian distributions to describe the different characteristics of the human pose, and then obtains the missing skeleton point sequence. The missing skeleton point sequence is a preliminary correction and supplement to the skeleton points with missing or incorrect estimates based on the initial skeleton point sequence, but still retains some incomplete information caused by factors such as shielding.

[0027] In this way, by remodeling the human pose through the Gaussian mixture modeling method, the data in the initial skeleton point sequence can be more reasonably analyzed and processed using the characteristics of probability statistics, and the missing or incorrectly estimated skeleton points are preliminarily corrected and supplemented, thereby providing more accurate data basis for further optimization and improving the accuracy of human pose description. For example, in the above substation equipment maintenance operation scene, the leg is blocked by the transformer, resulting in missing or incorrect estimation of the skeleton points in the initial skeleton point sequence. After the Gaussian mixture modeling method is used, a plurality of Gaussian distributions are constructed to describe the leg pose characteristics according to the existing normal skeleton point positions and motion trends, and the missing skeleton points are preliminarily supplemented by adjusting the parameters of the Gaussian distributions, thereby obtaining the missing skeleton point sequence. Although the leg skeleton points may still be incomplete at this time, they have been significantly improved compared with the initial skeleton point sequence and are closer to the real leg pose.

[0028] In step S20, the initial skeleton point sequence is fitted by using the Gaussian mixture modeling method to obtain the missing skeleton point sequence, and the specific process is as follows: The monitoring system first acquires the Gaussian distribution of the historical human key point trajectory constructed by using the Gaussian mixture modeling method, and the Gaussian distribution is used to describe the characteristics of the historical human key point trajectory.

[0029] Then, the monitoring system combines the Gaussian distribution and the initial skeleton point sequence, and uses the following formula 2 to perform human pose remodeling processing on the initial skeleton point sequence to obtain the missing skeleton point sequence: Formula 2:

[0030] wherein, represents the missing skeleton point sequence, which is a skeleton point sequence containing uncertainties obtained after remodeling processing; represents a weight coefficient for balancing the contributions of the Gaussian distribution and the time series interpolator, and the value range is between 0 and 1, which is used to adjust the influence degree of the two in the final result; represents the Gaussian distribution, represents the position of the i-th key point in the Gaussian distribution, represents the variance of the Gaussian distribution, and the greater the variance, the more dispersed the data, which describes the uncertainty range of the key point position; represents the time series interpolator, represents the position of the previous key point at the previous time The time series interpolator uses the position information of the key point at the previous time to estimate the position of the current key point by interpolation according to the current situation, so as to supplement and correct the information in the initial skeleton point sequence.

[0031] Through the above formula 2, the Gaussian distribution of the historical human key point trajectory constructed by the Gaussian mixture modeling method captures the long-term change rule and distribution characteristics of the human posture, and the time series interpolator considers the continuity and correlation of the human posture in time. Through the combination of the two, the initial skeleton point sequence can be more reasonably modeled and corrected, and a more reasonable human motion rule is obtained. The missing skeleton point sequence provides more accurate basic data for further optimizing human posture estimation.

[0032] In the embodiments of the present application, the monitoring system inputs the missing skeleton point sequence into the pre-trained inverse diffusion network model. The inverse diffusion network model is a network structure based on a generative model, which generates data conforming to the real distribution from noisy data by gradually removing noise in the data.

[0033] In the embodiments of the present application, the monitoring system inputs the missing skeleton point sequence into the pre-trained inverse diffusion network model. The inverse diffusion network model is a network structure based on a generative model, which generates data conforming to the real distribution from noisy data by gradually removing noise in the data.

[0034] In the inverse diffusion network model, the missing skeleton point sequence will be optimized in combination with the adjacent frame skeleton point encoding information of each frame of video image to obtain the complete skeleton point sequence. The adjacent frame skeleton point encoding information refers to the corresponding skeleton point information in the previous and subsequent several frames of image adjacent to the current frame of video image. By encoding these adjacent frame skeleton points, key features and motion trends can be extracted, and the adjacent frame skeleton point encoding information can be used to optimize the missing skeleton point sequence. Specifically, the inverse diffusion network model will reasonably speculate and correct the missing skeleton points of the current frame according to the position change, motion rule and other information of the adjacent frame skeleton points, supplement the missing skeleton points, correct the estimated skeleton point positions, and finally obtain the complete skeleton point sequence.

[0035] In this way, by using the generation capability of the inverse diffusion network model and combining the encoding information of the adjacent frame skeleton points, the occluded part of the human body posture can be reasonably completed in structure. Moreover, by referring to the information of the adjacent frame, the dynamic change of the human body posture can be better understood, so that the completed skeleton point sequence is more consistent with the real situation of human motion, the complete skeleton structure is effectively restored, and the accuracy of human body posture estimation is improved. For example, in the substation equipment maintenance operation scene, when the legs of the operator are occluded by the transformer, the leg skeleton points in the missing skeleton point sequence are missing. After the missing skeleton point sequence is input into the inverse diffusion network model, the model combines the encoding information of the adjacent frame skeleton points, and finds that the leg skeleton points in the adjacent frame show a forward stepping trend. According to this trend, the model reasonably infers and completes the missing leg skeleton points in the current frame, corrects the possible wrong estimated positions, and finally obtains the complete leg skeleton point sequence, so that the whole body skeleton structure is more complete and accurate.

[0036] To implement the processing of the missing skeleton point sequence, the inverse diffusion network model needs to be trained in advance, and the training process includes the following steps S50 to S90: S50, obtaining a sample video stream, and performing initial posture estimation on each frame of video image in the sample video stream by using a posture detection network model to obtain a sample skeleton point sequence.

[0037] The monitoring system will obtain a sample video stream, which can be selected from the monitoring video of the power production site, such as substation equipment inspection, distribution room operation, etc. The monitoring system will use the posture detection network model mentioned above (i.e. the lightweight posture detection network of HRNet architecture) to perform initial posture estimation on each frame of video image in the sample video stream to obtain a sample skeleton point sequence.

[0038] At the same time, the monitoring system loads the labeled real posture data, which is the accurate human skeleton point position information obtained by manual accurate labeling or other high-precision measurement methods, so as to simulate the degradation process from clear posture to uncertain distribution in the subsequent process, and provide training data with reality and uncertainty for subsequent model training, which helps to train the inverse diffusion network model that can adapt to the complex situation of power production site.

[0039] In step S50, i.e. obtaining a sample video stream, performing initial posture estimation on each frame of sample video image in the sample video stream by using a posture detection network model to obtain a sample skeleton point sequence, including the following steps: S51, performing frame processing on the sample video stream to obtain a plurality of sample video images.

[0040] The monitoring system first acquires a sample video stream, which can be derived from the monitoring video in the power production site, such as the daily inspection of the substation, the maintenance of power equipment, and the like, and then performs frame processing on the sample video stream, splits the continuous video stream into independent images according to a certain time interval or a fixed frame rate, such as splitting at a rate of 25 frames per second according to the original frame rate of the video, and finally obtains multiple sample video images. Each sample video image records the picture information of the personnel in the power production site at a certain moment, providing a basis for subsequent human region recognition and posture estimation, so that subsequent detailed analysis and processing can be performed on each image, providing necessary image data support for accurate human region acquisition and posture estimation, and ensuring that various posture information of personnel in the power production site can be fully captured.

[0041] S52, identifying a human region in each sample video image, and identifying the human region in each sample video image to obtain multiple sample human region images.

[0042] For the obtained multiple sample video images, the monitoring system identifies the human region in each sample video image. Specifically, a target detection algorithm based on deep learning can be used to identify the position and range of the human body in the image through learning and training of a large number of human body images.

[0043] After identifying the human region in each sample video image, the monitoring system crops the identified human region from the original image to obtain multiple sample human region images. The sample human region images remove irrelevant information such as the background in the original image, highlighting the human body, so that the subsequent posture detection network model can focus more on the extraction and analysis of human features, reduce interference factors in the subsequent posture estimation process, improve the accuracy and efficiency of posture estimation, reduce the amount of calculation, improve the processing efficiency, and better adapt to the posture estimation requirements in the complex background of the power production site.

[0044] For example, in the sample video images of the substation equipment maintenance, some images have a background containing a large number of power equipment and complex lines. Through human region identification, these background information is removed, and only the human region image of the maintenance personnel is retained, so that the subsequent posture estimation can be more accurately performed on the human body.

[0045] S53, inputting the multiple sample human region images into the posture detection network model, so that the posture detection network model performs initial posture estimation on each sample human region image to obtain a sample skeleton point sequence.

[0046] The monitoring system inputs the obtained multiple frames of sample human region images into the pose detection network model. Details of the pose detection network model can be referred to the foregoing description, and will not be described herein again. The pose detection network model performs initial pose estimation on each frame of sample human region image, identifies each key part of the human body by analyzing the contour, texture and other features of the human body in the image, determines the key point positions corresponding to the key parts, and finally obtains a sample skeleton point sequence containing preliminary information of the human pose, thereby providing basic data for subsequent simulation of the degradation process and training of the inverse diffusion network model.

[0047] S60, gradually adding Gaussian noise to the sample skeleton point sequence to generate noise pose data.

[0048] In the embodiment of the present application, the monitoring system gradually adds Gaussian noise to the sample skeleton point sequence. Gaussian noise is noise with a normal distribution probability density function, which is often used to simulate random interference in image and data processing. In the embodiment of the present application, Gaussian random noise is gradually added according to a certain step and rule on the basis of the initial sample skeleton point sequence to generate noise pose data.

[0049] In actual application, noise pose data fitted with the Gaussian mixture modeling result can be generated after 50 steps of continuous diffusion (i.e. 50 times of addition). The amount of noise added at each step gradually increases, so that the sample skeleton point sequence gradually changes from a clear state to a noise pose with uncertainty, simulating the uncertainty of pose information caused by various interference factors in actual human pose estimation, making the training data more abundant and closer to reality, which helps to improve the adaptability of the inverse diffusion network model to noise and uncertainty, and enhances the robustness of the model in actual application. The specific process of adding Gaussian noise to generate noise pose data is as follows: First, a preset initial noise intensity coefficient is obtained. The initial noise intensity coefficient is used to control the intensity of the first added Gaussian noise. At the same time, a preset number of noise additions is determined, which determines the total number of steps in the entire noise addition process. Then, the sample skeleton point sequence and the initial noise intensity coefficient are used to construct an initial noise Gaussian distribution. Gaussian distribution is a probability distribution with a bell-shaped curve, which is often used to simulate random noise in data processing. In the embodiment of the present application, the mean and variance parameters of the initial noise Gaussian distribution are determined by the relevant features of the sample skeleton point sequence and the initial noise intensity coefficient, so as to construct the required initial noise Gaussian distribution.

[0050] Subsequently, the monitoring system samples from the initial noise Gaussian distribution according to Formula 3 below to obtain a noise posture sequence after adding Gaussian noise for the first time, and continues to obtain a second noise intensity coefficient, uses the noise posture sequence after adding Gaussian noise for the first time and the second noise intensity coefficient to construct a second noise Gaussian distribution, and continues to sample from the second noise Gaussian distribution according to the following formula to obtain a noise posture sequence after adding Gaussian noise for the second time, until the number of noise additions reaches the preset number of noise additions, to obtain the noise posture data: Formula 3:

[0051] wherein, denotes the conditional probability distribution from the posture to the posture , denotes the noise posture sequence after adding Gaussian noise for the first time, denotes the noise posture sequence after adding Gaussian noise for the first time, denotes the noise posture sequence after adding Gaussian noise for the first time, denotes the noise intensity, the value of is a natural number between 1 and the preset number of noise additions, and the value of gradually increases with the increase of , denotes the noise Gaussian distribution with as the mean value and as the covariance matrix, denotes the preset unit matrix. In this way, by gradually adding Gaussian noise and constructing the corresponding noise Gaussian distribution for sampling, the uncertainty and noise influence of human posture estimation caused by various interference factors (such as equipment occlusion, light change, sensor noise, etc.) in the actual power production scene can be simulated, so that the generated noise posture data is closer to the posture data in the real scene, providing more abundant and more actual training data for the training of the subsequent inverse diffusion network model, which helps to improve the adaptability of the model to noise and uncertainty and enhance the robustness of the model in actual application.

[0052] S70, input the noise posture sequence of the continuous multiple frame sample video images in the noise posture data into the LSTM (Long Short-Term Memory, Long Short-Term Memory Network) network architecture for processing to obtain the conditional encoding of the time sequence context information of the human action.

[0053] S70, input the noise posture sequence of the continuous multiple frame sample video images in the noise posture data into the LSTM (Long Short-Term Memory, Long Short-Term Memory Network) network architecture for processing to obtain the conditional encoding of the time sequence context information of the human action.

[0054] ​​The monitoring system inputs the noisy pose sequences of multiple consecutive frames of sample video images from the noisy pose data into a Long Short-Term Memory (LSTM) network architecture for processing. LSTM is a type of Recurrent Neural Network (RNN) that can effectively process and predict time-series data and has the ability to remember long-term information. In practical applications, five consecutive frames of pose sequences can be selected and input into the LSTM network. The LSTM network analyzes these five frames of pose sequences to capture the trends and temporal characteristics of human movements, ultimately obtaining conditional codes that include the temporal contextual information of human movements. These conditional codes will serve as important input information for the subsequent inverse diffusion network, helping the model better understand the continuity and changing patterns of human poses over time, especially in scenarios such as power production sites where human movements are complex and temporal.

[0055] S80. Based on noise attitude data and conditional coding, reverse diffusion processing is performed on the preset network architecture to obtain the diffusion processing result.

[0056] In this embodiment, since video acquisition at power sites is often affected by strong light, shadows, blur, and equipment vibration, the monitoring system performs reverse diffusion processing on the preset network architecture based on noisy pose data and conditional coding. Through this diffusion process, noise is gradually reduced, optimizing the initially incomplete and low-reliability pose estimation. This reduces the impact of input quality on the final estimation accuracy, thus maintaining stable performance under a wider range of real-world conditions. The preset network architecture includes a Graph Convolutional Network (GCN) network and an attention stacking network. The GCN network can process data with a graph structure; human skeleton points can be viewed as a graph structure, with each key point as a node and the connections between nodes as edges. The GCN network can effectively extract spatial relationship features between skeleton points. The attention stacking network focuses on important feature information and automatically adjusts the weights of different features.

[0057] In practical applications, when performing reverse diffusion processing on a preset network architecture, we can start from the pose distribution with the highest noise and optimize it through 50 iterative steps. During the reverse diffusion process, each step uses a Gaussian Network (GCN) and an attention stacking network to correct and optimize skeletal keypoints. In this process, while independently predicting N keypoints, we also combine them with conditional coding of neighboring frames to obtain more motion trend information, resulting in more accurate keypoint predictions. Each denoising step is the inverse elimination of Gaussian noise, gradually restoring the noisy pose to data that more closely approximates the true pose. The specific process of performing reverse diffusion processing on the preset network architecture is as follows: First, the monitoring system inputs the noisy pose data and conditional coding into the GCN network within a pre-defined network architecture. The GCN network is a neural network used to process graph-structured data. Human skeletons can be viewed as a graph, with each skeleton as a node and the connections between nodes as edges. The GCN network can process and update the topological relationships between skeletons in the noisy pose data according to the conditional coding, resulting in processed noisy pose data. Here, topological relationships refer to the connection methods and relative positions between skeletons. Utilizing the GCN network's ability to process spatial relationships between skeletons, combined with the temporal context information in the conditional coding, the noisy pose data undergoes preliminary correction and optimization.

[0058] Next, the processed noisy pose data and conditional encoding are input into the attention stacking network in the preset network structure. The attention stacking network is a network structure that can automatically focus on important feature information. It can assign weights to the skeleton points in the noisy pose data according to the conditional encoding. The weights reflect the importance of each skeleton point in the current processing. By assigning different weights to different skeleton points, the attention stacking network can pay more attention to the key skeleton point information and further update the noisy pose data.

[0059] Finally, the processing result of the attention stacking network is used as the diffusion processing result. This result is data that is closer to the real human posture after combining the spatial relationship of skeletal points and temporal context information and being optimized by the attention mechanism.

[0060] In this way, by using a pre-defined network architecture that combines GCN network and attention stacking network for reverse diffusion processing, we can make full use of the spatial relationships and temporal context information between skeleton points to effectively denoise and restore noisy pose data. In scenarios such as power production sites where human poses are complex and occluded, we can better restore the complete skeleton structure and improve the accuracy of human pose estimation.

[0061] In summary, the process of reverse diffusion of the preset network architecture in step S80 is summarized as follows: See Figure 2 The architecture diagram shown includes a context encoder and a pre-defined network architecture. The pre-defined network architecture consists of a GCN layer and an attention layer. The context encoder, also known as an LSTM network architecture, is responsible for extracting the spatiotemporal context features of video frames and outputting spatial-temporal context features. ) and the k-th step reverse diffusion embedding ( Spatial-temporal contextual features integrate information from the video in both spatial and temporal dimensions, helping the model to better understand the dynamic changes in human posture; the k-th step backdiffusion embedding is related to the current step of backdiffusion, providing step-specific information for subsequent processing.

[0062] During the back-diffusion process, the process proceeds step-by-step from k=K to k=0. In each step, the input hidden state (e.g., h1K, ..., hNK) passes through the GCN network layer and the attention stacking network layer sequentially. The GCN network layer processes human skeleton point data with a graph structure, capturing the topological relationships between skeleton points, i.e., the connections and relative position information between each skeleton point. It updates the feature representation of the skeleton points by analyzing these relationships. The attention stacking network layer automatically focuses on important feature information, assigning different weights to different skeleton points or features, highlighting key information, and further improving the model's ability to capture important pose features.

[0063] The entire network architecture acquires comprehensive spatiotemporal information through a context encoder, and then combines the GCN network layer and the attention stacking network layer to perform fine processing on the data in each back diffusion step, gradually removing noise and recovering a more accurate human pose. It is suitable for human pose estimation tasks in complex scenarios such as power production sites.

[0064] S90. Calculate the loss value using the diffusion processing results, update and optimize the parameters of the preset network architecture based on the loss value, and perform the next reverse diffusion processing on the updated and optimized preset network architecture using the diffusion processing results until the number of reverse diffusion processing reaches the threshold. The model formed by the currently trained preset network architecture is then used as the reverse diffusion network model.

[0065] In this embodiment, the monitoring system uses the diffusion processing results to calculate the loss value. This embodiment employs the MSE (Mean Squared Error) loss function to calculate the loss value. The MSE loss function measures the model's prediction accuracy by calculating the mean squared error between the predicted value and the true value. Specifically, the diffusion processing results can be compared with the initial accurate skeleton points (i.e., the labeled true pose data) to calculate the loss value.

[0066] Step S90, which involves calculating the loss value using the diffusion processing results, includes the following process: First, the monitoring system acquires the true pose data corresponding to the current reverse diffusion processing round. This true pose data is accurate information about the human skeleton's position points, obtained through precise manual annotation or other high-precision measurement methods; it represents the actual human pose. Next, the following formula 4 is used to calculate the loss value by combining the diffusion processing result and the true pose data: Formula 4:

[0067] in, Represents the loss function. Indicates the current round of reverse diffusion processing. The value ranges from 1 to the number of times threshold. Indicates the threshold number of times. This indicates the diffusion processing result obtained after the current round of reverse diffusion processing. This represents the actual attitude data corresponding to the current round of reverse diffusion processing.

[0068] After the loss value is calculated through the above process, the monitoring system will update and optimize the parameters of the preset network architecture based on the calculated loss value. The loss value is backpropagated to the network through the backpropagation algorithm to analyze the contribution of each parameter to the loss value. The network weights and biases are adjusted according to optimization strategies such as stochastic gradient descent to reduce prediction errors and make the diffusion processing results closer to the real pose data. Next, the results of the diffusion process are used to perform another round of reverse diffusion on the updated and optimized preset network architecture. This process is repeated until the number of reverse diffusion processes reaches a threshold (e.g., 50). The model formed by the currently trained preset network architecture is then used as the reverse diffusion network model. In practical applications, the threshold can be dynamically adjusted for more complex scenarios to achieve higher generation accuracy. This allows the model to learn the global distribution and topological dependencies of the human skeleton during training. The generated poses not only fill in missing parts but also conform to human anatomy. Under various complex actions, non-frontal views, and skeletal twisting conditions, the model can still maintain the rationality and stability of joint connections, better adapt to the complex conditions of power production sites, accurately restore human poses, and improve the adaptability and robustness to occluded scenes, providing reliable model support for human pose estimation in power production sites.

[0069] In summary, the model training process involved in this application is summarized as follows: See Figure 3First, the monitoring video from a substation or distribution room is input as a sample video stream. The video is segmented into frames, and a Region of Interest (ROI) is extracted. The ROI is the aforementioned sample human body region image. Next, the HR-Net architecture is used to perform initial pose estimation on the extracted video frames to obtain initial pose information. Then, Gaussian mixture modeling is used to fit the sequence of missing skeleton points, and Gaussian noise is added in multiple steps using skeleton point labels to fit the missing pose until the fitting reaches 50 steps. Then, the backdiffusion and neighboring frame skeleton point encoding stages begin. After that, MSE_Loss is calculated, and the model parameters are optimized and updated based on the calculation results. The model continues to be trained until the number of iterations reaches 50 steps. The current optimized network parameters are then fed back to complete the entire training process, so that the human pose can be evaluated later by loading the network parameters.

[0070] S40. Based on the complete skeleton point sequence, generate and output the human pose estimation results.

[0071] In this embodiment, the monitoring system generates and outputs human pose estimation results based on a complete skeletal point sequence. The complete skeletal point sequence accurately describes the position and connection relationships of various key points in the human body. By connecting and visualizing these skeletal points according to the human body structure, an intuitive human pose estimation result can be generated. In practical applications, the complete skeletal point sequence can be mapped to the human body region identified in each frame of video image according to the skeletal point topology indicated by the sequence. The skeletal point topology defines the connection methods and relative positional relationships between various skeletal points in the human body, such as the connection relationship between arm skeletal points and shoulder skeletal points. Through mapping, the skeletal points correspond to the actual positions of the human body. Then, the video stream to be identified after mapping is used as the human pose estimation result and output, thus intuitively displaying the human pose estimation result and making the result easier to understand and apply. Simultaneously, accurate human pose estimation results can provide important information for safety management at power production sites, such as timely detection of non-standard operating postures of workers and prevention of safety accidents.

[0072] In addition, in practical applications, human posture estimation results can be output in the form of images, animations, or data files, which facilitates subsequent applications and analysis. For example, in a power production monitoring system, the generated human posture estimation results can be displayed on the monitoring screen in real time, allowing monitoring personnel to intuitively understand the posture of the workers. The results can also be stored as data files for subsequent data analysis and safety assessment, providing intuitive and accurate information for power production site monitoring and safety assessment, enabling relevant personnel to understand the posture of workers in a timely manner, promptly identify potential safety hazards, and improve the safety management level of power production sites.

[0073] In summary, in practical applications, the process of generating human pose estimation results involved in this application can be as follows: See Figure 4 First, the system inputs monitoring video from a substation or distribution room. Using the HR-Net architecture, initial pose estimation is performed to obtain preliminary human pose information. Next, Gaussian mixture modeling is used to fit a sequence of missing skeletal points to characterize pose uncertainty. Then, an online server loads the parameters of an inverse diffusion network, providing a model foundation for subsequent pose recovery. Inverse diffusion and neighboring frame skeletal point encoding are then performed. Through multi-step iterative optimization (setting the number of steps to 50), the uncertainty in the pose is gradually removed, recovering a more accurate human pose. Finally, a precise two-dimensional human pose estimation result is output, which can be presented as a 17*2 skeletal coordinate sequence for applications such as safety monitoring and personnel behavior analysis at power production sites.

[0074] By training a model with generative modeling capabilities, the technical solution of this application is equipped with the ability to express multiple solutions and uncertainties. In scenarios with dense crowds of people, overlapping poses, or blurred key points, the model can generate multiple potentially reasonable pose candidates, avoiding the erroneous output caused by the "forced decision" of traditional single discriminative methods. This helps to improve the interpretability and fault tolerance of the overall system.

[0075] Furthermore, in the technical solution of this application, a pre-trained attitude detection network model can be used to design a pre-position estimation module in the monitoring system, and an inverse diffusion network model can be used to design an inverse diffusion module, thereby supporting flexible integration and engineering implementation. The pre-position estimation module provides efficient and lightweight initial positioning capabilities, while the subsequent inverse diffusion module can be deployed as needed in key areas or anomaly detection scenarios to achieve focused repair and enhancement, meeting the comprehensive requirements of real-time performance and accuracy in actual industrial scenarios.

[0076] Thus, the technical solution of this application not only improves the quality and completeness of attitude estimation, but also provides a high-quality data foundation for subsequent intelligent sensing tasks such as human behavior recognition, safe operation analysis, and abnormal operation detection, thereby promoting the construction of intelligent visual sensing systems in the power production environment.

[0077] The method provided in this application acquires a video stream to be identified, uses a pre-trained pose detection network model to perform initial pose estimation on each frame of the video stream to obtain an initial skeleton point sequence, and uses a Gaussian mixture modeling method to fit the initial skeleton point sequence to obtain a missing skeleton point sequence. The missing skeleton point sequence is input into a pre-trained inverse diffusion network model, and the missing skeleton point sequence is optimized by combining the skeleton point encoding information of neighboring frames of each video image in the inverse diffusion network model to obtain a complete skeleton point sequence. Based on the complete skeleton point sequence, a human pose estimation result is generated and output. The pose detection network model quickly locates the visible key points of human pose in the video stream, and uses the generation capability of the inverse diffusion network model to structurally complete the occluded parts of the human pose by referring to the skeleton point encoding information of neighboring frames of each video image, effectively restoring the complete skeleton structure. This enables accurate identification of human pose in high-occlusion and high-dynamic-change scenes such as power production sites, improving the adaptability and robustness to occluded scenes.

[0078] Furthermore, as Figure 1 To specifically implement the method, this application provides a human pose estimation result generation device based on a generative model, such as... Figure 5 As shown, the device includes: a front attitude estimation module 501, a fitting module 502, an inverse diffusion module 503, and an output module 504.

[0079] The pre-pose estimation module 501 is used to acquire the video stream to be identified, and to perform initial pose estimation on each frame of the video image in the video stream to be identified using a pre-trained pose detection network model to obtain an initial skeleton point sequence. The fitting module 502 is used to fit the initial bone point sequence using a Gaussian mixture modeling method to obtain the defective bone point sequence; The inverse diffusion module 503 is used to input the missing bone point sequence into a pre-trained inverse diffusion network model, and optimize the missing bone point sequence by combining the bone point encoding information of the neighboring frames of each video image in the inverse diffusion network model to obtain a complete bone point sequence. The output module 504 is used to generate and output human pose estimation results based on the complete skeleton point sequence.

[0080] In a specific application scenario, the pre-pose estimation module 501 is used to acquire real-time monitoring video streams of the target scene as the video stream to be identified, perform frame-by-frame processing on the video stream to be identified to obtain multiple video images; perform illumination normalization and resolution enhancement processing on each of the multiple video images, and input the processed multiple video images into the pose detection network model, whereby the pose detection network model performs initial pose estimation on each of the processed video images to obtain the initial skeleton point sequence.

[0081] In a specific application scenario, the fitting module 502 is used to obtain the Gaussian distribution of the historical human keypoint trajectory constructed using the Gaussian mixture modeling method. Combining the Gaussian distribution and the initial skeletal point sequence, the initial skeletal point sequence is remodeled to re-model the human posture using the following formula to obtain the missing skeletal point sequence:

[0082] in, This represents the sequence of the missing bone points. These represent the weighting coefficients used to balance the contributions of the Gaussian distribution and the time-series interpolator. This represents the Gaussian distribution. This represents the position of the i-th key point in the Gaussian distribution. This represents the variance of the Gaussian distribution. This refers to the timing interpolator. Indicates the previous moment The location of the next key point.

[0083] In specific application scenarios, the device further includes: The training module is used to acquire sample video streams, perform initial pose estimation on each frame of the sample video stream using the pose detection network model, and obtain sample skeleton point sequences. Gaussian noise is progressively added to the sample skeleton point sequences to generate noisy pose data. The noisy pose sequences of multiple consecutive sample video images in the noisy pose data are input into a Long Short-Term Memory (LSTM) network architecture for processing to obtain conditional codes that include temporal context information of human actions. Based on the noisy pose data and the conditional codes, a backdiffusion process is performed on a preset network architecture to obtain a diffusion processing result. The preset network architecture includes a Graph Convolutional Network (GCN) network and an attention stacking network. Using the diffusion processing result, a loss value is calculated, and the parameters of the preset network architecture are updated and optimized based on the loss value. The updated and optimized preset network architecture is then subjected to another backdiffusion process using the diffusion processing result until the number of backdiffusion processes reaches a threshold. The model formed by the currently trained preset network architecture is then used as the backdiffusion network model.

[0084] In a specific application scenario, the training module is used to perform frame-by-frame processing on the sample video stream to obtain multiple frames of sample video images; identify human body regions in each frame of the sample video images, and identify human body regions in each frame of the sample video images to obtain multiple frames of sample human body region images; input the multiple frames of sample human body region images into the pose detection network model, so that the pose detection network model performs initial pose estimation on each frame of the sample human body region image to obtain the sample skeleton point sequence.

[0085] In a specific application scenario, the training module is used to obtain a preset initial noise intensity coefficient, determine a preset number of noise additions, and construct an initial noise Gaussian distribution using the sample skeletal point sequence and the initial noise intensity coefficient. According to the following formula, sampling is performed from the initial noise Gaussian distribution to obtain the noise posture sequence after the first addition of Gaussian noise. Then, a second noise intensity coefficient is obtained, and a second noise Gaussian distribution is constructed using the noise posture sequence after the first addition of Gaussian noise and the second noise intensity coefficient. Sampling is then performed from the second noise Gaussian distribution according to the following formula to obtain the noise posture sequence after the second addition of Gaussian noise, until the preset number of noise additions is reached, thus obtaining the noise posture data.

[0086] in, Indicates from posture to posture The conditional probability distribution, Indicates the first The noise posture sequence after adding Gaussian noise. Indicates the first The noise posture sequence after adding Gaussian noise. Indicates noise intensity. The value of is a natural number between 1 and the preset number of noise additions, and The value of varies with It gradually increases with the increase of [something]. Indicates For the mean, The noise Gaussian distribution is the covariance matrix. This represents the preset identity matrix.

[0087] In a specific application scenario, the training module is used to input the noisy pose data and the conditional encoding into the GCN network in the preset network architecture. The GCN network processes and updates the topological relationships between the skeleton points in the noisy pose data with reference to the conditional encoding to obtain the processed noisy pose data. The processed noisy pose data and the conditional encoding are then input into the attention stacking network in the preset network structure. The attention stacking network assigns weights to the skeleton points in the noisy pose data with reference to the conditional encoding and updates the data. The processing result output by the attention stacking network is used as the diffusion processing result.

[0088] In specific application scenarios, the training module is used to acquire the real pose data corresponding to the current reverse diffusion processing round, and calculates the loss value by combining the diffusion processing result and the real pose data using the following formula:

[0089] in, Represents the loss function. Indicates the current round of reverse diffusion processing. The value ranges from 1 to the specified number threshold. This represents the threshold number of times. This indicates the diffusion processing result obtained after the current round of reverse diffusion processing. This represents the actual attitude data corresponding to the current reverse diffusion processing round.

[0090] In specific application scenarios, the output module is used to map the complete skeleton point sequence to the human body region identified on each frame of video image according to the skeleton point topology relationship indicated by the complete skeleton point sequence; and output the video stream to be identified after mapping as the human body pose estimation result.

[0091] The apparatus provided in this application acquires a video stream to be identified, performs initial pose estimation on each frame of the video stream using a pre-trained pose detection network model to obtain an initial skeletal point sequence, fits the initial skeletal point sequence using a Gaussian mixture modeling method to obtain a missing skeletal point sequence, inputs the missing skeletal point sequence into a pre-trained inverse diffusion network model, optimizes the missing skeletal point sequence by combining the skeletal point encoding information of neighboring frames of each video image in the inverse diffusion network model to obtain a complete skeletal point sequence, generates and outputs a human pose estimation result based on the complete skeletal point sequence, quickly locates the visible key points of human pose in the video stream through the pose detection network model, and utilizes the generation capability of the inverse diffusion network model to structurally complete the occluded parts of the human pose by referring to the skeletal point encoding information of neighboring frames of each video image, effectively restoring the complete skeleton structure, realizing accurate identification of human pose in high-occlusion and high-dynamic-change scenes such as power production sites, and improving the adaptability and robustness to occlusion scenes.

[0092] It should be noted that other corresponding descriptions of the functional units involved in the human pose estimation result generation device based on a generative model provided in this application embodiment can be found in the following references. Figures 1 to 4 The corresponding description in [the document] will not be repeated here.

[0093] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0094] The above embodiments and the technical features in the embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0095] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented in hardware or by using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) and includes several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0097] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application.

[0098] Those skilled in the art will understand that the modules in the apparatus of the implementation scenario can be distributed within the apparatus of the implementation scenario as described, or they can be located in one or more apparatuses different from this implementation scenario, with corresponding changes. The modules of the above-described implementation scenario can be combined into one module, or they can be further divided into multiple sub-modules.

[0099] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of the implementation scenario.

[0100] The above disclosures are only a few specific implementation scenarios of this application. However, this application is not limited to these. Any variations that can be conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A method for generating human pose estimation results based on a generative model, characterized in that, include: The video stream to be identified is acquired, and an initial pose estimation is performed on each frame of the video stream using a pre-trained pose detection network model to obtain an initial skeleton point sequence. The Gaussian mixture modeling method is used to fit the initial bone point sequence to obtain the missing bone point sequence; The missing bone point sequence is input into a pre-trained inverse diffusion network model. In the inverse diffusion network model, the missing bone point sequence is optimized by combining the bone point encoding information of the neighboring frames of each video image to obtain a complete bone point sequence. Based on the complete skeletal point sequence, human pose estimation results are generated and output.

2. The method according to claim 1, characterized in that, The process of acquiring the video stream to be identified involves using a pre-trained pose detection network model to perform initial pose estimation on each frame of the video stream to obtain an initial skeleton point sequence, including: The real-time monitoring video stream of the target scene is collected as the video stream to be identified, and the video stream to be identified is processed by frame segmentation to obtain multiple frames of video images; Each frame of the multi-frame video image is subjected to illumination normalization and resolution enhancement processing, and the processed multi-frame video image is input into the pose detection network model. The pose detection network model performs initial pose estimation on each processed video image to obtain the initial skeleton point sequence.

3. The method according to claim 1, characterized in that, The Gaussian mixture modeling method is used to fit the initial skeletal point sequence to obtain the defective skeletal point sequence, including: Obtain the Gaussian distribution of the historical human keypoint trajectory constructed using the Gaussian mixture modeling method. Combine the Gaussian distribution with the initial skeletal point sequence, and use the following formula to remodel the human pose of the initial skeletal point sequence to obtain the missing skeletal point sequence: in, This represents the sequence of the missing bone points. These represent the weighting coefficients used to balance the contributions of the Gaussian distribution and the time-series interpolator. This represents the Gaussian distribution. This represents the position of the i-th key point in the Gaussian distribution. This represents the variance of the Gaussian distribution. This refers to the timing interpolator. Indicates the previous moment The location of the next key point.

4. The method according to claim 1, characterized in that, Before inputting the missing skeleton point sequence into a pre-trained inverse diffusion network model, and optimizing the missing skeleton point sequence by combining the skeleton point encoding information of neighboring frames of each video image in the inverse diffusion network model to obtain a complete skeleton point sequence, the method further includes: Acquire a sample video stream, and use the pose detection network model to perform initial pose estimation on each frame of the video image in the sample video stream to obtain a sample skeleton point sequence; Gaussian noise is gradually added to the sample skeletal point sequence to generate noisy pose data; The noisy pose sequence of multiple consecutive sample video images in the noisy pose data is input into the Long Short-Term Memory (LSTM) network architecture for processing to obtain conditional encoding that includes the temporal context information of human actions. Based on the noise pose data and the conditional coding, a reverse diffusion process is performed on the preset network architecture to obtain the diffusion processing result. The preset network architecture includes a graph convolutional network (GCN) and an attention stacking network. Using the diffusion processing results, a loss value is calculated, and the parameters of the preset network architecture are updated and optimized based on the loss value. The updated and optimized preset network architecture is then subjected to another reverse diffusion process using the diffusion processing results until the number of reverse diffusion processes reaches a threshold. The model formed by the currently trained preset network architecture is then used as the reverse diffusion network model.

5. The method according to claim 4, characterized in that, The process of acquiring the sample video stream involves using the pose detection network model to perform initial pose estimation on each frame of the sample video stream to obtain a sample skeleton point sequence, including: The sample video stream is segmented into frames to obtain multiple sample video images; Human body regions are identified in each frame of the sample video image, and human body regions in each frame of the sample video image are identified to obtain multi-frame sample human body region images. The multi-frame sample human body region images are input into the pose detection network model so that the pose detection network model performs an initial pose estimation for each frame of the sample human body region image to obtain the sample skeletal point sequence.

6. The method according to claim 4, characterized in that, The step of progressively adding Gaussian noise to the sample skeletal point sequence to generate noisy pose data includes: Obtain the preset initial noise intensity coefficient, determine the preset number of noise additions, and construct the initial noise Gaussian distribution using the sample skeleton point sequence and the initial noise intensity coefficient; According to the following formula, sampling is performed from the initial noise Gaussian distribution to obtain the noise posture sequence after the first addition of Gaussian noise, and a second noise intensity coefficient is obtained. Using the noise posture sequence after the first addition of Gaussian noise and the second noise intensity coefficient, a second noise Gaussian distribution is constructed, and sampling is performed from the second noise Gaussian distribution according to the following formula to obtain the noise posture sequence after the second addition of Gaussian noise, until the number of noise additions reaches the preset number of noise additions, thus obtaining the noise posture data: in, Indicates from posture to posture The conditional probability distribution, Indicates the first The noise posture sequence after adding Gaussian noise. Indicates the first The noise posture sequence after adding Gaussian noise. Indicates noise intensity. The value of is a natural number between 1 and the preset number of noise additions, and The value of varies with It gradually increases with the increase of [something]. Indicates For the mean, The noise Gaussian distribution is the covariance matrix. This represents the preset identity matrix.

7. The method according to claim 4, characterized in that, The step of performing reverse diffusion processing on the preset network architecture based on the noise pose data and the conditional coding to obtain the diffusion processing result includes: The noise pose data and the conditional coding are input into the GCN network in the preset network architecture. The GCN network processes the topological relationship between the skeleton points in the noise pose data with reference to the conditional coding and updates it to obtain the processed noise pose data. The processed noise pose data and the conditional encoding are input into the attention stacking network in the preset network structure. The attention stacking network assigns weights to the skeleton points in the noise pose data with reference to the conditional encoding and updates them. The processing result output by the attention stacking network is used as the diffusion processing result.

8. The method according to claim 4, characterized in that, The calculation of the loss value using the diffusion processing result includes: Obtain the actual pose data corresponding to the current reverse diffusion processing round, and calculate the loss value by applying the diffusion processing result and the actual pose data using the following formula: in, Represents the loss function. Indicates the current round of reverse diffusion processing. The value ranges from 1 to the specified number threshold. This represents the threshold number of times. This indicates the diffusion processing result obtained after the current round of reverse diffusion processing. This represents the actual attitude data corresponding to the current reverse diffusion processing round.

9. The method according to claim 1, characterized in that, The process of generating and outputting human pose estimation results based on the complete skeletal point sequence includes: According to the skeletal point topology indicated by the complete skeletal point sequence, the complete skeletal point sequence is mapped to the human body region identified on each frame of video image; The mapped video stream to be identified is used as the human pose estimation result and output.

10. A device for generating human pose estimation results based on a generative model, characterized in that, include: The pre-pose estimation module is used to acquire the video stream to be identified, and to perform initial pose estimation on each frame of the video image in the video stream to be identified using a pre-trained pose detection network model to obtain an initial skeleton point sequence. The fitting module is used to fit the initial bone point sequence using a Gaussian mixture modeling method to obtain the defective bone point sequence. The inverse diffusion module is used to input the missing bone point sequence into a pre-trained inverse diffusion network model. In the inverse diffusion network model, the missing bone point sequence is optimized by combining the bone point encoding information of the neighboring frames of each video image to obtain a complete bone point sequence. The output module is used to generate and output human pose estimation results based on the complete skeleton point sequence.

Images