Road construction cone barrel falling detection method based on attitude estimation

Abstract

The invention discloses the field of computer vision, and particularly discloses a road construction cone barrel falling detection method based on attitude estimation. Comprising the following steps: S1, collecting a cone barrel image; s2, multi-dimensional features of the cone-barrel image are extracted, the multi-dimensional features comprise contour features, edge features and color features, and the extraction effect of the multi-dimensional features is compensated through adaptive brightness; s3, key point detection and key point connection are carried out on the cone-barrel image according to the multi-dimensional features, and image information used for attitude estimation is acquired; s4, performing attitude estimation on the image information to obtain a predicted attitude; and S5, generating a safety early warning signal according to the predicted attitude. Through combination of multi-dimensional feature extraction, key point detection and attitude estimation, the state of the cone barrel is judged in real time, the safety early warning signal is generated according to the predicted attitude, and the safety management level of a construction area is remarkably improved.

Description

Technical Field

[0001] The invention relates to the field of computer vision, and in particular to a method for detecting the fall of road construction cones based on posture estimation. Background Art

[0002] Road construction is an important part of urban infrastructure construction and maintenance, but the safety management of construction areas has always faced major challenges. Among them, the status of signage facilities in construction areas is a key factor affecting road safety. Cone warning barrels are basic traffic control facilities. The correct placement and upright state of cone warning barrels are crucial to guiding drivers to slow down and change lanes and reduce the risk of traffic accidents in construction areas. However, traditional manual inspection methods are inefficient and costly, and it is even more difficult to achieve comprehensive and real-time monitoring in bad weather and long-distance construction scenarios.

[0003] Existing cone status monitoring methods mainly rely on image recognition and area division technology, and use preset thresholds to determine whether the cone is in a specific area. This type of method has obvious shortcomings: the existing method can usually only detect whether the cone is in the predetermined area, and it is difficult to accurately determine the standing or falling state of the cone, resulting in frequent missed reports and false alarms, affecting the accuracy of construction safety management; although manual observation based on video surveillance can capture the state of the cone, it lacks a special algorithm for the fallen state and relies entirely on manual analysis, which is difficult to meet the real-time monitoring needs of large-scale construction areas; under complex lighting conditions, such as reflection on the surface of the cone, shadow interference and light source changes, the existing detection method is prone to recognition errors, affecting the monitoring effect.

[0004] To sum up, the current cone barrel status monitoring technology has obvious shortcomings in detection accuracy, real-time and environmental adaptability. A technical means is needed to efficiently, accurately and in real time monitor the standing and falling status of cone barrels in order to improve the safety management level of the construction area. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art in the monitoring of the status of cone warning barrels for road construction, such as low efficiency, insufficient accuracy and lack of real-time performance, and to provide an intelligent detection method for the fallen state of cone barrels based on posture estimation. The present invention aims to solve the problems of low efficiency, high cost, missed reports and false reports of traditional manual inspection methods, while overcoming the limitation of the prior art that it is impossible to accurately judge the standing or falling state of cone barrels. By introducing computer vision and deep learning technology, a cone barrel posture estimation framework is constructed, which can identify the standing or falling state of cone barrels in real time. Through multi-dimensional feature extraction, adaptive brightness compensation and key point detection, the accuracy problem of cone barrel feature extraction in complex brightness and occlusion environments is solved. Further combined with rigid constraints and rotational invariance, a highly robust posture estimation algorithm is constructed, which can operate efficiently in large-scale construction scenarios.

[0006] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0007] A method for detecting the fall of road construction cones based on posture estimation comprises the following steps:

[0008] S1. Collect cone barrel images;

[0009] S2. extracting multi-dimensional features of the cone barrel image, wherein the multi-dimensional features include contour features, edge features and color features, and compensating the extraction effect of the multi-dimensional features by adaptive brightness;

[0010] S3. Perform key point detection and key point connection on the cone barrel image according to the multi-dimensional features to obtain image information for posture estimation;

[0011] S4. Estimating the posture of the image information to obtain a predicted posture;

[0012] S5. Generate a safety warning signal based on the predicted posture.

[0013] Preferably, extracting the contour features in step S2 includes the following steps:

[0014] S211. grayscale and denoise the cone barrel image;

[0015] S212. Construct a scale pyramid of downsampling at different multiples, and obtain a multi-scale spatial representation through Gaussian pyramid decomposition;

[0016] S213. Extract the contour features of the cone barrel in the cone barrel image in each scale space and calculate the Hu moment features.

[0017] By graying the cone barrel image, performing denoising and extracting multi-scale Hu moment features, the cone barrel contour features can be effectively extracted. The scale pyramid is used to perform multi-scale decomposition of the image to capture the cone barrel contour features at different resolutions, which enhances the algorithm's ability to adapt to changes in target scale. Based on the invariance of the Hu moment, the geometric shape features of the cone barrel are ensured to remain stable under rotation, translation and scaling, laying a stable geometric foundation for subsequent posture estimation. Combined with graying and denoising, the accuracy of contour feature extraction is significantly improved, and the interference of environmental noise on feature recognition is reduced.

[0018] Preferably, the edge feature extraction of the cone barrel in step S2 adopts the Canny operator algorithm, which includes the following steps:

[0019] S221. Obtaining the initial edge by calculating the cone image gradient amplitude and direction;

[0020] S222. Processing the gradient amplitude and the direction of the cone barrel image, and refining the initial edge by non-maximum suppression;

[0021] S223. Calculate the high threshold and low threshold of the refined edge based on the local variance of the content of the cone barrel image;

[0022] S224. Connect the real edge points through edge tracking and hysteresis threshold, remove the pseudo edges and noise, and obtain edge features.

[0023] The Canny operator is used to extract edge features from the cone barrel image. Through non-maximum suppression and dynamic threshold adjustment, pseudo edges and noise are effectively removed, and the edge contour of the cone barrel can be accurately extracted. The high and low thresholds are dynamically set in combination with local variance calculation, making the edge feature extraction more robust to lighting changes, reflections and shadow interference.

[0024] Preferably, the extraction of the color features in step S2 includes the following steps:

[0025] S231. Convert the cone barrel image from the RGB color space to the HSV color space, and extract the characteristic information of hue H, saturation S and brightness V;

[0026] S232. Setting a threshold range of the HSV color space for identifying the color region of the cone barrel image;

[0027] S233. Combine the color reference value in the RGB color space, calculate the color distance of the pixel, and filter the cone color area that meets the conditions;

[0028] S234. Compare the threshold range and the color distance, optimize the screening result, and extract the color features of the cone barrel image.

[0029] By performing color feature extraction in the RGB-HSV two-color space, the present invention can still accurately identify the color features of the cone barrel under complex lighting conditions. Specifically, by combining the hue, saturation, and brightness characteristics of the HSV color space and the color distance calculation of the RGB space, the color information of the cone barrel can be accurately extracted. Setting the dynamic threshold range of the HSV color space and combining the regional color reference value significantly improves the algorithm's adaptability to brightness changes and reflective interference. By comparing and screening color distances, the color feature extraction results are optimized, providing more reliable feature support for subsequent posture estimation.

[0030] Preferably, the method comprises adopting a regional adaptive brightness compensation method to extract the features of the cone barrel image, comprising the following steps:

[0031] S241. Perform local brightness statistics and gradient distribution analysis on the Hu moment feature to extract contour features at different brightnesses;

[0032] S242. A dynamic threshold adjustment mechanism is introduced to the edge feature, edge detection parameters are adaptively adjusted according to local contrast, and pseudo edges caused by brightness changes are constrained by edge continuity to extract edge features at different brightnesses;

[0033] S243. Adopting a region segmentation adaptive brightness mapping method for the color features and combining color constraints, extracting color features under different brightness.

[0034] The introduction of regional adaptive brightness compensation method further improves the accuracy and environmental adaptability of multi-dimensional feature extraction. Through local brightness statistics and gradient distribution analysis, the extraction effect of Hu moment contour features and edge features is adaptively adjusted to enhance the robustness of the algorithm in scenes with brightness changes; the dynamic threshold adjustment mechanism and edge continuity constraint are used to solve the influence of material reflection and shadow on edge feature extraction; the adaptive brightness mapping algorithm combined with regional segmentation effectively extracts color features under complex lighting conditions, providing more accurate data support for cone barrel state determination.

[0035] Preferably, in step S3, the key point detection model is used to identify the image information of the cone barrel image, comprising the following steps:

[0036] S31. Using a three-point marking method to mark key points of the cone barrel image to obtain annotation data;

[0037] S32. Input the labeled data into a key point detection model based on the Cascaded Pyramid Model, perform key point detection, and output the predicted coordinates of the key points;

[0038] S33. According to the predicted coordinates, key points are connected to convert the predicted coordinates into image information for posture estimation.

[0039] The geometric structure of the cone barrel is accurately identified through the key point detection model, achieving efficient input for posture estimation. The three-point marking method and Cascaded Pyramid Model (CPM) model are used to accurately detect and connect the key points at the top and bottom of the cone barrel to ensure the accuracy of the key point coordinates. Key point detection has strong adaptability to complex backgrounds, occlusions and lighting changes, providing reliable data input for subsequent posture estimation.

[0040] Preferably, step S4 uses a method of main direction vector attitude representation to estimate the attitude of the image information, comprising the following steps:

[0041] S411. According to the geometric characteristics of the cone barrel, a rotation matrix and a main direction vector are constructed to describe the state of the cone barrel in three-dimensional space;

[0042] S412. Input the multi-dimensional features into a deep learning model based on the rigid_transform layer, and generate rotation features through grid sampling;

[0043] S413. stacking a plurality of the rotation features and performing a maximum pooling operation to extract the cone barrel posture features with rotation invariance, and generating a cone barrel posture representation;

[0044] S414. According to the state discrimination method based on the main direction vector, target posture analysis is performed on the posture representation, posture estimation is performed on the cone barrel, and a predicted posture is obtained.

[0045] By adopting a method based on the main direction vector posture representation, the rotation-invariant posture feature of the cone barrel is constructed. Specifically, the three-dimensional spatial state of the cone barrel is described by the rotation matrix and the main direction vector in the rigid body transformation, ensuring that the posture features remain consistent under rotation and translation, thereby improving the stability of the posture estimation. The rigid_transform layer is used to perform grid sampling and feature stacking on multi-dimensional features, and the rotation-invariant posture features are extracted through maximum pooling, which effectively enhances the representation ability of the posture features. The state discrimination method based on the main direction vector can accurately distinguish the vertical and dumped states of the cone barrel, providing reliable support for the generation of safety warning signals.

[0046] Preferably, step S4 uses a method of projection invariant and affine transformation geometric representation to estimate the posture of the image information and obtain a predicted posture; wherein, when the ratio of the height of the cone barrel projection to the length of the bottom side is less than a preset threshold, the predicted posture is a fallen state. Using the ratio of the height of the cone barrel to the length of the bottom side as a judgment criterion, the projection geometric characteristics are analyzed through affine transformation to accurately judge the fallen state of the cone barrel, thereby avoiding misjudgment caused by projection compression in traditional methods. Constructing judgment criteria based on geometric characteristics and projection invariants is intuitive and easy to implement, and has strong applicability in complex scenes.

[0047] Preferably, step S3 further includes marking key points of the cone barrel image to obtain the true posture; according to the difference between the predicted posture and the true posture, step S4 further includes optimizing the posture estimation, including the following steps:

[0048] S421. Constructing a loss function according to the predicted posture and the true posture;

[0049] S422. Using the loss function as the input of the stochastic gradient descent algorithm, dynamically adjusting the learning rate in combination with the Adam optimizer, and outputting the optimized posture estimation result;

[0050] S423. Use the early stopping mechanism and the validation set to evaluate the optimized pose estimation results. When the validation set loss reaches a preset value of epochs, terminate the iteration.

[0051] Through the optimization steps of key point annotation and predicted posture, the accuracy and robustness of posture estimation are further improved. Specifically, the loss function is constructed using the difference between the predicted posture and the true posture, and the model parameters are dynamically optimized with the stochastic gradient descent algorithm to effectively reduce the error of posture estimation; combined with the Adam optimizer and early stopping mechanism, while improving the optimization accuracy, the computational cost and overfitting risk in the training process are reduced; through continuous iterative optimization, the model can adapt to the complexity of different scenes and generate high-quality posture estimation results.

[0052] Preferably, the loss function in step S421 includes 1) a Euclidean distance metric representing the predicted posture and the true posture and 2) a regularization constraint on the predicted posture. This preferred solution uses the Euclidean distance metric between the predicted posture and the true posture as the core part of the loss function, directly quantifies the difference between the two, and improves the accuracy of posture estimation. By adding regularization constraints, the overfitting of the model is suppressed, the generalization ability of the model is enhanced, and it is suitable for complex construction environments.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention provides a method for detecting the fall of road construction cones based on posture estimation. By combining multi-dimensional feature extraction, key point detection and posture estimation, the accuracy and real-time performance of cone fall detection are significantly improved. Specifically, a comprehensive extraction method of contour features, edge features and color features is adopted to fully capture the geometry, shape and color information of the cones, providing a comprehensive feature basis for posture estimation. Combined with the regional adaptive brightness compensation algorithm, the problems of cone surface reflection, brightness change and shadow interference are solved, and the stability and environmental adaptability of feature extraction are enhanced. In addition, through key point detection and key point connection methods, the geometric structure information of the cones is accurately obtained, providing reliable data support for posture estimation and state determination. The present invention judges the state of the cones in real time through posture estimation, and generates a safety warning signal according to the predicted posture, which significantly improves the safety management level of the construction area. The overall method has high robustness, real-time and environmental adaptability, and can be widely used in safety monitoring of complex construction scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a method flow chart of a method for detecting the fall of a road construction cone based on posture estimation according to Embodiment 1 of the present invention;

[0056] Figure 2A technical roadmap of a method for detecting the fall of a road construction cone based on posture estimation according to Embodiment 2 of the present invention;

[0057] Figure 3 This is a structural diagram of the rigid_transform layer of Embodiment 2 of the present invention;

[0058] Figure 4 This is an example diagram of cone barrel image annotation according to Embodiment 3 of the present invention;

[0059] Figure 5 This is a visualization diagram of the cone barrel detection frame and key point prediction of Example 3 of the present invention;

[0060] Figure 6 This is a schematic diagram of a model training evaluation curve of Example 3 of the present invention;

[0061] Figure 7 This is a schematic diagram of the cone bucket fall detection result of Example 3 of the present invention. DETAILED DESCRIPTION

[0062] The following is a further detailed description of a road construction cone fall detection method based on posture estimation provided by the present invention in conjunction with the accompanying drawings and specific embodiments. However, this should not be understood as the scope of the above-mentioned subject matter of the present invention being limited to the following embodiments, and all technologies implemented based on the content of the present invention belong to the scope of the present invention. Combined with the following description, the advantages and features of the present invention will become clearer. It should be noted that the drawings are all in a very simplified form and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0063] Example 1

[0064] A road construction cone fall detection method based on posture estimation, such as Figure 1 As shown, the following steps are included:

[0065] S1. Collect cone barrel images;

[0066] S2. extracting multi-dimensional features of the cone barrel image, wherein the multi-dimensional features include contour features, edge features and color features, and compensating the extraction effect of the multi-dimensional features by adaptive brightness;

[0067] S3. Perform key point detection and key point connection on the cone barrel image according to the multi-dimensional features to obtain image information for posture estimation;

[0068] S4. Estimating the posture of the image information to obtain a predicted posture;

[0069] S5. Generate a safety warning signal based on the predicted posture.

[0070] Preferably, extracting the contour features in step S2 includes the following steps:

[0071] S211. grayscale and denoise the cone barrel image;

[0072] S212. Construct a scale pyramid of downsampling at different multiples, and obtain a multi-scale spatial representation through Gaussian pyramid decomposition;

[0073] S213. Extract the contour features of the cone barrel in the cone barrel image in each scale space and calculate the Hu moment features.

[0074] By graying the cone barrel image, performing denoising and extracting multi-scale Hu moment features, the cone barrel contour features can be effectively extracted. The scale pyramid is used to perform multi-scale decomposition of the image to capture the cone barrel contour features at different resolutions, which enhances the algorithm's ability to adapt to changes in target scale. Based on the invariance of the Hu moment, the geometric shape features of the cone barrel are ensured to remain stable under rotation, translation and scaling, laying a stable geometric foundation for subsequent posture estimation. Combined with graying and denoising, the accuracy of contour feature extraction is significantly improved, and the interference of environmental noise on feature recognition is reduced.

[0075] Preferably, the edge feature extraction of the cone barrel in step S2 adopts the Canny operator algorithm, which includes the following steps:

[0076] S221. Obtaining the initial edge by calculating the cone image gradient amplitude and direction;

[0077] S222. Processing the gradient amplitude and the direction of the cone barrel image, and refining the initial edge by non-maximum suppression;

[0078] S223. Calculate the high threshold and low threshold of the refined edge based on the local variance of the content of the cone barrel image;

[0079] S224. Connect the real edge points through edge tracking and hysteresis threshold, remove the pseudo edges and noise, and obtain edge features.

[0080] The Canny operator is used to extract edge features from the cone barrel image. Through non-maximum suppression and dynamic threshold adjustment, pseudo edges and noise are effectively removed, and the edge contour of the cone barrel can be accurately extracted. The high and low thresholds are dynamically set in combination with local variance calculation, making the edge feature extraction more robust to lighting changes, reflections and shadow interference.

[0081] Preferably, the extraction of the color features in step S2 includes the following steps:

[0082] S231. Convert the cone barrel image from the RGB color space to the HSV color space, and extract the characteristic information of hue H, saturation S and brightness V;

[0083] S232. Setting a threshold range of the HSV color space for identifying the color region of the cone barrel image;

[0084] S233. Calculate the color distance of the pixel by combining the color reference value in the RGB color space, and filter the cone color area that meets the conditions;

[0085] S234. Compare the threshold range and the color distance, optimize the screening result, and extract the color features of the cone barrel image.

[0086] By performing color feature extraction in the RGB-HSV two-color space, the present invention can still accurately identify the color features of the cone barrel under complex lighting conditions. Specifically, by combining the hue, saturation, and brightness characteristics of the HSV color space and the color distance calculation of the RGB space, the color information of the cone barrel can be accurately extracted. Setting the dynamic threshold range of the HSV color space and combining the regional color reference value significantly improves the algorithm's adaptability to brightness changes and reflective interference. By comparing and screening color distances, the color feature extraction results are optimized, providing more reliable feature support for subsequent posture estimation.

[0087] Preferably, a regional adaptive brightness compensation method is used to extract the features of the cone barrel image, comprising the following steps:

[0088] S241. Perform local brightness statistics and gradient distribution analysis on the Hu moment feature to extract contour features at different brightnesses;

[0089] S242. A dynamic threshold adjustment mechanism is introduced to the edge feature, edge detection parameters are adaptively adjusted according to local contrast, and pseudo edges caused by brightness changes are constrained by edge continuity to extract edge features at different brightnesses;

[0090] S243. Adopting a region segmentation adaptive brightness mapping method for the color features and combining color constraints, extracting color features under different brightness.

[0091] The introduction of regional adaptive brightness compensation method further improves the accuracy and environmental adaptability of multi-dimensional feature extraction. Through local brightness statistics and gradient distribution analysis, the extraction effect of Hu moment contour features and edge features is adaptively adjusted to enhance the robustness of the algorithm in scenes with brightness changes; the dynamic threshold adjustment mechanism and edge continuity constraint are used to solve the influence of material reflection and shadow on edge feature extraction; the adaptive brightness mapping algorithm combined with regional segmentation effectively extracts color features under complex lighting conditions, providing more accurate data support for cone barrel state determination.

[0092] Preferably, in step S3, the key point detection model is used to identify the image information of the cone barrel image, comprising the following steps:

[0093] S31. Using a three-point marking method to mark key points of the cone barrel image to obtain annotation data;

[0094] S32. Input the labeled data into a key point detection model based on the Cascaded Pyramid Model, perform key point detection, and output the predicted coordinates of the key points;

[0095] S33. According to the predicted coordinates, key points are connected to convert the predicted coordinates into image information for posture estimation.

[0096] The geometric structure of the cone barrel is accurately identified through the key point detection model, achieving efficient input for posture estimation. The three-point marking method and Cascaded Pyramid Model (CPM) model are used to accurately detect and connect the key points at the top and bottom of the cone barrel to ensure the accuracy of the key point coordinates. Key point detection has strong adaptability to complex backgrounds, occlusions and lighting changes, providing reliable data input for subsequent posture estimation.

[0097] Preferably, step S4 uses a method of main direction vector attitude representation to estimate the attitude of the image information, comprising the following steps:

[0098] S411. According to the geometric characteristics of the cone barrel, a rotation matrix and a main direction vector are constructed to describe the state of the cone barrel in three-dimensional space;

[0099] S412. Input the multi-dimensional features into a deep learning model based on the rigid_transform layer, and generate rotation features through grid sampling;

[0100] S413. stacking a plurality of the rotation features and performing a maximum pooling operation to extract the cone barrel posture features with rotation invariance, and generating a cone barrel posture representation;

[0101] S414. According to the state discrimination method based on the main direction vector, target posture analysis is performed on the posture representation, posture estimation is performed on the cone barrel, and a predicted posture is obtained.

[0102] By adopting a method based on the main direction vector posture representation, the rotation-invariant posture feature of the cone barrel is constructed. Specifically, the three-dimensional spatial state of the cone barrel is described by the rotation matrix and the main direction vector in the rigid body transformation, ensuring that the posture features remain consistent under rotation and translation, thereby improving the stability of the posture estimation. The rigid_transform layer is used to perform grid sampling and feature stacking on multi-dimensional features, and the rotation-invariant posture features are extracted through maximum pooling, which effectively enhances the representation ability of the posture features. The state discrimination method based on the main direction vector can accurately distinguish the vertical and dumped states of the cone barrel, providing reliable support for the generation of safety warning signals.

[0103] Preferably, step S4 uses a method of projection invariant and affine transformation geometric representation to estimate the posture of the image information and obtain a predicted posture; wherein, when the ratio of the height of the cone barrel projection to the length of the bottom side is less than a preset threshold, the predicted posture is a fallen state. Using the ratio of the height of the cone barrel to the length of the bottom side as a judgment criterion, the projection geometric characteristics are analyzed through affine transformation to accurately judge the fallen state of the cone barrel, thereby avoiding misjudgment caused by projection compression in traditional methods. Constructing judgment criteria based on geometric characteristics and projection invariants is intuitive and easy to implement, and has strong applicability in complex scenes.

[0104] Preferably, step S3 further includes marking key points of the cone barrel image to obtain the true posture; according to the difference between the predicted posture and the true posture, step S4 further includes optimizing the posture estimation, including the following steps:

[0105] S421. Constructing a loss function according to the predicted posture and the true posture;

[0106] S422. Using the loss function as the input of the stochastic gradient descent algorithm, dynamically adjusting the learning rate in combination with the Adam optimizer, and outputting the optimized posture estimation result;

[0107] S423. Use the early stopping mechanism and the validation set to evaluate the optimized pose estimation results. When the validation set loss reaches a preset value of epochs, terminate the iteration.

[0108] Through the optimization steps of key point annotation and predicted posture, the accuracy and robustness of posture estimation are further improved. Specifically, the loss function is constructed using the difference between the predicted posture and the true posture, and the model parameters are dynamically optimized with the stochastic gradient descent algorithm to effectively reduce the error of posture estimation; combined with the Adam optimizer and early stopping mechanism, while improving the optimization accuracy, the computational cost and overfitting risk in the training process are reduced; through continuous iterative optimization, the model can adapt to the complexity of different scenes and generate high-quality posture estimation results.

[0109] Preferably, the loss function in step S421 includes 1) a Euclidean distance metric representing the predicted posture and the true posture and 2) a regularization constraint on the predicted posture. This preferred solution uses the Euclidean distance metric between the predicted posture and the true posture as the core part of the loss function, directly quantifies the difference between the two, and improves the accuracy of posture estimation. By adding regularization constraints to suppress model overfitting, the generalization ability of the model is enhanced, making it suitable for complex construction environments.

[0110] Example 2

[0111] Example 2 As a preferred solution of the above-mentioned embodiment, a specific implementation method of a road construction cone bucket fall detection method based on posture estimation is provided. The technical route is as follows: Figure 2 As shown, the main steps are as follows:

[0112] (1) Feature extraction: This stage uses three parallel feature extraction methods to process the input cone barrel image and compensates the feature extraction effect through adaptive brightness.

[0113] (2) Key point detection: Based on the features extracted in step (1), the key point detection network is used to locate the key points of the cone barrel. The CPM network can accurately identify and locate the feature points of the cone barrel. This image information is crucial for subsequent posture estimation.

[0114] (3) Posture estimation: At this stage, the cone barrel is rotationally invariant based on rigid constraints. The system processes the detected image information, combines the main direction posture representation with the projection geometry features, and estimates the cone barrel posture. The three-dimensional spatial information is converted into a two-dimensional image posture representation to achieve accurate estimation of the cone barrel position and orientation.

[0115] (4) Posture optimization: The loss function is optimized using an iterative optimization strategy based on gradient descent. A mechanism for minimizing the error between the predicted posture and the true posture is established, which significantly improves the accuracy of posture estimation and provides a reliable technical guarantee for practical applications.

[0116] According to the above steps, the main algorithm principles and step implementation processes involved are as follows:

[0117] S1 multi-dimensional feature extraction

[0118] S1.1 Multi-scale Hu moment contour extraction

[0119] Multi-scale Hu moment is used to extract the cone barrel contour features. First, the image is grayed and denoised, and a scale pyramid with 2x and 4x downsampling is constructed. Specifically, the cone barrel image is first decomposed by Gaussian pyramid to obtain a series of scale space representations. At each scale, the contour C(σ) is extracted and the Hu moment feature {μ'p,q (σ)}, these features are composed of seven invariant moments, defined as shown in formula (1):

[0120]

[0121] Where p and q are the power exponents of the difference between the offset and the centroid of the contour coordinates x and y, respectively, (x(σ), y(σ)) is the centroid of the contour C(σ), N(σ) is the number of pixels on the contour, f(x, y, σ) is the intensity value of the contour image, and σ represents the scale parameter.

[0122] S1.2 Canny operator edge extraction

[0123] The Canny operator refines the edge by non-maximum suppression, and then applies dual thresholds Th and Tl to determine the real and potential edge points. Finally, through edge tracking and hysteresis threshold technology, the edge points are connected and the noise is removed, so as to effectively extract the edge features of the cone barrel. The algorithm first calculates the gradient magnitude M(x, y) and direction θ(x, y) of the image, where the gradient magnitude is obtained by the Sobel operator:

[0124]

[0125] Next, determine the high threshold Th and the low threshold Tl, which are based on the local variance σ of the image content 2 , the calculation formula is as follows:

[0126] Th=mean+k·σ (5)

[0127] Tl=k·Th (6)

[0128] Where, mean is the local mean of the image gradient, and k is the weight coefficient, which is usually between 0.5 and 0.7. S1.3 HSV-RGB two-color space extraction

[0129] First, convert the image from RGB color space to HSV color space to better separate color information. The conversion formula is as follows:

[0130]

[0131] Wherein, V=(R+G+B) / 3, H represents hue, S represents saturation, and V represents brightness.

[0132] Next, the color of the cone is extracted by setting the threshold range in the HSV color space. For example, for the orange cone, the range of H can be set to [11°, 25°] or [157°, 180°], and the thresholds of S and V are adjusted according to the actual brightness conditions. At the same time, combined with the color information in the RGB space, the color distance is calculated by the following formula:

[0133]

[0134] In the formula, (R 0 ,G 0 ,B 0 ) is the reference value of the cone color, and (R, G, B) is the RGB value of each pixel in the image. By comparing the color distance d with the preset threshold, the cone color area that meets the conditions can be further screened out, thereby achieving accurate extraction of the cone color.

[0135] S1.4 Regional Adaptive Brightness Compensation

[0136] Cone barrels can be effectively identified by extracting their contours, edges, and color features. However, in certain specific conditions, simple feature fusion extraction alone cannot achieve good results. For example, the reflective tape on the surface of the cone barrel produces local highlights, the dynamic changes of natural light and artificial light sources, and the shadows cast by surrounding objects are the main factors affecting the extraction of cone barrel features. In order to reduce the impact of the above reasons on the extraction of cone barrel features, a regional adaptive brightness compensation method is introduced.

[0137] First, the local brightness statistics and gradient distribution analysis of the extracted Hu moment contour features are performed to ensure the stability of the contour features under different brightness conditions; then, for the edge features extracted by the Canny operator, a dynamic threshold adjustment mechanism is introduced to adaptively adjust the edge detection parameters according to the local contrast, and the edge continuity constraint is used to suppress the pseudo edges caused by brightness changes; finally, in the HSV-RGB two-color space feature extraction, an adaptive brightness mapping based on region segmentation is used, combined with color preservation constraints, to ensure the stability of color features when brightness changes. The calculation formula for local brightness is:

[0138] L(x,y)=I(x,y)*G(x,y) (11)

[0139]

[0140] Where the symbol * represents the convolution operation, K is the normalization constant, G represents the Gaussian function, and σ is the standard deviation of the Gaussian function. The Gaussian function is defined to satisfy the equation:

[0141] ∫∫G(x,y)dxdy=1 (13)

[0142] The local brightness L obtained by equations (2) and (3) allows the calculation of the reflection component R (i.e., the enhancement result):

[0143]

[0144] In order to achieve the effective fusion of the three features, a weighted fusion strategy is adopted:

[0145]

[0146] The variable n represents the scale level, W n Represents the weight of the nth feature. For n=3, the weight is W 1 =W 2 =W 3 =1 / 3.

[0147] S2 key point detection

[0148] Through the aforementioned adaptive brightness compensation feature extraction method, the key features of the cone barrel have been successfully extracted. The next step is to accurately detect key points based on these feature information to lay the foundation for subsequent posture estimation.

[0149] S2.1 Key point annotation rules

[0150] Considering the triangular geometric properties of the cone barrel, the three-point marking method is used when annotating the data. A key point is marked for the center of the top of the cone barrel and on both sides of the bottom. The marked points at the same position have the same category attributes, and the marking box needs to mark the whole. The pose key point annotation format provided by the yolov8 algorithm framework is used for annotation. The specific annotation format is {class, x, y, w, h, x1, y1, type, x2, y2, type, x3, y3, type}. Among them, class is the category of the object, that is, the cone barrel, represented by 0; x, y, w, h represent the coordinates and width and height of the cone barrel detection box respectively; xi, yi represent the coordinates of the key point, i = 1, 2, 3; type is the key point category, when the value is 0, it means that it is not visible, when the value is 1, it means that it is blocked and invisible, and when the value is 2, it means that it is visible.

[0151] S2.2 CPM key point extraction

[0152] Cascaded Pyramid Model (CPM) is a method for extracting key points of human posture. It can be well transferred to other key point detection tasks, such as posture estimation and feature point detection of target objects. CPM uses a series of stacked convolution modules to gradually extract multi-dimensional features of the image. Each convolution stage generates a heat map prediction of the key point. By fusing these multi-dimensional features, the accuracy of key point positioning can be improved. The formula can be expressed as:

[0153] f=F n ·F n-1 ·…·F 1 (x) (17)

[0154] Where x is the input image, F i represents the i-th convolutional block, and f is the final key point heat map prediction.

[0155] The previously predicted key point features are fused with the current input image features, so that the spatial dependency between key points can be modeled. i-1 With the current input feature F i To perform splicing and fusion:

[0156] F i =[F i ,P i-1 ] (18) This context modeling helps to enhance the localization accuracy of key points and ultimately outputs the predicted coordinates of the three key points.

[0157] S2.3 Key point connection

[0158] The key point connection is formed by connecting the two key points on the left and right of the bottom of the cone barrel to form a bottom line reference. Then, a vertical line is drawn vertically from the top of the cone barrel and intersects with the bottom line to obtain the projection expression of the cone barrel structure on the two-dimensional plane. In S2.1, the three key points of the cone barrel are marked, and the coordinates of these three key points are connected by key points to obtain the true posture. Similarly, the predicted coordinates obtained in S2.2 are connected by key points to simplify the three-dimensional cone barrel into a plane figure, providing important image information for subsequent posture estimation.

[0159] S3 Cone-barrel Attitude Estimation Based on Rotation Invariance

[0160] S3.1 Rotational invariance

[0161] Rotational invariance refers to the ability to identify rotating targets in target detection algorithms. The human body is a kinematic chain system formed by multiple bones connected by joints. Each joint has its specific movement direction and angle restrictions. These physiological constraints provide important prior knowledge for human posture estimation. The cone barrel is an integral rigid object. Its movement only includes overall translation and rotation. There is no internal deformation and joint movement. The relative positions between all points remain unchanged. Therefore, a rigid body transformation is needed to describe the rotational invariance of its posture.

[0162] Rigid body transformation refers to the change of position and direction of the cone barrel when it moves in space, including the translational freedom in three orthogonal directions (t x ,t y ,tz ) and the rotational freedom around three orthogonal axes (r x ,r y ,r z ), a total of six degrees of freedom (6-DoF). By adding a rigid_transform layer to the backbone and neck in the yolov8 framework, such as Figure 3 As shown. First, the feature map output by the backbone is preprocessed by 3×3 convolution to capture the spatial relationship of the local area. The input feature X is preprocessed using BatchNorm and ReLU to obtain the feature F = ReLU (BN (Conv (X))). Then 360° is evenly divided into N parts. For each angle θ i (i=1,2,...,N), generate the rotation matrix R(θ i )=[[cosθ i ,-sinθ i ],[sinθi,cosθ i ]], and obtain the rotated feature F through grid sampling i =G(F,R(θ i )), where G represents the grid_sample operation; finally, the N rotation features are stacked on the channel dimension [F 1 ,F 2 ,...,F n ], through the maximum pooling operation F*=max([F 1 ,F 2 ,...,F n ]) to obtain a feature representation that is rotation invariant.

[0163] During the rigid body transformation process, the cone barrel as a rigid object has the following invariant properties: shape invariance ensures that the geometric dimensions of the cone barrel (such as the triangular shape and the bottom diameter) remain constant; angle invariance ensures that the angle formed by any three points on the cone barrel surface remains constant; projection invariance is reflected in the elliptical projection of the bottom of the cone barrel on the imaging plane maintaining a fixed ratio of the major and minor axes; symmetry invariance is reflected in the fact that the rotational symmetry of the cone barrel around its central axis is always maintained. Based on these rigid constraints, the attitude estimation of the cone barrel can be carried out from the following two key aspects: the attitude representation uses the rotation matrix R∈SO(3) and the main direction vector t∈R 3 The geometric representation is constructed through the shape invariance and projection invariance of the cone barrel. These features have good invariance under rigid body transformation and provide reliable constraints for posture estimation.

[0164] S3.2 Posture Representation

[0165] In order to represent the posture in the target posture analysis, a state discrimination method based on the main direction vector is proposed. First, a reference coordinate system is constructed in the image space, and the imaging plane is mapped to the standardized space through homogeneous coordinate transformation. The horizontal reference line on the ground constitutes the reference horizontal axis, and the direction orthogonal to it is defined as the vertical axis to constitute the reference coordinate system. In terms of state representation, quaternion is used to describe the spatial posture of the target, and the relative posture transformation matrix is ​​introduced to quantify the relative relationship between the target and the reference coordinate system. For the cone barrel target, its center of mass axis is extracted to obtain the target posture vector v=(v x ,v y ), the target posture vector can represent the main direction of the object in the image space. The target posture vector v is inner-producted with the vertical reference vector e=(0,1) in the reference coordinate system to obtain:

[0166]

[0167] Where θ is the attitude deviation measure, and the judgment criterion is established based on statistics: when cosθ>cos(45°), it is judged to be a stable vertical state; when cosθ≤cos(45°), it is judged to be an unstable tilting state.

[0168] S3.2 Geometric representation

[0169] Using the triangular geometric features of the cone barrel, when the cone barrel falls, the triangle displayed in the two-dimensional image is compressed. Since the bottom of the cone barrel is circular, the diameter does not change in the two-dimensional image compression, while the cone barrel height is compressed and shortened, so the ratio of the height to the bottom side length becomes smaller. According to the standard size of the cone barrel, when the ratio of the height to the bottom side length of the cone barrel is less than 3 / 2, the cone barrel is judged to be in a fallen state.

[0170] In the state recognition of the geometric features of the cone barrel, a shape analysis method based on projection invariants and affine transformation is proposed, and the geometric invariant properties of the cone barrel projection on the imaging plane are used for state discrimination. Under affine transformation, it is necessary to pay attention to the geometric feature projection of the target contour. When the cone barrel rotates around a certain diameter of its bottom surface, its side profile appears as an asymmetric triangle on the projection plane. In particular, the bottom circle remains as an ellipse under the projection transformation, and there is a one-to-one correspondence between the length of its main axis and the original diameter. Based on the theory of projective geometry, the longest axis of the ellipse is the diameter d of the bottom circle, which constitutes an important projection invariant. Let the original height of the cone barrel be h. Under the projection transformation, the height value h' follows the following mapping relationship:

[0171] h'=h·cos(θ) (20) where θ is the angle between the target principal axis and the normal vector of the imaging plane.

[0172] Introducing the shape ratio factor As a state characteristic quantity. According to the standard configuration analysis, when the target is in a stable upright state, λ approaches its nominal value λ0. When it falls over, due to the projection compression effect, the λ value will decrease significantly, and the judgment criterion is established:

[0173] State(object)={Vertical,ifλ>3 / 2 Fallen,ifλ≤3 / 2} (21) The attitude of the cone bucket is estimated according to the above attitude representation and geometric representation methods, and the predicted attitude of the cone bucket can be obtained.

[0174] S4 posture optimization

[0175] In order to reduce the difference between the predicted posture and the true posture, an iterative optimization strategy based on gradient descent is adopted. This method realizes end-to-end optimization of posture parameters by constructing a differentiable loss function. In the posture representation space, the loss function is defined as:

[0176]

[0177] where q gt is the true posture, q is the predicted posture, R(q) is the regularization term, and α is the balance factor. This loss function consists of two parts: the first term represents the Euclidean distance measure between the predicted posture and the true posture, which reflects the accuracy of the estimation; the second term R(q) is used as a regularization constraint to suppress overfitting and enhance the generalization ability of the model. Among them, α is the weight coefficient, which is used to balance the importance of the two objectives. The optimal value of this coefficient can be determined by cross-validation. Based on the stochastic gradient descent (SGD) algorithm, the parameter update follows:

[0178]

[0179] Where η is the adaptive learning rate, t represents the iteration step, q t Represents the predicted annotation information of the tth training iteration. The loss function L(q) calculates the difference between the predicted pose and the true pose. In order to continuously reduce the difference, the SGD algorithm is used to differentiate the loss function. The parameter q is updated once in each iteration. t+1 , and dynamically adjusted through the Adam optimizer. The key here is that the choice of learning rate η is crucial: too large a learning rate may cause the optimization process to oscillate or diverge, while too small a learning rate will lead to slow convergence. Therefore, the Adam optimizer is used to dynamically adjust the learning rate. The optimizer combines the momentum method and the adaptive learning rate mechanism to automatically adjust the learning step size of the parameters. Standardization is performed after each iteration:

[0180]

[0181] In order to improve the optimization efficiency, the momentum method is used to accelerate convergence:

[0182]

[0183] q t+1 =q t -v t+1 (26)

[0184] Where μ is the momentum coefficient. The introduction of the momentum term can accelerate parameter updates in areas with similar gradient directions, and at the same time play a buffering role in areas where the gradient direction changes dramatically, which greatly improves the convergence speed and stability of the algorithm. The momentum coefficient μ is usually set in the range of [0.9, 0.99], and the optimal value is determined by grid search. In the actual optimization process, the early stopping mechanism is introduced to prevent overfitting, and the score of this posture optimization model is evaluated using the validation set. When the loss on the validation set fails to break through the historical optimal value for k consecutive epochs, it is considered that the optimization has reached a local optimal solution. At this time, the iteration process is terminated and the final posture estimation result is output.

[0185] S5 Security Warning

[0186] Through the predicted posture obtained by posture estimation in step S3 and posture optimization in step S4, the standing or falling state of the cone barrel can be judged, and a safety warning signal is generated according to the judgment result: if the cone barrel is upright, a "normal state" signal is generated; if the cone barrel falls, a "safety warning" signal is generated. The warning signal is transmitted to the remote monitoring platform through the wireless module for construction management personnel to handle in time to ensure the safety of the construction area.

[0187] Example 3

[0188] As an optimization solution of the above-mentioned embodiment, this embodiment specifically provides a method for detecting the fall of road construction cones based on posture estimation, covering the whole process from data preparation to model training and detection results, which is described in detail as follows:

[0189] (1) This example is based on the data of various types of cones used in actual road construction. A total of 4095 cone images were collected. The colors of the cones are mainly orange and white and red and white. The data is annotated with key points using the annotation format of the yolov8-pose framework. The specific annotation format is {class, x, y, w, h, x1, y1, type, x2, y2, type, x3, y3, type}. Labelme software is used for annotation. The specific annotation example is as follows: Figure 4 shown.

[0190] (2) Randomly divide the dataset, 80% (3276 images) as training dataset, 10% (409 images) as validation dataset, and the remaining 10% (410 images) for testing model performance. The test dataset is specifically used to evaluate the performance of the trained model.

[0191] (3) Configure the experimental platform under the Windows 11 system environment. The hardware uses the NVIDIA4090 graphics card (24G video memory). The software environment is based on the Python language. Install dependent libraries such as ultralytics, PIL, math, and os, as well as pytorch2.4.1 and cuda 12.1 for deep learning model training;

[0192] (4) When extracting features from the cone image input model, Gaussian filtering is first used for noise reduction, and the filter kernel size is set to 3 × 3. In the multi-scale Hu moment, the scale parameter σ is set to 1.0, 1.5, and 2.0, respectively, and 7 invariant moment features are calculated for each scale; in the adaptive Canny operator, the high threshold is set to 1.5 times the image gradient mean, and the low threshold is 0.4 times the high threshold; in color extraction, the RGB image is converted to the HSV color space, and the mean and standard deviation of the RGB and HSV channels are extracted to enhance the capture of the cone color features.

[0193] (5) After the cone position is detected through feature extraction, CPM is used to detect key points. The visualization effect of detection frame positioning and key point detection is as follows: Figure 5 As shown, it can be observed that the model has good accuracy in both detection and key point positioning of cone barrel targets.

[0194] (6) During the model training process, the early stopping strategy is used to avoid overfitting. When the performance of the validation set does not improve for 10 consecutive epochs (stop parameter k = 10), the training is terminated. The evaluation curve generated during the training process is as follows: Figure 6 As shown, the convergence characteristics and stability of the model are clearly demonstrated.

[0195] (7) In the posture estimation stage, the detected image information is combined with the posture representation and rigid constraints to build a complete cone barrel posture estimation framework. Figure 7 As shown in the figure, the model accuracy reaches more than 90% and converges stably. There is no overfitting phenomenon in the validation set curve. The model can accurately estimate the standing or falling state of the cone barrel in three-dimensional space, showing high robustness and accuracy.

[0196] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware.

[0197] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0198] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for detecting the fall of road construction cones based on posture estimation, characterized in that: The following steps are involved: S1. Collect cone barrel images; S2. extracting multi-dimensional features of the cone barrel image, wherein the multi-dimensional features include contour features, edge features and color features, and compensating the extraction effect of the multi-dimensional features by adaptive brightness; S3. Perform key point detection and key point connection on the cone barrel image according to the multi-dimensional features to obtain image information for posture estimation; S4. Estimating the posture of the image information to obtain a predicted posture; S5. Generate a safety warning signal based on the predicted posture.

2. According to the method for detecting the fall of road construction cones based on posture estimation according to claim 1, it is characterized in that: Extracting contour features in step S2 includes the following steps: S211. grayscale and denoise the cone barrel image; S212. Construct a scale pyramid of downsampling at different multiples, and obtain a multi-scale spatial representation through Gaussian pyramid decomposition; S213. Extract the contour features of the cone barrel in the cone barrel image in each scale space and calculate the Hu moment features.

3. The method for detecting the fall of road construction cones based on posture estimation according to claim 1, characterized in that: The edge feature extraction of the cone barrel in step S2 adopts the Canny operator algorithm, which includes the following steps: S221. Obtaining the initial edge by calculating the cone image gradient amplitude and direction; S222. Processing the gradient amplitude and the direction of the cone barrel image, and refining the initial edge by non-maximum suppression; S223. Calculate the high threshold and low threshold of the refined edge based on the local variance of the content of the cone barrel image; S224. Connect the real edge points through edge tracking and hysteresis threshold, remove the pseudo edges and noise, and obtain edge features.

4. The method for detecting the fall of road construction cones based on posture estimation according to claim 2 is characterized in that: The extraction of the color features in step S2 includes the following steps: S231. Convert the cone barrel image from the RGB color space to the HSV color space, and extract the characteristic information of hue H, saturation S and brightness V; S232. Setting a threshold range of the HSV color space for identifying the color region of the cone barrel image; S233. Calculate the color distance of the pixel by combining the color reference value in the RGB color space, and filter the cone color area that meets the conditions; S234. Compare the threshold range and the color distance, optimize the screening result, and extract the color features of the cone barrel image.

5. A method for detecting the fall of road construction cones based on posture estimation according to any one of claims 1 to 4, characterized in that: The method comprises adopting a regional adaptive brightness compensation method to extract the features of the cone barrel image, comprising the following steps: S241. Perform local brightness statistics and gradient distribution analysis on the Hu moment feature to extract contour features at different brightnesses; S242. A dynamic threshold adjustment mechanism is introduced to the edge feature, edge detection parameters are adaptively adjusted according to local contrast, and pseudo edges caused by brightness changes are constrained by edge continuity to extract edge features at different brightnesses; S243. Adopting a region segmentation adaptive brightness mapping method for the color features and combining color constraints, extracting color features under different brightness.

6. The method for detecting the fall of road construction cones based on posture estimation according to claim 5 is characterized in that: In step S3, the key point detection model is used to identify the image information of the cone barrel image, including the following steps: S31. Using a three-point marking method to mark key points of the cone barrel image to obtain annotation data; S32. Input the labeled data into a key point detection model based on the Cascaded Pyramid Model, perform key point detection, and output the predicted coordinates of the key points; S33. According to the predicted coordinates, key points are connected to convert the predicted coordinates into image information for posture estimation.

7. The method for detecting the fall of road construction cones based on posture estimation according to claim 6, characterized in that: Step S4 uses a main direction vector attitude representation method to estimate the attitude of the image information, including the following steps: S411. According to the geometric characteristics of the cone barrel, a rotation matrix and a main direction vector are constructed to describe the state of the cone barrel in three-dimensional space; S412. Input the multi-dimensional features into a deep learning model based on the rigid_transform layer, and generate rotation features through grid sampling; S413. stacking a plurality of the rotation features and performing a maximum pooling operation to extract the cone barrel posture features with rotation invariance, and generating a cone barrel posture representation; S414. According to the state discrimination method based on the main direction vector, target posture analysis is performed on the posture representation, posture estimation is performed on the cone barrel, and a predicted posture is obtained.

8. The method for detecting the fall of road construction cones based on posture estimation according to claim 6, characterized in that: Step S4 uses a projection invariant and affine transformation geometric representation method to estimate the posture of the image information to obtain a predicted posture; wherein, when the ratio of the height of the cone barrel projection to the bottom length is less than a preset threshold, the predicted posture is a falling state.

9. A method for detecting the fall of road construction cones based on posture estimation according to any one of claims 7 to 8, characterized in that: Step S3 also includes marking key points of the cone barrel image to obtain the true posture; according to the difference between the predicted posture and the true posture, step S4 also includes optimizing the posture estimation, including the following steps: S421. Constructing a loss function according to the predicted posture and the true posture; S422. Using the loss function as the input of the stochastic gradient descent algorithm, dynamically adjusting the learning rate in combination with the Adam optimizer, and outputting the optimized posture estimation result; S423. Use the early stopping mechanism and the validation set to evaluate the optimized pose estimation results. When the validation set loss reaches a preset value of epochs, terminate the iteration.

10. The method for detecting the fall of road construction cones based on posture estimation according to claim 9, characterized in that: In step S421, the loss function includes 1) a Euclidean distance metric representing the predicted posture and the true posture and 2) a regularization constraint on the predicted posture.

Images