Human shape related data visualization method and system based on artificial intelligence

Abstract

The invention relates to the technical field of human shape visual identification, and solves the problems that joint proportion imbalance and spatial position distortion are easily caused by depth information loss and projection ambiguity in the processing of two-dimensional image coordinates in the prior art, and a single-frame estimation method is sensitive to time sequence jitter and shielding and is not easy to carry out. In order to solve the technical problems of unstable output and lack of joint points due to the fact that the human shape related data is not stable, the invention particularly relates to a human shape related data visualization method and system based on artificial intelligence, and the method comprises the following steps: S1, obtaining video data, and preprocessing the video data to obtain standardized boundary data. According to the method, the joint initial positioning reasonability is improved, inverse kinematics is combined with Kalman filtering to fuse time sequence information, a motion track is remarkably smoothed, jitter is inhibited, global context completion is performed on shielding by means of a self-attention mechanism, the integrity of an attitude sequence is ensured, and the recognition result is more accurate and efficient.

Description

Technical Field

[0001] This invention relates to the field of human visual recognition technology, and in particular to a method and system for visualizing human-related data based on artificial intelligence. Background Technology

[0002] Human-related data visualization refers to transforming human posture, movement, and behavior data into intuitive graphical representations by combining geometric modeling and semantic encoding. On the one hand, it uses skeletal wireframes, silhouettes, or voxel models to present the basic structure of the human body and maps physical attributes such as joint angles and movement speed through visual variables such as color, size, and transparency. On the other hand, based on behavior recognition results, it uses icons and dynamic effects to give the data semantics. However, existing technologies are prone to joint proportion imbalance and spatial position distortion in the process of recognizing and visualizing human-related data due to the lack of depth information and projection ambiguity. Furthermore, single-frame estimation methods are sensitive to temporal jitter and occlusion, resulting in unstable output and the problem of missing joint points. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a human-related data visualization method and system based on artificial intelligence. It solves the technical problems of existing technologies in processing secondary image coordinates, which are prone to joint proportion imbalance and spatial position distortion due to missing depth information and projection ambiguity. Furthermore, the single-frame estimation method is sensitive to temporal jitter and occlusion, resulting in unstable output and missing joint points.

[0004] To solve the above-mentioned technical problems, the present invention provides the following technical solution: The method includes the following steps: S1. Acquire video data and preprocess the video data to obtain standardized boundary data; S2. Based on standardized boundary data, a set of two-dimensional key point coordinates of the human body is obtained through high-dimensional probabilistic key point processing; S3. Based on the coordinate set of the two-dimensional key points of the human body, graph convolution processing, inverse kinematics Kalman filtering, and self-attention mechanism completion are performed sequentially to obtain world pose data; S4. Analyze the health status of humanoid figures based on global posture data and obtain humanoid status data; S5. Obtain BIM model data and perform visualization preprocessing based on world posture data and human form status data to obtain a three-dimensional situation map; S6. Generate an interactive security monitoring visualization interface based on the 3D situation map through rendering and graphical interaction processing.

[0005] Preferably, in S1, the specific implementation steps are as follows: S11. Obtain the video frame sequence based on the video data from the construction site using hardware decoding technology; S12. Perform target detection based on the video frame sequence to obtain a set of human body boundary points; S13. Obtain standardized boundary data by standardizing the set of human body boundary points.

[0006] Preferably, in S2, the specific implementation steps are as follows: S21. Obtain the high-dimensional posture feature vector by capturing posture features based on standardized boundary data; S22. Based on the high-dimensional feature vector of the pose, the transposed convolution operation is used to obtain the transposed convolution value. Based on the transposed convolution value, the convolution operation is performed again to obtain the high-precision value. Then, the sigmoid function is used to map the high-precision value to a probability heat map. S23. Based on the probability heat map set, perform two-dimensional key point recognition of the human body to obtain the coordinate set of two-dimensional key points of the human body.

[0007] Preferably, in S3, the specific implementation steps are as follows: S31. Divide the two-dimensional key point coordinate set of the human body into a limb coordinate set and a trunk coordinate set based on the position of the human limbs and trunk. Calculate the geometric center formula of the limb coordinate set to obtain the limb bone nodes, and obtain the trunk center node according to the geometric center formula of the trunk coordinate set. S32. Based on the coordinate set of two-dimensional key points of the human body, the limb bone nodes and the trunk center node, the initial three-dimensional pose parameters are obtained by regression through graph convolution. S33. Optimized attitude parameters are obtained by inverse kinematics processing based on the initial three-dimensional attitude parameters; S34. Filter and complete the optimized attitude parameters to obtain complete 3D attitude data, and perform coordinate transformation based on the complete 3D attitude data to obtain world attitude data.

[0008] Preferably, in S34, the specific implementation steps are as follows: S341. Based on the optimized attitude parameters, smooth three-dimensional attitude data is obtained by processing them through a Kalman filter; S342. Based on the smooth 3D pose data, the occlusion data is completed using a self-attention mechanism model to obtain complete 3D pose data; S343. Obtain world attitude data by transforming the complete three-dimensional attitude data into world coordinates.

[0009] Preferably, in S4, the specific implementation steps are as follows: S41. Obtain behavioral feature embedding vectors by spatiotemporal graph convolution based on world pose data; S42. Classify behaviors using a multilayer perceptron classifier based on the behavior feature embedding vector, and obtain the initial behavior category probability distribution; S43. Obtain enhanced behavior data by processing the initial behavior category probability distribution using a conditional random field. S44. A dynamic risk index is obtained by performing risk assessment through a deep reinforcement learning decision-maker based on enhanced behavioral data. S45. Calculate the upper and lower thresholds based on the dynamic risk index, judge the humanoid behavior based on the upper and lower thresholds, and obtain humanoid status data. If the dynamic risk index is between the upper and lower thresholds, then the current humanoid behavior is humanoid health data; otherwise, the current humanoid behavior is humanoid risk data.

[0010] Preferably, in S5, the specific implementation steps are as follows: S51. Obtain BIM model data from the BIM database, and process the BIM model data through the BIM model lightweight parsing engine to obtain lightweight 3D scene mesh and semantic information; S52. Based on the world attitude data and the lightweight 3D scene mesh, the scene attitude dataset is obtained through 3D scene transformation mapping. S53. Based on the human form state data and scene pose dataset, a risk density distribution map is obtained by processing the kernel density estimation method. S54. Multi-layer scene data is obtained by processing the lightweight 3D scene mesh, scene pose dataset and risk density distribution map using dynamic rendering technology. S55. A three-dimensional situation map is obtained by visual encoding processing based on multi-layer scene data.

[0011] Preferably, in S6, the specific implementation steps are as follows: S61. Generate a multi-view coordinated visualization scene based on the 3D situation map; S62. Based on the multi-view coordinated visualization scene, the pixel frame buffer is obtained by WebGL rendering. S63. The user inputs their intention information, and an interactive security monitoring visualization interface is generated through graphical interaction processing based on the pixel frame buffer and the intention information.

[0012] The technical solution also provides a system for the aforementioned AI-based humanoid data visualization method, the system comprising: The preprocessing module is used to acquire video data and preprocess the video data to obtain standardized boundary data; The key point module is used to obtain a set of two-dimensional key point coordinates of the human body through high-dimensional probabilistic key point processing based on standardized boundary data; The pose module is used to obtain world pose data by sequentially performing graph convolution processing, inverse kinematics Kalman filtering, and self-attention mechanism completion based on the coordinate set of two-dimensional human body key points. The humanoid health module is used to analyze the health status of humanoids based on world posture data and obtain humanoid status data. The 3D situation module is used to acquire BIM model data and perform visualization preprocessing based on world posture data and human figure status data to obtain a 3D situation map. The visualization module is used to generate an interactive security monitoring visualization interface based on the 3D situation map through rendering and graphical interaction processing.

[0013] By employing the above technical solution, the present invention provides a method and system for visualizing humanoid-related data based on artificial intelligence, which has at least the following beneficial effects: 1. This invention constructs a differentiable path from high-dimensional features to dense space by introducing transposed convolution and quadratic convolution operations, and generates a normalized probability heatmap, enabling the entire process to support end-to-end training. This effectively improves the model's ability to capture subtle changes in pose and the spatial positioning accuracy of key points, and significantly improves the robustness and accuracy of human pose estimation in complex scenarios.

[0014] 2. This invention provides a strong structural prior for graph convolutional networks by calculating the geometric center of joints, which improves the rationality of initial joint localization. Furthermore, by using inverse kinematics combined with Kalman filtering to fuse temporal information, the motion trajectory is significantly smoothed and jitter is suppressed. At the same time, the self-attention mechanism is used to perform global context completion on occluded or missing frames, ensuring the integrity of the pose sequence and making human recognition results more accurate and efficient.

[0015] 3. By combining spatiotemporal graph convolution and conditional random fields, this invention effectively captures the spatiotemporal dependencies of behavior and generates a dynamic risk index, thereby achieving enhanced understanding of the behavioral context and adaptive threshold determination. This significantly improves the accuracy of human behavior recognition in complex scenarios and the real-time performance and robustness of risk assessment.

[0016] 4. This invention utilizes a lightweight engine to transform BIM models into real-time rendered grids and semantic maps, ensuring that the virtual scene is both realistic and interactive. It integrates multi-layer information such as scene grids, human body positions, and risk density to generate a unified three-dimensional situation map, enabling seamless integration of abstract behavioral data with specific spatial environments and risk distributions. This results in more diverse and comparative display screens, providing users with a more comprehensive and dynamic user experience. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1This is a flowchart of the humanoid-related data visualization method based on artificial intelligence according to the present invention; Figure 2 This is a structural block diagram of the artificial intelligence-based humanoid data visualization system of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects, and to facilitate its implementation.

[0019] Example 1 Existing techniques for processing quadratic image coordinates are prone to joint scaling and spatial distortion due to missing depth information and projection ambiguity. Furthermore, single-frame estimation methods are highly sensitive to temporal jitter and occlusion, leading to unstable output and missing joint points. Please refer to [reference needed]. Figure 1 This embodiment provides an artificial intelligence-based method for visualizing human-related data, which can prevent joint proportion discrepancies and spatial position distortions, and can also continuously shake to make the data more complete. The method includes the following steps: S1. Acquire video data and preprocess it to obtain standardized boundary data. Existing technologies for target detection in video frame sequences mostly rely directly on the absolute coordinates or pixel positions of the original human bounding boxes for subsequent analysis. This makes the detection results susceptible to the influence of shooting angle, target-camera distance, and individual pose differences, leading to scale sensitivity and spatial layout deviations. This results in inconsistent recognition of the same behavior in different scenarios. To address this issue, the specific implementation steps are as follows: S11. Obtain the video frame sequence from the video data at the construction site using hardware decoding technology. The video data can be collected through monitoring, and the hardware decoding technology can use H.265 / HEVC hardware decoding technology to decode the video stream and obtain the decoded video frame sequence.

[0020] S12. Target detection is performed based on the video frame sequence to obtain a set of human body boundary points. Target detection in complex scenes can be performed using an improved YOLOv9 model. The improved YOLOv9 model uses the EMA (Efficient Multi-Scale Attention) mechanism for feature extraction to obtain fused data, and optimizes the model training based on the Focaler-IoU loss function to improve the accuracy and training efficiency of human body bounding box recognition. The EMA mechanism involves grouping the video frame sequence to obtain grouped data, and then performing 1D global average pooling and convolution operations on the grouped data to obtain multi-scale processed data. The multi-scale data is then processed... The data is processed using 2D global average pooling to encode global spatial information. An attention map is generated through matrix dot product, and then the features of multiple attention maps are aggregated to obtain fused data. The Focaler-IoU loss function reconstructs the IoU loss through a linear interval mapping method, enabling the loss function to dynamically adjust the focus based on the difficulty distribution of samples in the detection task: in tasks where difficult samples, such as small targets, are the main focus, the model pays more attention to these difficult samples; when simple samples are the main focus, the model pays more attention to simple samples. Since the Focaler-IoU loss function and the EMA efficient multi-scale attention mechanism are commonly used methods, they will not be elaborated here.

[0021] S13. Standardized boundary data is obtained by standardizing the human body boundary point set. Before standardization, image data of uniform size needs to be obtained by bilinear interpolation scaling. Then, enhanced image with contrast enhancement is obtained by adaptive histogram equalization. Standardization is performed on the enhanced image. The standardization process requires first calculating the boundary mean and boundary standard deviation of the enhanced image. The standardized boundary data is obtained by subtracting the boundary mean from the boundary data in the human body boundary point set and then dividing by the boundary standard deviation. This invention transforms the original human body boundary point set into standardized boundary data that is independent of scale and position by introducing standardization processing. This effectively eliminates the interference of environmental variation, improves the robustness and cross-scene adaptability of human body representation, and provides a more stable and unified data foundation for subsequent behavior recognition or trajectory analysis.

[0022] S2. Based on standardized boundary data, a set of 2D human body keypoint coordinates is obtained through high-dimensional probabilistic keypoint processing. In existing technologies, 2D human body keypoint detection usually relies on direct regression coordinates or localization based on the maximum points of ordinary heatmaps. These methods suffer from problems such as unstable gradient training and sensitivity to regression target offset, which easily leads to low sub-pixel accuracy of keypoint localization. To solve the above problems, the specific implementation steps are as follows: S21. Obtain the high-dimensional pose feature vector by capturing pose features based on the standardized boundary data; calculate the high-dimensional pose feature vector using the HRNet formula based on the standardized boundary data. The HRNet formula is a method that maintains high-resolution feature representation throughout the process based on a high-resolution network. It is a common method for obtaining high-dimensional features, which will not be elaborated here.

[0023] S22. Based on the high-dimensional feature vector of the pose, transpose convolution is performed to obtain the transposed convolution value. Based on the transposed convolution value, a convolution operation is performed again to obtain the high-precision value. Then, the sigmoid function is used to map the high-precision value to a probability heatmap. The probability heatmap should be understood as the probability heatmap of each frame of the image, which is the name of a set containing multiple probability heatmaps. The transpose convolution operation is to process the high-dimensional feature vector of the pose using the TransposeConv formula. Convolution and transpose convolution are commonly used image spatial precision processing methods. The sigmoid function is a very basic and important activation function in machine learning and deep learning, which will not be elaborated here.

[0024] S23. Human 2D keypoint coordinate set is obtained by identifying human 2D keypoints based on the probability heatmap set. A temperature coefficient is preset to control the sharpness of the output probability distribution. The probability heatmap is multiplied by the temperature coefficient, and an exponential operation is performed to obtain the exponential value. All exponential values ​​are then summed to obtain a normalized denominator. The exponential value of each probability heatmap is then divided by the normalized denominator to obtain the normalized distribution value. Finally, each pixel in each probability heatmap is multiplied by the normalized distribution value and weighted to obtain the human 2D keypoint coordinates. The weight coefficients can be obtained using the analytic hierarchy process (AHP). This invention constructs a differentiable path from high-dimensional features to dense space mapping by introducing transposed convolution and quadratic convolution operations. It combines this with the Sigmoid function to generate a normalized probability heatmap. Finally, SA sub-pixel correction technology is used to calculate the keypoint coordinates, enabling end-to-end training throughout the process. This effectively improves the model's ability to capture subtle changes in posture and the spatial positioning accuracy of keypoints, significantly improving the robustness and accuracy of human posture estimation in complex scenes.

[0025] S3. Based on the coordinate set of the two-dimensional human body key points, graph convolution, inverse kinematics Kalman filtering, and self-attention mechanism are sequentially performed to complete the world pose data. Existing technologies are prone to joint proportion misalignment and spatial position distortion in processing secondary image coordinates due to missing depth information and projection ambiguity. Single-frame estimation methods are sensitive to temporal jitter and occlusion, resulting in unstable output and missing key points. To solve the above problems, the specific solution steps are as follows: S31. Divide the human body's two-dimensional keypoint coordinate set into a limb coordinate set and a trunk coordinate set based on the positions of the limbs and torso. Calculate the geometric center formula of the limb coordinate set to obtain the limb bone nodes, and obtain the trunk center node according to the geometric center formula of the trunk coordinate set. The human body's two-dimensional keypoint coordinate set can be roughly divided into the positions of the limbs, torso, and head. Based on their positional characteristics, the nodes in the limb coordinate set that are always close to the torso are the nodes connecting the upper arm and shoulder or the thigh and torso. The other end must be the hands or feet. Therefore, the key is to find the elbow or knee as the center node. Since the elbow and knee are approximately located at the center of the limbs, the approximate joint nodes can be obtained by calculating the geometric center of the limb coordinate set and the trunk coordinate set. The geometric center formula is a commonly used formula in geometry and will not be elaborated here.

[0026] S32. Initial 3D pose parameters are obtained by regression through graph convolution based on the 2D keypoint coordinate set of the human body, limb bone nodes, and trunk center nodes. Based on the 2D keypoint coordinate set of the human body, limb bone nodes, and trunk center nodes, prior knowledge is obtained through human anatomy. Pairs of joints with physical connections, such as shoulder-elbow and elbow-hand, are connected by edges to form an undirected graph with joints as nodes and bone connections as edges. This graph is then represented in data form using an adjacency matrix, and the 3D pose parameters are obtained by graph convolution regression. The 3D pose parameters can be inferred through GCN, which stands for graph convolution operation. GCN is a neural network specifically designed to process graph structure data, which will not be elaborated here.

[0027] S33. Optimized attitude parameters are obtained by performing inverse kinematics processing based on the initial 3D attitude parameters. First, a camera model N is constructed. Camera model N is a mathematical function that describes how a 3D point in the real world is projected onto the 2D camera imaging plane. For example, it receives the coordinates (X, Y, Z) of a 3D point in the world coordinate system as input. By simulating the camera's internal parameters, such as focal length and optical center, and external parameters, such as the camera's position and orientation in space, the pixel coordinates (u, v) of this 3D point on the 2D image are calculated. The 3D joints are then projected onto the 2D image plane using camera model N. The objective function is used to calculate the difference between the predicted human skeleton length and the known two-dimensional keypoints, which are the coordinate sets of the human body's two-dimensional keypoints. Then, the deviation between the predicted human skeleton length and the nominal skeleton length based on prior human anatomy is calculated, and the lengths of all skeletons are summed. Finally, by minimizing this objective function, under the dual constraints of strictly matching two-dimensional observation evidence and following reasonable human physiological structure, the three-dimensional pose parameters are iteratively adjusted to output an optimized pose parameter that is visually accurate and physiologically reasonable. The predicted human skeleton length is obtained by connecting the three-dimensional pose parameters through the arm nodes, which will not be elaborated here.

[0028] S34. Filter and complete the optimized attitude parameters to obtain complete 3D attitude data, and perform coordinate transformation based on the complete 3D attitude data to obtain world attitude data; the specific implementation steps are as follows: S341. Based on the optimized attitude parameters, smooth 3D attitude data is obtained by processing with a Kalman filter. The Kalman filter can optimally estimate the current state based on the motion model and effectively filter out noise. This step requires collecting historical data of the video data. The historical data and the optimized attitude parameters at the current moment are used as input, and the Kalman filter is used for temporal smoothing. The calculation formula of the Kalman filter is roughly divided into two parts. First, the predicted state parameters are obtained by multiplying the attitude parameters of the previous moment in the historical data by the state transition matrix and adding the process noise. A Kalman gain is obtained by predicting the covariance. The Kalman product value is obtained by multiplying the Kalman gain by the difference between the actual video data and the expected data. The predicted state parameters are then added to obtain the smooth 3D attitude data. Covariance prediction is a commonly used method to obtain the Kalman gain, which will not be elaborated here.

[0029] S342. Based on the smoothed 3D pose data, the occlusion data is completed using a self-attention mechanism model to obtain complete 3D pose data. The self-attention mechanism model is a mechanism in deep learning that captures the correlation between elements within sequential data. First, each element of the smoothed 3D pose data is mapped to a query vector, key vector, and value vector through a linear transformation. Then, the similarity between the query vector of each element and the key vectors of all other elements is calculated, such as by scaling after dot product or using other similarity functions, to obtain an attention score. Then, these scores are normalized, for example by using the softmax function, to obtain attention weights, which reflect the tightness of the correlation between elements. Finally, the value vectors of all elements are weighted and summed according to the attention weights to obtain a new representation of each element. The new representation integrates the information of other elements in the sequence, that is, it can complete the image missing due to occlusion to obtain complete 3D pose data.

[0030] S343. World pose data is obtained by transforming the complete 3D pose data into world coordinates. World pose data in the world coordinate system is obtained by multiplying the complete 3D pose data by the rotation matrix and adding the translation vector. The rotation matrix and translation vector are calculated by camera parameters during camera calibration. Camera calibration is a commonly used method for obtaining rotation matrix and translation vector, which will not be elaborated here. This invention provides strong structure priors for graph convolutional networks by dividing joints into torso and limbs and calculating their geometric centers, which improves the rationality of initial joint positioning. The coordinate regression problem is transformed into an optimization problem under the constraints of bone length and angle by using inverse kinematics processing, which ensures the physiological feasibility of pose in human figure recognition. On the other hand, by fusing temporal information through Kalman filtering, the motion trajectory is significantly smoothed and jitter is suppressed. Furthermore, the self-attention mechanism is used to perform global context completion on occluded or abnormal frames, which ensures the integrity of the pose sequence. This allows the human figure recognition results to generate more accurate and efficient 3D pose data in the world coordinate system in terms of naturalness.

[0031] S4. Analyze the humanoid health status based on world posture data and obtain humanoid status data; existing technologies in world posture data analysis usually rely on static posture data for behavior classification, lacking in-depth mining of behavioral contextual relationships, and risk assessment is mostly based on fixed thresholds or single classification results, resulting in insufficient adaptability to complex and continuous behavioral changes, and easy to misjudgment and omission. In order to solve this problem, the specific implementation steps are as follows: S41. Based on world pose data, behavioral feature embedding vectors are obtained through spatiotemporal graph convolution processing. Spatiotemporal graph convolution processing treats each joint point in the world pose data as a node in the graph, and the connections between joints form the edges of the graph. Spatiotemporal graph convolution processing performs convolution operations in both spatial and temporal dimensions. In the spatial dimension, it considers the local connections between joints to capture the spatial structural information of human pose. In the temporal dimension, it convolves the joint position information at different times in the sequence to capture the dynamic information of human pose changes over time. This spatiotemporal joint convolution processing can effectively extract discriminative behavioral features from the input world pose data, improving the accuracy of human action pose recognition.

[0032] S42. Based on the behavior feature embedding vector, a multilayer perceptron classifier is used to classify behaviors and obtain the initial behavior category probability distribution. The multilayer perceptron classifier is a feedforward artificial neural network model, consisting of an input layer, one or more hidden layers, and an output layer. The layers are connected in a fully connected manner, that is, each neuron in the previous layer is connected to all neurons in the next layer. Each neuron contains a linear weighted summation operation and a non-linear activation function. The specific steps are as follows: first, a linear transformation of the behavior feature embedding vector is performed to obtain a linear value, and then the linear value is processed by a non-linear activation function to obtain non-linear features. The non-linear activation function can be the ReLU function. The learnable parameters are optimized by combining the backpropagation algorithm with the cross-entropy loss function. The backpropagation algorithm and the cross-entropy loss function are commonly used methods for optimizing model parameters, which will not be elaborated here.

[0033] S43. Enhanced behavior data is obtained by processing the initial behavior category probability distribution using a conditional random field. First, the sum of univariate potential energy is obtained based on the initial behavior category probability distribution. For example, if an initial behavior category is selected, the score for walking is 0.8, and the score for running is 0.2. Therefore, the univariate potential energy for walking would be 0.8. The univariate potential energy of all frames is summed to obtain the total univariate potential energy. Then, a comprehensive judgment is made based on the categories of adjacent frames of the selected initial behavior category. For example, if the transition from walking to running is reasonable, a high positive value is assigned. If the transition from sitting to running is physically disjointed and therefore penalized, a low negative value is assigned. Finally, the paired potential energy of a pair of adjacent frames is obtained. The weighted sum of all paired potential energy is obtained to obtain the total paired potential energy. The weighted sum of the univariate potential energy and the total paired potential energy is then summed to maximize the result, yielding the enhanced behavior data. The weight coefficients can be obtained using the analytic hierarchy process (AHP).

[0034] S44. A dynamic risk index is obtained by risk assessment using a deep reinforcement learning decision-maker based on enhanced behavioral data. The deep reinforcement learning decision-maker is a core component of an intelligent agent that integrates the powerful perception capabilities of deep learning with the intelligent decision-making capabilities of reinforcement learning. It utilizes deep neural networks to process complex, high-dimensional input data, such as images and sensor signals. Enhanced behavioral data is input into the deep reinforcement learning decision-maker, which combines multidimensional information such as behavior type, duration, location, and other factors, including illumination, temperature, humidity, or historical behavior. The deep reinforcement learning decision-maker in this invention can consist of an input embedding layer, a temporal feature layer, a fully connected layer, and a decision output layer. The input embedding layer normalizes classification information, the temporal feature layer extracts temporal information using LSTM, the fully connected layer learns more abstract feature representations through nonlinear activation functions such as ReLU and GeLU, and the decision output layer provides a dynamic risk index based on the actor critic method. The actor critic method integrates value function-based and policy-based methods and is a commonly used and important hybrid algorithm in the field of reinforcement learning, which will not be elaborated upon here.

[0035] S45. Calculate the upper and lower thresholds based on the dynamic risk index, judge the humanoid behavior based on the upper and lower thresholds, and obtain humanoid status data. If the dynamic risk index is between the upper and lower thresholds, the current humanoid behavior is considered humanoid health data; otherwise, it is considered humanoid risk data. The mean, standard deviation, and coefficient of determination are calculated based on the dynamic risk index. The upper threshold is obtained by multiplying the standard deviation by the coefficient of determination and adding the mean. The lower threshold is obtained by multiplying the standard deviation by the coefficient of determination and subtracting the mean. This invention effectively captures the spatiotemporal dependencies of behavior by combining spatiotemporal graph convolution and conditional random fields. It also utilizes a deep reinforcement learning decision-maker to generate the dynamic risk index, achieving enhanced understanding of the behavioral context and adaptive threshold determination. This significantly improves the accuracy of humanoid behavior recognition in complex scenarios and enhances the real-time performance and robustness of risk assessment.

[0036] S5. Acquire BIM model data and perform visualization preprocessing based on world posture data and human form status data to obtain a 3D situation map. Existing technologies often end by outputting human posture in the world coordinate system, or simply overlay it onto a 2D planar map, resulting in a lack of real-world semantic support for positioning and difficulty in intuitively mapping risk analysis to specific spatial locations. To solve the above problems, the specific solution steps are as follows: S51. Obtain BIM model data from the BIM database. Process the BIM model data using the lightweight BIM model parsing engine to obtain a lightweight 3D scene mesh and semantic information. In actual operation, the system will have a pre-set BIM database. The BIM database is a data storage and management platform specifically designed for building information models. Based on the 3D model, it integrates various types of information throughout the entire life cycle of a building project. In this step, it can be used to build a static 3D environmental model. The lightweight BIM model parsing engine is a technical tool that compresses, optimizes, and transforms BIM model data so that it can be efficiently loaded, rendered, and interacted with on ordinary hardware devices, such as ordinary computers, mobile phones, tablets, and web pages, without the need for professional modeling software, while maintaining the key information and accuracy of the model.

[0037] S52. Based on the world pose data and the lightweight 3D scene mesh, a scene pose dataset is obtained through 3D scene transformation mapping. Based on the lightweight 3D scene mesh, a transformation matrix can be obtained through ORB processing. The scene pose data is obtained by multiplying the transformation matrix by the world pose data. All world pose data are processed through 3D scene transformation mapping to obtain the scene pose dataset. ORB processing is a classic computer vision feature detection and description algorithm. It can quickly extract key points from images and generate corresponding feature descriptors. It is a common machine vision analysis technique and will not be elaborated here.

[0038] S53. Based on the humanoid state data and scene pose dataset, a risk density distribution map is obtained through kernel density estimation. The humanoid state data includes the state results from the previous step and humanoid node action data, such as node speed and action. Kernel density estimation is a non-parametric statistical method used to estimate the probability density function of random variables. This invention cleans and aligns the humanoid state data and scene pose dataset, defines quantitative risk indicators, extracts multi-dimensional feature vectors, and then uses kernel density estimation. This invention selects a suitable kernel function and optimizes the bandwidth to complete the calculation of the risk density distribution in the feature space, and finally generates a risk density distribution map. The kernel function in kernel density estimation can be a Gaussian kernel, and the bandwidth can be optimized using cross-validation. Gaussian kernel and cross-validation are common kernel function and bandwidth optimization methods, which will not be elaborated here.

[0039] S54. Based on the lightweight 3D scene mesh, scene pose dataset, and risk density distribution map, multi-layer scene data is obtained through dynamic rendering technology. In dynamic rendering technology, data from different sources, such as lightweight 3D scenes, scene pose data, and risk density distribution maps, are mixed and overlaid in different rendering channels using shaders to obtain multi-layer scene data. Dynamic rendering technology is based on real-time data streams and dynamically adjusts the visual presentation content and effects through the graphics rendering pipeline. It can input multi-source heterogeneous data, such as lightweight 3D scenes, scene pose data, and risk density distribution maps, into the rendering engine in the form of data streams. Shaders or independent rendering channels are used to perform spatial alignment, transparency blending, color mapping, and other operations on different data layers. Finally, in a unified scene, the real-time fusion and visualization of geometric structures, dynamic objects, and semantic information are achieved. Semantic information, such as risk levels, is visualized to lay the groundwork for subsequent steps.

[0040] S55. A 3D situation map is obtained through visualization encoding processing based on multi-layer scene data. Visual encoding processing is a method of mapping abstract data into intuitive visual elements using graphic symbols. Specifically, based on multi-layer scene data, visual variables such as shape, color, size, and texture are designed. Behavioral types in the multi-layer scene data, such as using red skeletal lines to encode climbing actions and risk levels, the size of the halo radius corresponding to risk values, spatial relationships, and the direction of arrows indicating movement trends, are transformed into perceptible graphics. Ultimately, a well-defined and semantically clear situation map is generated in the 3D scene, achieving efficient transfer from data to cognition. This invention utilizes a lightweight engine to transform the BIM model into a real-time renderable mesh and semantic map, ensuring that the virtual scene is both realistic and interactive. Simultaneously, relying on dynamic rendering and visualization encoding technology, multi-layer information such as scene mesh, human body position, and risk density are integrated to generate a unified 3D situation map. This allows for seamless integration of abstract behavioral data with the specific spatial environment and risk distribution, ultimately outputting an intuitive and visually appealing image that reflects human dynamics in real time, accurately locates risk areas, and supports 3D spatial analysis and decision-making.

[0041] S6. Generate an interactive safety monitoring visualization interface based on the 3D situation map through rendering and graphical interaction processing; the existing technology's 3D scene, 2D plan view, and risk heat map are disconnected in terms of state, and operations cannot be linked, resulting in an increased cognitive load on users. To solve the above problems, the specific implementation steps are as follows: S61. Generate a multi-view coordinated visualization scene based on the 3D situation map; The 3D situation map uses viewport management and layout algorithms to divide the screen into multiple linked views such as a 3D panoramic main view, a 2D planar top view, a personal perspective detail view, and an alarm list, providing users with comprehensive situation information and generating a multi-view coordinated visualization scene.

[0042] S62. Based on the multi-view coordinated visualization scene, the pixel frame buffer is obtained by WebGL rendering. Based on the multi-view coordinated visualization scene, the basic rendering screen is processed through shader programming and post-processing channels, such as blooming and ambient occlusion, to generate the final screen to enhance visual expressiveness and highlight key information.

[0043] S63. The user inputs their intention information, and an interactive security monitoring visualization interface is generated through graphical interaction processing based on the pixel frame buffer and the intention information. By listening to and processing the user's mouse clicks, drags, selections, and swipes, the system enables rich interactive functions such as clicking to query personnel information, selecting multiple people for tracking, timeline rewinding, and filtering specific risk events, forming a complete monitoring-analysis-decision-making closed loop. This invention uses multi-view coordination technology to ensure the linked updates of the 3D main view, floor plan, and data panel, providing users with a unified spatial and data understanding. It employs WebGL for browser-side hardware acceleration rendering, directly generating pixel frame buffers, achieving cross-platform, plugin-free real-time rendering of high-definition 3D scenes. Through graphical interaction processing, user intention information, such as clicks, selections, and viewpoint switching, is deeply integrated with the rendered screen, dynamically generating an interactive security monitoring interface that can be freely explored and detailed, significantly improving the usability and decision-making efficiency of the monitoring system.

[0044] Example 2 Existing techniques for processing quadratic image coordinates are prone to joint scaling misalignment and spatial distortion due to missing depth information and projection ambiguity. Furthermore, single-frame estimation methods are highly sensitive to temporal jitter and occlusion, resulting in unstable output and missing joint points. Please refer to [link to relevant documentation]. Figure 2 This embodiment provides an artificial intelligence-based humanoid data visualization system that can prevent joint proportion misalignment and spatial position distortion, and can also continuously shake to make the data more complete. The system includes a preprocessing module, a key point module, a posture module, a humanoid health module, a three-dimensional situation module, and a visualization module.

[0045] The preprocessing module is used to acquire video data and preprocess the video data to obtain standardized boundary data; The key point module is used to obtain a set of two-dimensional key point coordinates of the human body through high-dimensional probabilistic key point processing based on standardized boundary data; The pose module is used to obtain world pose data by sequentially performing graph convolution processing, inverse kinematics Kalman filtering, and self-attention mechanism completion based on the coordinate set of two-dimensional human body key points. The humanoid health module is used to analyze the health status of humanoids based on world posture data and obtain humanoid status data. The 3D situation module is used to acquire BIM model data and perform visualization preprocessing based on world posture data and human figure status data to obtain a 3D situation map. The visualization module is used to generate an interactive security monitoring visualization interface based on the 3D situation map through rendering and graphical interaction processing.

[0046] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0047] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A human-related data visualization method based on artificial intelligence, characterized in that, The method includes the following steps: S1. Acquire video data and preprocess the video data to obtain standardized boundary data; S2. Based on standardized boundary data, a set of two-dimensional key point coordinates of the human body is obtained through high-dimensional probabilistic key point processing; S3. Based on the coordinate set of two-dimensional key points of the human body, graph convolution processing, inverse kinematics Kalman filtering, and self-attention mechanism completion are performed sequentially to obtain world pose data; S4. Analyze the health status of humanoid figures based on global posture data and obtain humanoid status data; S5. Obtain BIM model data and perform visualization preprocessing based on world posture data and human form status data to obtain a three-dimensional situation map; S6. Generate an interactive security monitoring visualization interface based on the 3D situation map through rendering and graphical interaction processing.

2. The human-related data visualization method based on artificial intelligence according to claim 1, characterized in that, In S1, the specific implementation steps are as follows: S11. Obtain the video frame sequence based on the video data from the construction site using hardware decoding technology; S12. Perform target detection based on the video frame sequence to obtain a set of human body boundary points; S13. Obtain standardized boundary data by standardizing the set of human body boundary points.

3. The human-related data visualization method based on artificial intelligence according to claim 1, characterized in that, In S2, the specific implementation steps are as follows: S21. Obtain the high-dimensional posture feature vector by capturing posture features based on standardized boundary data; S22. Based on the high-dimensional feature vector of the pose, the transposed convolution operation is used to obtain the transposed convolution value. Based on the transposed convolution value, the convolution operation is performed again to obtain the high-precision value. Then, the sigmoid function is used to map the high-precision value to a probability heat map. S23. Based on the probability heat map set, perform two-dimensional key point recognition of the human body to obtain the coordinate set of two-dimensional key points of the human body.

4. The human-related data visualization method based on artificial intelligence according to claim 1, characterized in that, In S3, the specific implementation steps are as follows: S31. Divide the two-dimensional key point coordinate set of the human body into a limb coordinate set and a trunk coordinate set based on the position of the human limbs and trunk. Calculate the geometric center formula of the limb coordinate set to obtain the limb bone nodes, and obtain the trunk center node according to the geometric center formula of the trunk coordinate set. S32. Based on the coordinate set of two-dimensional key points of the human body, the limb bone nodes and the trunk center node, the initial three-dimensional pose parameters are obtained by regression through graph convolution. S33. Obtain optimized attitude parameters by performing inverse kinematics processing based on the initial three-dimensional attitude parameters; S34. Filter and complete the optimized attitude parameters to obtain complete 3D attitude data, and perform coordinate transformation based on the complete 3D attitude data to obtain world attitude data.

5. The artificial intelligence-based humanoid data visualization method according to claim 4, characterized in that, In S34, the specific implementation steps are as follows: S341. Smooth three-dimensional attitude data is obtained by processing the optimized attitude parameters through a Kalman filter. S342. Based on the smooth 3D pose data, the occlusion data is completed using a self-attention mechanism model to obtain complete 3D pose data; S343. Obtain world attitude data by transforming the complete three-dimensional attitude data into world coordinates.

6. The human-related data visualization method based on artificial intelligence according to claim 1, characterized in that, In S4, the specific implementation steps are as follows: S41. Obtain behavioral feature embedding vectors by spatiotemporal graph convolution based on world pose data; S42. Classify behaviors using a multilayer perceptron classifier based on the behavior feature embedding vector, and obtain the initial behavior category probability distribution; S43. Obtain enhanced behavior data by processing the initial behavior category probability distribution using a conditional random field. S44. A dynamic risk index is obtained by performing risk assessment through a deep reinforcement learning decision-maker based on enhanced behavioral data. S45. Calculate the upper and lower thresholds based on the dynamic risk index, judge the humanoid behavior based on the upper and lower thresholds, and obtain humanoid status data. If the dynamic risk index is between the upper and lower thresholds, then the current humanoid behavior is humanoid health data; Conversely, the current humanoid behavior is humanoid risk data.

7. The human-related data visualization method based on artificial intelligence according to claim 1, characterized in that, In S5, the specific implementation steps are as follows: S51. Obtain BIM model data from the BIM database, and process the BIM model data through the BIM model lightweight parsing engine to obtain lightweight 3D scene mesh and semantic information; S52. Based on the world attitude data and the lightweight 3D scene mesh, the scene attitude dataset is obtained through 3D scene transformation mapping. S53. Based on the human form state data and scene pose dataset, a risk density distribution map is obtained by processing the kernel density estimation method. S54. Multi-layer scene data is obtained by processing the lightweight 3D scene mesh, scene pose dataset and risk density distribution map using dynamic rendering technology. S55. A three-dimensional situation map is obtained by visual encoding processing based on multi-layer scene data.

8. The human-related data visualization method based on artificial intelligence according to claim 1, characterized in that, In S6, the specific implementation steps are as follows: S61. Generate a multi-view coordinated visualization scene based on the 3D situation map; S62. Based on the multi-view coordinated visualization scene, the pixel frame buffer is obtained by WebGL rendering. S63. The user inputs their intention information, and an interactive security monitoring visualization interface is generated through graphical interaction processing based on the pixel frame buffer and the intention information.

9. A system applied to the artificial intelligence-based humanoid data visualization method according to any one of claims 1-8, characterized in that, The system includes: The preprocessing module is used to acquire video data and preprocess the video data to obtain standardized boundary data; The key point module is used to obtain a set of two-dimensional key point coordinates of the human body through high-dimensional probabilistic key point processing based on standardized boundary data; The pose module is used to obtain world pose data by sequentially performing graph convolution processing, inverse kinematics Kalman filtering, and self-attention mechanism completion based on the coordinate set of two-dimensional human body key points. The humanoid health module is used to analyze the health status of humanoids based on world posture data and obtain humanoid status data. The 3D situation module is used to acquire BIM model data and perform visualization preprocessing based on world posture data and human figure status data to obtain a 3D situation map. The visualization module is used to generate an interactive security monitoring visualization interface based on the 3D situation map through rendering and graphical interaction processing.

Images