Construction technology anomaly detection alarm system and comparison algorithm thereof

Abstract

The invention discloses a construction process anomaly detection alarm system and a comparison algorithm thereof, and relates to the field of cable construction, and the system specifically comprises an alternating current display module, a supervision module, a construction module, a manufacturer module, a site construction image collection module, a storage module, a construction process setting module, a construction process anomaly module and a construction image separation module. The construction process setting module is used for setting a construction process and recording a corresponding safety construction operation video; the on-site construction image acquisition module is used for acquiring on-site construction images, the construction image separation module is used for separating the acquired construction images, classifying the uploaded construction images and classifying the construction images into corresponding construction steps, and the construction process abnormity module is used for analyzing the construction images and analyzing the construction process abnormity. The operation data is obtained, and the obtained operation data is compared with the operation data in the corresponding correct construction steps.

Description

Technical Field

[0001] This invention relates to the field of cable construction control technology, specifically to a construction process anomaly detection alarm system and its comparison algorithm. Background Technology

[0002] In the process of power cable splicing, the splicing process generally follows a sequential order, and after the later process is completed, it covers the earlier process, becoming a concealed project. There is a need to improve the refined quality control methods for splicing, and more sophisticated management methods are required for the quality assessment and control of cable splices. Currently, during the installation of splice accessories, traditional mechanical measuring instruments are used to measure accessories and process management, and for most procedures, only point-like data records are kept. Although paper records and CD storage methods are now available, there are concerns that the recorded data may not be complete, and paper reports are easily damaged or modified. Furthermore, in actual construction, it is generally necessary for the construction unit, supervision unit, and product supplier to provide guidance. However, due to the distance of construction sites, it is difficult for supervision units and product suppliers to provide timely guidance, which causes inconvenience to the construction work. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a construction process anomaly detection alarm system and its comparison algorithm.

[0004] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0005] The present invention provides a construction process anomaly detection alarm system, comprising an exchange and display module, a supervision module, a construction module, a manufacturer module, a field construction image acquisition module, a storage module, a construction process setting module, a construction process anomaly module, and a construction image segmentation module;

[0006] The supervision module, construction module, manufacturer module, and communication and display module are connected, enabling users of the supervision module, construction module, and manufacturer module to exchange construction information online.

[0007] Furthermore, the supervision module is used to set up and manage the login of supervision users, the construction module is used to set up and manage the login of construction party users, and the vendor module is used to set up and manage the login of supplier users.

[0008] The construction process setting module is used to set the construction process and record corresponding safe construction operation videos; the on-site construction image acquisition module is used to acquire on-site construction images; the construction image segmentation module is used to segment the acquired construction images, classify the uploaded construction images, and classify them into corresponding construction steps; the construction process anomaly module is used to analyze the construction images, obtain operation data, and compare the obtained operation data with the operation data in the corresponding correct construction steps. When there is an error between the obtained operation data and the operation data in the correct construction steps, a construction process anomaly signal is issued.

[0009] And send the construction process abnormality signal to the supervision module, construction module, and manufacturer module;

[0010] The storage module is used to store the operation data of the on-site construction image acquisition module and the corresponding construction steps.

[0011] As a preferred technical solution of the present invention, users of the supervision module, construction module, and manufacturer module can communicate with each other through one or more methods, including voice, video, and text messages.

[0012] As a preferred embodiment of the present invention, the supervision module, construction module, and manufacturer module can arbitrarily view the on-site construction image acquisition module and the corresponding construction step operation data in the storage module.

[0013] As a preferred technical solution of the present invention, a construction process anomaly detection and comparison algorithm is characterized by comprising the following steps:

[0014] Step 1: Set behavior labels for the video streams acquired by the construction image acquisition module to classify each type of behavior video, divide the video into different action combinations, and extract the basic recognition features as complete actions;

[0015] Step 2: Preprocess the data, extract the key point coordinates of the operator in the collected images, and extract the posture information of the person under different categories as action features, and then integrate the features.

[0016] Step 3: After detecting the human body, the next step is to detect key joints, extract the posture information of people under different categories in the dataset as action features, integrate them, and then construct a training mode.

[0017] Step 4: Track the positions of key joints between different video frames to understand the continuity and changes in body movements;

[0018] Step 5: Analyze the movement and relative position of key joints, predict the position and relationship of key points, reconstruct the human posture, and obtain the human movement.

[0019] Step 6: Compare the obtained human body movements with the video movements in the construction process setting module. If there is a large error in the movements, an abnormality in the construction process will be detected.

[0020] As a preferred technical solution of the present invention, the preprocessing of the acquired video images includes denoising, image enhancement, and image stabilization; the acquired images are put into the OpenPose skeleton extraction network to extract the key point coordinate data of pedestrians, and the posture information of people under different categories is extracted as action features and saved as corresponding TXT documents; the extracted feature information and corresponding images are integrated into a TXT file, while useless redundant datasets are removed, and the integrated TXT information is used as input and output label CSV files respectively.

[0021] As a preferred embodiment of the present invention, the method for extracting the operational images from the safe construction operation video includes the following steps:

[0022] Step 1: Input a video of a safe construction operation involving people, and extract the frames from the input video to obtain video frames;

[0023] Step 2: Extract the human skeleton pose from the video frames extracted in Step 1 using the OpenPose human pose estimation algorithm to obtain a skeleton pose sequence frame, and use joint coordinates to represent the skeleton pose data information of each frame.

[0024] Step 3: Construct a topological graph structure using the joint coordinate data from Step 2; the graph structure G = (V, E) consists of two types of edges: intra-frame edges, which are built on the natural connecting nodes of the human skeleton in each frame, forming a node set V = {Vti│t = 1, 2, ... T, i = 1, 2, ... N}; and inter-frame edges, which connect the same nodes in two consecutive frames, forming an edge set ES = {VtiVtj│(i, j) ∈ N} and EF = {VtiV(t+1)j│i ∈ N}.

[0025] Step 4: Use the joint coordinate vector of the joint sequence in 2D or 3D coordinates in the graph structure obtained in Step 3 as the input of the network, and perform normalization and standardization preprocessing on the input data;

[0026] Step 5: Perform adaptive graph convolution operation on the preprocessed data from Step 4;

[0027] Step 6: Cross-apply the 9-layer Sparegraph ConventionNetwork and Temporal Graph ConventionNetwork operations to the input data processed in Step 5 to generate higher-level feature map output;

[0028] Step 7: The high-level feature map output after step 6 is processed by average pooling and fully connected FC. The processed result is then classified by a Soft Max classifier to obtain the action category prediction result corresponding to each frame action.

[0029] Step 8: Fuse the frame-by-frame motion prediction results obtained in Step 7 with the character limb skeleton pose sequence frames extracted from the corresponding original video to obtain the fused video frames, that is, to fuse the motion prediction results with the skeleton pose frames.

[0030] Step 9: Using the video frames from Step 8 as elements, set the synthesis parameters to obtain a new character action category recognition video.

[0031] As a preferred technical solution of the present invention, the obtained human body movements are compared with the movements in the character movement category recognition video. When there is a large error in the movements, an abnormality in the construction process is detected.

[0032] The beneficial effects of this invention are:

[0033] 1. This construction process anomaly detection and alarm system includes a communication and display module, a supervision module, a construction module, a manufacturer module, a site construction image acquisition module, a storage module, a construction process setting module, a construction process anomaly module, and a construction image segmentation module. The construction process setting module configures the construction process and records corresponding safe construction operation videos. The site construction image acquisition module acquires site construction images. The construction image segmentation module segments the acquired construction images, classifying them into corresponding construction steps. The construction process anomaly module analyzes the construction images, obtains operational data, and compares this data with the correct operational data for each construction step. When discrepancies exist between the acquired and correct operational data, a construction process anomaly signal is issued and sent to the supervision, construction, and manufacturer modules. This allows for simultaneous three-way monitoring. Users of the supervision, construction, and manufacturer modules can exchange construction information online, provide remote guidance, and collect and record site construction images.

[0034] 2. This invention also provides a construction process anomaly detection and comparison algorithm. Specifically, it assigns behavior labels to the video streams acquired by the construction image acquisition module, classifies each type of behavior video, divides the video into different action combinations, extracts the basic recognition features of the complete action, preprocesses the data, extracts the key point coordinates of the operator in the acquired images, and extracts the posture information of the person under different categories as action features. Then, it integrates the features. After detecting the human body, it detects key joints, extracts the posture information of the person under different categories in the dataset as action features, integrates them, and then constructs a training mode to track the position of key joints between different video frames to grasp the continuity and changes of limb movements. It analyzes the movement and relative position of key joints, predicts the position and correlation of key points, reconstructs the human posture, and obtains the human action. The obtained human action is compared with the video action of the construction process setting module. When the action has a large error, a construction process anomaly is detected. This construction process anomaly detection facilitates timely correction and cooperates with third parties to detect safe construction and safe operation. Attached Figure Description

[0035] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0036] In the attached diagram:

[0037] Figure 1 This is a flowchart illustrating a construction process anomaly detection and alarm system according to the present invention;

[0038] Figure 2 This is a flowchart illustrating a construction process anomaly detection alarm comparison algorithm according to the present invention.

[0039] The diagram shows: 1. Communication and display module; 2. Supervision module; 3. Construction module; 4. Vendor module; 5. On-site construction image acquisition module; 6. Storage module; 7. Construction process setting module; 8. Construction process anomaly module; 9. Construction image separation module. Detailed Implementation

[0040] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0041] Example: Figure 1 As shown, the present invention provides a construction process anomaly detection alarm system, comprising an communication and display module 1, a supervision module 2, a construction module 3, a manufacturer module 4, a field construction image acquisition module 5, a storage module 6, a construction process setting module 7, a construction process anomaly module 8, and a construction image separation module 9.

[0042] The supervision module 2, construction module 3, and manufacturer module 4 are connected to the communication and display module 1, enabling users of the supervision module 2, construction module 3, and manufacturer module 4 to exchange construction information online.

[0043] Furthermore, the supervision module 2 is used to set up and manage the login of supervision users, the construction module 3 is used to set up and manage the login of construction users, and the manufacturer module 4 is used to set up and manage the login of supplier users.

[0044] The construction process setting module 7 is used to set the construction process and record corresponding safe construction operation videos; the on-site construction image acquisition module 5 is used to acquire on-site construction images; the construction image segmentation module 9 is used to segment the acquired construction images, classify the uploaded construction images, and classify them into the corresponding construction steps; the construction process anomaly module 8 is used to analyze the construction images, obtain operation data, and compare the obtained operation data with the operation data in the corresponding correct construction steps. When there is an error between the obtained operation data and the operation data in the correct construction steps, a construction process anomaly signal is issued.

[0045] And send the construction process abnormality signal to the supervision module 2, construction module 3, and manufacturer module 4;

[0046] The storage module 6 is used to store the operation data of the on-site construction image acquisition module and the corresponding construction steps. This enables simultaneous three-party supervision, allowing users of the supervision module, construction module, and manufacturer module to exchange construction information online, provide remote guidance, and acquire and record on-site construction images.

[0047] Users of the supervision module 2, construction module 3, and manufacturer module 4 can communicate with each other through voice, video, and text messages in one or more ways.

[0048] The supervision module 2, construction module 3, and manufacturer module 4 can all view the on-site construction image acquisition module and the corresponding construction step operation data in the storage module 6, which facilitates later retrieval.

[0049] One such construction process anomaly detection comparison algorithm is... Figure 2 As shown, it includes the following steps:

[0050] Step 1: Set behavior labels for the video streams acquired by the construction image acquisition module to classify each type of behavior video, divide the video into different action combinations, and extract the basic recognition features as complete actions;

[0051] Step 2: Preprocess the data, extract the key point coordinates of the operator in the collected images, and extract the posture information of the person under different categories as action features, and then integrate the features.

[0052] Step 3: After detecting the human body, the next step is to detect key joints, extract the posture information of people under different categories in the dataset as action features, integrate them, and then construct a training mode.

[0053] Step 4: Track the positions of key joints between different video frames to understand the continuity and changes in body movements;

[0054] Step 5: Analyze the movement and relative position of key joints, predict the position and relationship of key points, reconstruct the human posture, and obtain the human movement.

[0055] Step 6: Compare the obtained human body movements with the video movements in the construction process setting module (7). If there is a large error in the movements, an abnormality in the construction process will be detected. This allows for timely correction of construction process abnormalities and facilitates the detection of safe construction and operation by all three parties.

[0056] The method for extracting operational images from the aforementioned safe construction operation video includes the following steps:

[0057] Step 1: Input a video of a safe construction operation involving people, and extract the frames from the input video to obtain video frames;

[0058] Step 2: Extract the human skeleton pose from the video frames extracted in Step 1 using the OpenPose human pose estimation algorithm to obtain a skeleton pose sequence frame, and use joint coordinates to represent the skeleton pose data information of each frame.

[0059] Step 3: Construct a topology graph structure using the joint coordinate data from Step 2;

[0060] The graph structure G = (V, E) consists of two types of edges: intra-frame edges, which are built on the natural connecting nodes of the human skeleton in each frame, forming a node set V = {Vti│t = 1, 2, ... T, i = 1, 2, ... N}; and inter-frame edges, which connect the same nodes in two consecutive frames, forming an edge set ES = {VtiVtj│(i, j) ∈ N} and EF = {VtiV(t+1)j│i ∈ N}.

[0061] Step 4: Use the joint coordinate vector of the joint sequence in 2D or 3D coordinates in the graph structure obtained in Step 3 as the input of the network, and perform normalization and standardization preprocessing on the input data;

[0062] Step 5: Perform adaptive graph convolution operation on the preprocessed data from Step 4;

[0063] Step 6: Cross-apply the 9-layer Sparegraph ConventionNetwork and Temporal Graph ConventionNetwork operations to the input data processed in Step 5 to generate higher-level feature map output;

[0064] The high-level feature map output after step 6 is processed by average pooling (POOL) and fully connected FC. The processed result is then classified by a Soft Max classifier to obtain the action category prediction result corresponding to each frame action.

[0065] Step 8: Fuse the frame-by-frame motion prediction results obtained in Step 7 with the character limb skeleton pose sequence frames extracted from the corresponding original video to obtain the fused video frames, that is, to fuse the motion prediction results with the skeleton pose frames.

[0066] Step 9: Using the video frames from Step 8 as elements, set the synthesis parameters to obtain a new character action category recognition video.

[0067] The process involves comparing the obtained human body movements with the movements in the character movement category recognition video. When there is a large error in the movement, an abnormality in the construction process is detected. This method of comparing video movements is used to determine whether there is an abnormality in the construction process, thereby achieving the effect of automatic detection.

[0068] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A construction process anomaly detection and alarm system, characterized in that, It includes a communication and display module (1), a supervision module (2), a construction module (3), a manufacturer module (4), a site construction image acquisition module (5), a storage module (6), a construction process setting module (7), a construction process anomaly module (8), and a construction image separation module (9); The supervision module (2), construction module (3), and manufacturer module (4) are connected to the communication and display module (1), enabling users of the supervision module (2), construction module (3), and manufacturer module (4) to exchange construction information online. Furthermore, the supervision module (2) is used to set up the login of the supervision user and manage the login of the supervision user, the construction module (3) is used to set up the login of the construction party user and manage the login of the construction party user, and the manufacturer module (4) is used to set up the login of the supplier user and manage the login of the supplier user; The construction process setting module (7) is used to set the construction process and record the corresponding safe construction operation video; the on-site construction image acquisition module (5) is used to acquire on-site construction images; the construction image segmentation module (9) is used to segment the acquired construction images, classify the uploaded construction images, and classify them into the corresponding construction steps; the construction process abnormal module (8) is used to analyze the construction images, obtain operation data, and compare the obtained operation data with the operation data in the corresponding correct construction steps. When there is an error between the obtained operation data and the operation data in the correct construction steps, a construction process abnormal signal is issued. And send the construction process abnormality signal to the supervision module (2), construction module (3), and manufacturer module (4); The storage module (6) is used to store the operation data of the on-site construction image acquisition module and the corresponding construction steps.

2. The construction process anomaly detection alarm system according to claim 1, characterized in that, Users of the supervision module (2), construction module (3), and manufacturer module (4) can communicate with each other through voice, video, and text messages in one or more ways.

3. The construction process anomaly detection alarm system according to claim 2, characterized in that, The supervision module (2), construction module (3), and manufacturer module (4) can arbitrarily view the on-site construction image acquisition module and the corresponding construction step operation data in the storage module (6).

4. The construction process anomaly detection and comparison algorithm according to any one of claims 1 to 3, characterized in that: Includes the following steps: Step 1: Set behavior labels for the video streams acquired by the construction image acquisition module to classify each type of behavior video, divide the video into different action combinations, and extract the basic recognition features as complete actions; Step 2: Preprocess the data, extract the key point coordinates of the operator in the collected images, and extract the posture information of the person under different categories as action features, and then integrate the features. Step 3: After detecting the human body, the next step is to detect key joints, extract the posture information of people under different categories in the dataset as action features, integrate them, and then construct a training mode. Step 4: Track the positions of key joints between different video frames to understand the continuity and changes in body movements; Step 5: Analyze the movement and relative position of key joints, predict the position and relationship of key points, reconstruct the human posture, and obtain the human movement. Step 6: Compare the obtained human body movements with the video movements of the construction process setting module (7). If there is a large error in the movements, an abnormality in the construction process will be issued.

5. The construction process anomaly detection and comparison algorithm according to claim 4, characterized in that, The process involves preprocessing the acquired video images, including denoising, image enhancement, and image stabilization. The acquired images are then fed into an OpenPose skeleton extraction network to extract key point coordinates of pedestrians and to extract pose information of people under different categories as action features, which are saved as corresponding TXT documents. The extracted feature information and corresponding images are then integrated into a TXT file, while useless and redundant datasets are removed. The integrated TXT information is used as input and output label CSV files, respectively.

6. A multi-stage precise temperature control method based on a neural network algorithm according to any one of claims 1-5, characterized in that, The method for extracting operational images from the aforementioned safe construction operation video includes the following steps: Step 1: Input a video of a safe construction operation involving people, and extract the frames from the input video to obtain video frames; Step 2: Extract the human skeleton pose from the video frames extracted in Step 1 using the OpenPose human pose estimation algorithm to obtain a skeleton pose sequence frame, and use joint coordinates to represent the skeleton pose data information of each frame. Step 3: Construct a topology graph structure using the joint coordinate data from Step 2; The graph structure G = (V, E) consists of two types of edges: intra-frame edges, which are built on the natural connecting nodes of the human skeleton in each frame, forming a node set V = {Vti│t = 1, 2, ... T, i = 1, 2, ... N}; and inter-frame edges, which connect the same nodes in two consecutive frames, forming an edge set ES = {VtiVtj│(i, j) ∈ N} and EF = {VtiV(t+1)j│i ∈ N}. Step 4: Use the joint coordinate vector of the joint sequence in 2D or 3D coordinates in the graph structure obtained in Step 3 as the input of the network, and perform normalization and standardization preprocessing on the input data; Step 5: Perform adaptive graph convolution operation on the preprocessed data from Step 4; Step 6: Cross-apply the 9-layer Sparegraph ConventionNetwork and Temporal Graph ConventionNetwork operations to the input data processed in Step 5 to generate higher-level feature map output; The high-level feature map output after step 6 is processed by average pooling (POOL) and fully connected FC. The processed result is then classified by a Soft Max classifier to obtain the action category prediction result corresponding to each frame action. Step 8: Fuse the frame-by-frame motion prediction results obtained in Step 7 with the character limb skeleton posture sequence frames extracted from the corresponding original video to obtain the fused video frames, that is, to fuse the motion prediction results with the skeleton posture frames. Step 9: Using the video frames from Step 8 as elements, set the synthesis parameters to obtain a new character action category recognition video.

7. The construction process anomaly detection and comparison algorithm according to claim 6, characterized in that, The obtained human body movements are compared with the movements in the character movement category recognition video. When there is a large error in the movement, an abnormality in the construction process is detected.

Images