Construction intelligent supervision method and system based on multi-modal perception

Abstract

The application relates to the technical field of engineering management, and particularly discloses a construction intelligent supervision method and system based on multi-modal perception, which comprises the following steps: performing feature decoupling on original perception data to obtain positioning data flow, image frame sequences and environmental monitoring value sets; performing space-time alignment on positions and postures to obtain identity identification and continuous action frame sequences, performing safety threshold determination on abnormal environmental states to obtain first space-time positioning information; performing feature matching on the continuous action frame sequences, performing event attribution mapping of second space-time positioning information and the identity identification, and obtaining violation behavior records; performing space-time coupling analysis on abnormal environmental states and dangerous operation behaviors to obtain linkage early warning information; encoding and delivering hierarchical intervention instructions to obtain intervention feedback data; and performing cleaning and archiving on the violation behavior records, the linkage early warning information and the intervention feedback data to construct individual safety history archives; and the application can improve the efficiency of construction intelligent supervision.

Description

Technical Field

[0001] This invention relates to the field of engineering management technology, and in particular to a construction intelligent supervision method and system based on multimodal perception. Background Technology

[0002] Existing technologies for intelligent monitoring of construction sites use a single-dimensional analysis method to process multi-source sensing data. They do not perform targeted feature decoupling processing on the original sensing data, and cannot effectively separate and extract positioning, image, and environmental monitoring data. This results in the feature information of different types of data being mixed together, making it difficult to accurately identify the position, posture, and on-site environmental conditions of workers. Consequently, the data utilization efficiency is low, and it cannot provide clear and effective data support for subsequent monitoring operations.

[0003] Existing construction site supervision technologies lack the ability to analyze the spatiotemporal coupling of abnormal environmental conditions and personnel operational behaviors. They can only make single judgments and issue warnings for abnormal environments or dangerous operations, and cannot conduct spatiotemporal correlation analysis between the two. It is difficult to identify the superimposed risks formed by the interaction between the environment and behavior. At the same time, a deep correlation mapping between personnel identity, violations, and warning events has not been established, and a complete individual safety record file for personnel cannot be constructed. As a result, the targeting and comprehensiveness of supervision and warning are insufficient, and the intervention of safety risks at construction sites lacks systematicness and effectiveness. Summary of the Invention

[0004] This invention provides a construction intelligent supervision method and system based on multimodal perception to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides a construction intelligent monitoring method based on multimodal perception, comprising: S1. Decouple the original sensing data of the construction site by feature decoupling to obtain the location data stream, image frame sequence and environmental monitoring data set of the construction site; S2. Based on the positioning data stream and image frame sequence, the position and posture of the workers in the construction site are spatiotemporally aligned to obtain the workers' identification and continuous action frame sequence. Based on the environmental monitoring data set, the safety threshold of the abnormal environmental state of the construction site is determined to obtain the first spatiotemporal positioning information of the construction site. S3. Based on the preset standard action feature library, perform feature matching on the continuous action frame sequence, and perform event attribution mapping between the second spatiotemporal positioning information of the identified dangerous operation behavior and the identity identifier to obtain the record of the violation behavior at the construction site. S4. Based on the first and second spatiotemporal positioning information, a spatiotemporal coupling analysis is performed on the abnormal environmental state and dangerous operation behavior to obtain the linkage early warning information of the construction site. S5. Based on violation records and linkage early warning information, the graded intervention instructions at the construction site are coded and delivered to obtain intervention feedback data at the construction site. S6. Clean and archive records of violations, linked early warning information, and intervention feedback data to build individual safety record files for construction sites.

[0006] In a preferred embodiment, the step of decoupling the original sensing data from the construction site to obtain the location data stream, image frame sequence, and environmental monitoring data set of the construction site includes: The positioning pulse signal at the construction site is demodulated by time difference to obtain the positioning data stream at the construction site; Motion contour extraction is performed on the real-time video stream of the construction site to obtain the image frame sequence of the construction site; The dimensions of the analog signals at the construction site are restored to obtain the environmental monitoring data set of the construction site.

[0007] In a preferred embodiment, the step of spatiotemporally aligning the position and posture of workers at the construction site based on the positioning data stream and image frame sequence to obtain the workers' identification and continuous action frame sequence includes: The location beacons in the location data stream are timestamped to obtain the set of location points of the workers at the construction site; By performing joint spatial coordinate detection on the image frame sequence, the set of posture parameters of the workers at the construction site is obtained; Based on the time tags of the worker location point set, the worker posture parameter set is registered on the time axis to obtain the position and posture association pair at the construction site. By interpolating the position and attitude correlation pairs, a continuous sequence of action frames at the construction site is obtained. The facial region of the first frame in a continuous action frame sequence is feature-encoded, and the encoded facial feature vector is matched with a pre-stored personnel information database to obtain the identity identifier of the construction site.

[0008] In a preferred embodiment, the step of determining a safety threshold for abnormal environmental conditions at the construction site based on an environmental monitoring data set to obtain the first spatiotemporal location information of the construction site includes: The threshold intervals of the monitoring values ​​in the environmental monitoring data set are traversed to obtain the abnormal trigger values ​​at the construction site; By attributing and tracing the abnormal trigger values, the physical addresses and timestamp sequences of the sensors at the construction site are obtained; Based on the physical addresses of the sensors, spatial coordinate mapping is performed on the sensor deployment point map of the construction site to obtain the three-dimensional spatial coordinates of the construction site location. By associating and encapsulating three-dimensional spatial coordinates with timestamp sequences, the first spatiotemporal positioning information of the construction site is obtained.

[0009] In a preferred embodiment, the step of performing feature matching on a continuous sequence of action frames based on a preset standard action feature library, and mapping the second spatiotemporal location information of the identified dangerous operation behavior with the identity identifier to obtain a record of violations at the construction site, includes: Trajectory fitting is performed on the motion data in a continuous sequence of motion frames to obtain the motion trajectory feature vector of the construction site; Based on a pre-set standard action feature library, template indexing is performed on the action trajectory feature vectors to obtain dangerous operational behaviors at the construction site. Spatial coordinate inversion is performed on dangerous operation behaviors to obtain the three-dimensional spatial coordinates of the start time, end time and trigger location of the dangerous operation behaviors, which constitute the second spatiotemporal positioning information of the construction site; Based on the identity identifier, the second spatiotemporal location information is associated and indexed, and the associated data after indexing is packaged into events to obtain the record of violations at the construction site.

[0010] In a preferred embodiment, the step of performing spatiotemporal coupling analysis on abnormal environmental conditions and dangerous operational behaviors based on first and second spatiotemporal positioning information to obtain coordinated early warning information for the construction site includes: Protocol parsing is performed on the first spatiotemporal positioning information to obtain the spatiotemporal anchor points of environmental events at the construction site; Trajectory delineation is performed on the second spatiotemporal positioning information to obtain spatiotemporal anchor points for behavioral events at the construction site; Based on the time stamp code of the spatiotemporal anchor points of environmental events, the time windows of the spatiotemporal anchor points of behavioral events are compared between intervals to obtain the time coupling criteria of the construction site. Based on the spatial coordinates of spatiotemporal anchor points of environmental events, a proximity search is performed on the spatial trajectory point series of spatiotemporal anchor points of behavioral events to obtain the spatial coupling criterion of the construction site. Based on time-coupled criteria and spatial-coupled criteria, risk accumulation is performed on spatiotemporal anchor points of environmental events and spatiotemporal anchor points of behavioral events to obtain primary early warning events at the construction site. Risk levels are determined for primary early warning events, and verification packages are generated based on preset early warning message templates to obtain coordinated early warning information for the construction site.

[0011] In a preferred embodiment, the step of accumulating risks for spatiotemporal anchor points of environmental events and behavioral events based on time-coupling and spatial coupling criteria to obtain primary early warning events at the construction site includes: The spatiotemporal correlation strength between environmental event spatiotemporal anchor points and behavioral event spatiotemporal anchor points is quantitatively characterized, and the calculation formula for the spatiotemporal correlation strength is as follows: ; In the formula, For spatiotemporal correlation strength, The time of occurrence of the abnormal environmental event within the spatiotemporal anchor point of the environmental event. This refers to the midpoint of the time window for dangerous operational behaviors within the spatiotemporal anchor point of the behavioral event. It is the shortest spatial distance from the hazardous operation behavior trajectory point in the spatiotemporal anchor point of the behavioral event to the location of the abnormal environmental event in the spatiotemporal anchor point of the environmental event. As a preset time tolerance, The preset space tolerance; When the spatiotemporal correlation strength is not less than the preset coupling activation threshold, abnormal environmental events and dangerous operational behaviors are fused to obtain primary early warning events at the construction site.

[0012] In a preferred embodiment, the step of encoding and delivering tiered intervention instructions at the construction site based on violation records and linked early warning information to obtain intervention feedback data at the construction site includes: Risk level assessment is performed on records of violations to determine the first intervention priority indicator for the construction site. By analyzing the event type of the linked early warning information, the second intervention priority identifier of the construction site is obtained; Arbitrate the first intervention priority identifier and the second intervention priority identifier to obtain the intervention instruction header at the construction site; Based on the identification of the personnel involved and the location of the incident in the violation record, the target address field and location field of the intervention command header are loaded to obtain the intervention command frame at the construction site; The intervention command frame is encapsulated and then delivered in a targeted manner to obtain intervention feedback data from the construction site.

[0013] In a preferred embodiment, the step of cleaning and archiving violation records, linked early warning information, and intervention feedback data to construct an individual safety record file at the construction site includes: By collecting and summarizing records of violations, linked early warning information, and intervention feedback data, the original event pool of the construction site is obtained; Perform field integrity checks on the original event pool, and reformat the data that passes the checks to obtain a standardized event record set for the construction site; The standardized event record set is deduplicated and compared, and the merged event records are correlated and mapped with the intervention feedback data to obtain event intervention correlation pairs at the construction site; Based on the identification of the personnel involved in the event intervention association pair, the event records of the operators are classified and aggregated to obtain the individual event sequence of the operators; Key fields are extracted from event records in individual event sequences, and the extracted key fields are arranged into documents according to a preset file template to obtain individual safety record files for construction sites.

[0014] To address the aforementioned problems, this invention also provides a construction intelligent monitoring system based on multimodal perception, the system comprising: The data decoupling processing module is used to decouple the features of the raw sensing data at the construction site to obtain the location data stream, image frame sequence and environmental monitoring data set of the construction site. The spatiotemporal alignment determination module is used to perform spatiotemporal alignment of the position and posture of workers in the construction site based on the positioning data stream and image frame sequence, to obtain the worker's identity and continuous action frame sequence, and to determine the safety threshold of abnormal environmental conditions at the construction site based on the environmental monitoring data set, so as to obtain the first spatiotemporal positioning information of the construction site. The behavior matching and attribution module is used to perform feature matching on a continuous action frame sequence based on a preset standard action feature library, and to perform event attribution mapping between the second spatiotemporal location information of the identified dangerous operation behavior and the identity identifier to obtain the record of violations at the construction site. The coupling analysis and early warning module is used to perform spatiotemporal coupling analysis on abnormal environmental conditions and dangerous operational behaviors based on the first and second spatiotemporal positioning information to obtain linkage early warning information for the construction site. The instruction coding and delivery module is used to encode and deliver graded intervention instructions at the construction site based on violation records and linkage early warning information, so as to obtain intervention feedback data at the construction site. The archive building module is used to clean and archive records of violations, linkage early warning information, and intervention feedback data to build individual safety history archives at the construction site.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention achieves precise separation and independent extraction of positioning, image, and environmental monitoring data by performing targeted feature decoupling processing on the original sensing data of the construction site. At the same time, it completes the spatiotemporal alignment of the position and posture of the workers, and can accurately obtain the worker's identification, continuous action frame sequence, and spatiotemporal positioning information of abnormal on-site environment. This makes the acquisition of basic data for construction supervision more accurate and comprehensive, greatly improves the utilization efficiency of construction supervision data, and provides clear and effective data support for subsequent full-process supervision operations.

[0016] 2. This invention realizes the spatiotemporal coupling analysis of abnormal environmental conditions and dangerous operational behaviors, which can accurately identify the superimposed risks formed by their interaction. At the same time, it achieves targeted risk intervention by relying on the coded delivery of graded intervention instructions, and constructs a complete individual safety record file for personnel by cleaning and archiving various types of regulatory data. This makes the early warning of construction safety supervision more comprehensive and targeted, and the risk intervention more systematic and effective, thereby comprehensively improving the overall level and actual effect of intelligent supervision at construction sites. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a construction intelligent monitoring method based on multimodal perception provided in an embodiment of the present invention. Figure 2 A functional module diagram of a construction intelligent monitoring system based on multimodal perception provided in an embodiment of the present invention; The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] This application provides a construction intelligent supervision method based on multimodal perception. The executing entity of the construction intelligent supervision method based on multimodal perception includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application embodiment: a server, a terminal, etc. In other words, the construction intelligent supervision method based on multimodal perception can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0020] Reference Figure 1 The diagram shown is a flowchart illustrating a construction intelligent monitoring method based on multimodal perception provided in an embodiment of the present invention. In this embodiment, the construction intelligent monitoring method based on multimodal perception includes: S1. Decouple the original sensing data of the construction site by feature decoupling to obtain the location data stream, image frame sequence and environmental monitoring data set of the construction site; In this embodiment of the invention, the step of decoupling the original sensing data of the construction site to obtain the location data stream, image frame sequence, and environmental monitoring data set of the construction site includes: The positioning pulse signal at the construction site is demodulated by time difference to obtain the positioning data stream at the construction site; Motion contour extraction is performed on the real-time video stream of the construction site to obtain the image frame sequence of the construction site; The dimensions of the analog signals at the construction site are restored to obtain the environmental monitoring data set of the construction site.

[0021] The system continuously receives positioning pulse signals emitted by positioning terminals deployed according to a pre-defined layout throughout the construction site. It timestamps the pulse signals collected by each receiver, arranging all positioning pulse signals in absolute chronological order to establish a pulse signal time sequence. Based on the time difference between the same positioning pulse signal received by different receivers, it performs signal demodulation processing. Through demodulation, it reconstructs the spatial location information of personnel and construction equipment carried in the positioning pulse signals. The reconstructed spatial location information is then systematically organized according to the corresponding time nodes, forming a set of location information continuously arranged along the time dimension, thus obtaining the positioning data stream of the construction site.

[0022] The video stream from the camera equipment deployed at key work areas and work points on the construction site is captured frame by frame at equal time intervals to obtain continuous on-site image frames. Pixel grayscale values ​​of each image are analyzed, and pixel separation of the foreground and background areas is achieved through grayscale value differences. Object areas in the image that exhibit dynamic movement are identified, and the contour features of the objects within these areas are extracted. Image frames with complete contour features are retained. All image frames with extracted motion contour features are arranged sequentially according to the original video stream's time acquisition order to form a continuous image frame sequence, thus obtaining the image frame sequence of the construction site.

[0023] The system continuously receives analog signals collected in real time from various environmental monitoring sensors deployed at the construction site, such as those for temperature, humidity, dust concentration, and harmful gas content. A signal conversion circuit transforms these continuously changing analog signals into discrete digital signals. Pre-defined dimensional standards corresponding to various environmental monitoring indicators are retrieved. These standards include the legal measurement dimensions for the corresponding monitoring indicators and the customized monitoring dimensions for the construction site. Based on these standards, the converted digital signals are numerically restored to eliminate dimensional deviations generated during sensor acquisition and signal transmission, ensuring that the restored values ​​are consistent with the physical quantities of the actual environmental monitoring indicators. The restored environmental monitoring values ​​are then categorized according to the monitoring indicator type and systematically integrated within each category according to the signal acquisition time sequence, forming a categorized and chronologically arranged set of environmental monitoring values, thus obtaining the environmental monitoring value set for the construction site.

[0024] The beneficial effects are that targeted feature decoupling processing is carried out on the original sensing data at the construction site. By demodulating the time difference of positioning pulse signals, extracting the motion contours of real-time video streams, and restoring the dimensions of analog signals, the precise separation and independent extraction of positioning, image, and environmental monitoring data can be achieved. This effectively eliminates the problem of feature mixing of different types of sensing data, making the data sources of positioning data streams, image frame sequences, and environmental monitoring data sets more accurate and the data features clearer. At the same time, it ensures the integrity and timeliness of various types of data, providing high-quality and highly available basic data support for subsequent full-process supervision operations such as the analysis of the position and posture of construction site workers and the determination of environmental conditions. This improves the accuracy and effectiveness of intelligent construction supervision from the data source.

[0025] S2. Based on the positioning data stream and image frame sequence, the position and posture of the workers in the construction site are spatiotemporally aligned to obtain the workers' identification and continuous action frame sequence. Based on the environmental monitoring data set, the safety threshold of the abnormal environmental state of the construction site is determined to obtain the first spatiotemporal positioning information of the construction site. In this embodiment of the invention, the step of spatiotemporally aligning the position and posture of workers at the construction site based on the positioning data stream and image frame sequence to obtain the workers' identification and continuous action frame sequence includes: The location beacons in the location data stream are timestamped to obtain the set of location points of the workers at the construction site; By performing joint spatial coordinate detection on the image frame sequence, the set of posture parameters of the workers at the construction site is obtained; Based on the time tags of the worker location point set, the worker posture parameter set is registered on the time axis to obtain the position and posture association pair at the construction site. By interpolating the position and attitude correlation pairs, a continuous sequence of action frames at the construction site is obtained. The facial region of the first frame in a continuous action frame sequence is feature-encoded, and the encoded facial feature vector is matched with a pre-stored personnel information database to obtain the identity identifier of the construction site.

[0026] The method of determining safety thresholds for abnormal environmental conditions at the construction site based on environmental monitoring data sets to obtain the first spatiotemporal location information of the construction site includes: The threshold intervals of the monitoring values ​​in the environmental monitoring data set are traversed to obtain the abnormal trigger values ​​at the construction site; By attributing and tracing the abnormal trigger values, the physical addresses and timestamp sequences of the sensors at the construction site are obtained; Based on the physical addresses of the sensors, spatial coordinate mapping is performed on the sensor deployment point map of the construction site to obtain the three-dimensional spatial coordinates of the construction site location. By associating and encapsulating three-dimensional spatial coordinates with timestamp sequences, the first spatiotemporal positioning information of the construction site is obtained.

[0027] The positioning beacons carried in the positioning data stream are parsed one by one. The timestamp information and spatial location information stored in the preset fields of each positioning beacon are extracted. The timestamp information is used as a unique time tag and bound to the corresponding spatial location information. All the bound positioning beacon information is integrated in an orderly manner according to the order of the timestamps to form a set containing the spatial location of the workers at different time points, thus obtaining the set of worker location points at the construction site.

[0028] Human target detection is performed on each frame of the image frame sequence. After locking the human contour region of the worker in the image, the key joints of the human body in the region are identified and located. The three-dimensional spatial coordinates of each key joint in the image coordinate system are extracted. The spatial coordinates of all joints of the same worker in a single frame image are integrated to form the single frame posture parameters. Then, the posture parameters corresponding to all image frames are arranged in the time order of the image frames to obtain the posture parameter set of the workers at the construction site.

[0029] Retrieve the time tags bound to all entries in the worker location point set, establish a unified time axis reference, match each set of posture parameters in the worker posture parameter set to the time node corresponding to the time axis reference according to its collection time, so that each time node is associated with the worker's spatial location information and posture parameter information. Bind the location information and posture parameters that have completed the time node matching one-to-one to obtain the location and posture association pair of the construction site.

[0030] The position and attitude association pairs are continuously traversed according to the time axis reference to identify missing nodes on the time axis that are not associated with any position information and attitude parameters. Linear completion is performed based on the position information and attitude parameters of the adjacent bound position and attitude association pairs before and after the missing node. The position information and attitude parameters corresponding to the missing node are completed and bound, forming a position and attitude association sequence without gaps and continuously arranged according to the time axis, thus obtaining a continuous action frame sequence of the construction site.

[0031] Face region detection is performed on the first frame image in the continuous action frame sequence. The face region of the worker is locked according to the preset face region pixel range threshold. The texture, contour and other feature information in the face region are feature encoded to generate a unique corresponding face feature vector. The face feature vectors of all personnel in the pre-stored personnel information database are retrieved and matched one by one with the vector. When the feature overlap of the matching reaches the preset feature matching threshold, the corresponding personnel identity information in the personnel information database is extracted to obtain the identity identifier of the construction site.

[0032] The preset safety threshold ranges corresponding to various environmental monitoring indicators in the environmental monitoring data set are retrieved. These threshold ranges are the numerical ranges jointly determined by the statutory safety limits and customized safety control limits of each environmental monitoring indicator at the construction site. For each monitoring value in the environmental monitoring data set, the corresponding safety threshold range is matched one by one according to the monitoring indicator category. The full threshold range is traversed and verified for the monitoring values. Monitoring values ​​that exceed the upper limit or fall below the lower limit of the corresponding safety threshold range are extracted and marked to obtain the abnormal trigger values ​​of the construction site.

[0033] Data source analysis is performed on the extracted and marked abnormal trigger values. The sensor identification information stored in the abnormal trigger value data field is retrieved and matched with the corresponding sensor physical address. At the same time, the timestamp information bound when the abnormal trigger value is generated is extracted. The timestamp information corresponding to all abnormal trigger values ​​under the same sensor physical address is arranged in chronological order to form a continuous set of timestamp information, thus obtaining the sensor physical address and timestamp sequence of the construction site.

[0034] The pre-stored sensor deployment point map at the construction site is retrieved. This map contains a one-to-one correspondence between the physical addresses of all sensors at the construction site and the three-dimensional spatial coordinates of their corresponding deployment locations. The sensor physical addresses obtained through attribution and tracing are accurately matched in the sensor deployment point map, and the three-dimensional spatial coordinates of the deployment locations bound to the sensor physical addresses are extracted. This completes the accurate conversion from sensor physical addresses to spatial locations, resulting in the three-dimensional spatial coordinates of the construction site location.

[0035] The matched three-dimensional spatial coordinates of the location are associated with the corresponding timestamp sequence. Each timestamp in the timestamp sequence is bound and encapsulated with the three-dimensional spatial coordinates of the location, so that each abnormal time node corresponds to a unique spatial location of the abnormality. This forms an integrated data set containing the time information and spatial location information of the abnormal environmental state, thus obtaining the first spatiotemporal positioning information of the construction site.

[0036] The beneficial effects are as follows: by performing spatiotemporal alignment processing on the positioning data stream and image frame sequence to achieve accurate matching and continuous presentation of the position and posture information of the workers, the system can accurately identify the identity of the workers, enabling construction supervision to obtain comprehensive behavioral and identity-related information of the workers. Furthermore, by determining the safety threshold of abnormal environmental conditions using environmental monitoring data sets, the system can accurately identify and spatiotemporally locate abnormal environmental conditions at the construction site, accurately capturing the location and time information of abnormal environmental occurrences. These two types of processing form a dual accurate perception system of personnel behavior and environmental conditions, providing accurate, comprehensive, and relevant basic data for subsequent identification of hazardous behaviors at the construction site and coupled analysis of the environment and behavior. This improves the accuracy and comprehensiveness of intelligent construction supervision from the two core dimensions of personnel and environment, allowing supervision operations to conduct targeted analysis and handling based on accurate basic information.

[0037] S3. Based on the preset standard action feature library, perform feature matching on the continuous action frame sequence, and perform event attribution mapping between the second spatiotemporal positioning information of the identified dangerous operation behavior and the identity identifier to obtain the record of the violation behavior at the construction site. In this embodiment of the invention, the step of performing feature matching on a continuous sequence of action frames based on a preset standard action feature library, and mapping the second spatiotemporal location information of the identified dangerous operation behavior with the identity identifier to obtain a record of violations at the construction site, includes: Trajectory fitting is performed on the motion data in a continuous sequence of motion frames to obtain the motion trajectory feature vector of the construction site; Based on a pre-set standard action feature library, template indexing is performed on the action trajectory feature vectors to obtain dangerous operational behaviors at the construction site. Spatial coordinate inversion is performed on dangerous operation behaviors to obtain the three-dimensional spatial coordinates of the start time, end time and trigger location of the dangerous operation behaviors, which constitute the second spatiotemporal positioning information of the construction site; Based on the identity identifier, the second spatiotemporal location information is associated and indexed, and the associated data after indexing is packaged into events to obtain the record of violations at the construction site.

[0038] The posture parameters and spatial position information of the operator corresponding to each frame in the continuous action frame sequence are extracted. The extracted posture parameters and spatial position information are continuously connected in time axis order. The motion trajectory of each joint of the operator is completely delineated and integrated into a unified feature representation form, forming a feature data set that can completely reflect the entire process of the operator's actions, and obtaining the action trajectory feature vector of the construction site.

[0039] The system retrieves a pre-defined standard action feature library, which stores trajectory feature templates corresponding to various standard and dangerous operation actions at the construction site. Each feature template is assigned a unique template index identifier. The action trajectory feature vector is compared and matched one by one with all feature templates in the standard action feature library. When the feature overlap reaches the pre-defined action matching threshold, the dangerous operation behavior category information corresponding to that feature template is extracted to obtain the dangerous operation behavior at the construction site.

[0040] The feature vector of the action trajectory corresponding to the identified dangerous operation behavior is analyzed in all dimensions. The first data node on the time axis of the vector is extracted as the start time of the dangerous operation behavior, and the last data node on the time axis is extracted as the end time of the dangerous operation behavior. At the same time, the spatial coordinate information corresponding to the key trigger position in the action trajectory is extracted and converted into three-dimensional spatial coordinates. The three-dimensional spatial coordinates of the start time, end time and trigger position are integrated to obtain the second spatiotemporal positioning information of the construction site.

[0041] Using the worker's identification as a search keyword, the second spatiotemporal location information is associated with an index, so that the spatiotemporal information of dangerous operation behavior is uniquely bound to the corresponding worker's identity. The bound identification, dangerous operation behavior category information and second spatiotemporal location information are uniformly encapsulated and processed. The data fields are filled and organized according to the preset event record format to obtain the record of violations at the construction site.

[0042] The beneficial effects are as follows: by fitting motion trajectories to continuous action frame sequences and indexing templates in a standard feature library, dangerous operational behaviors at construction sites can be accurately identified. Spatial coordinate inversion enables precise extraction of spatiotemporal information of dangerous operational behaviors, forming corresponding secondary spatiotemporal positioning information. This information is then mapped to the worker's identity for event attribution and event packaging, achieving precise binding between dangerous operational behaviors and involved personnel. The resulting violation record comprehensively includes personnel identity, dangerous operation type, and corresponding spatiotemporal positioning information, making the tracing of violations at construction sites more accurate and the recording more comprehensive. This provides accurate and complete violation data support for subsequent risk intervention, coupling analysis, and personnel safety management, improving the accuracy and effectiveness of behavior recognition and event recording in intelligent construction supervision.

[0043] S4. Based on the first and second spatiotemporal positioning information, a spatiotemporal coupling analysis is performed on the abnormal environmental state and dangerous operation behavior to obtain the linkage early warning information of the construction site. In this embodiment of the invention, the step of performing spatiotemporal coupling analysis on abnormal environmental states and dangerous operational behaviors based on first and second spatiotemporal positioning information to obtain coordinated early warning information for the construction site includes: Protocol parsing is performed on the first spatiotemporal positioning information to obtain the spatiotemporal anchor points of environmental events at the construction site; Trajectory delineation is performed on the second spatiotemporal positioning information to obtain spatiotemporal anchor points for behavioral events at the construction site; Based on the time stamp code of the spatiotemporal anchor points of environmental events, the time windows of the spatiotemporal anchor points of behavioral events are compared between intervals to obtain the time coupling criteria of the construction site. Based on the spatial coordinates of spatiotemporal anchor points of environmental events, a proximity search is performed on the spatial trajectory point series of spatiotemporal anchor points of behavioral events to obtain the spatial coupling criterion of the construction site. Based on time-coupled criteria and spatial-coupled criteria, risk accumulation is performed on spatiotemporal anchor points of environmental events and spatiotemporal anchor points of behavioral events to obtain primary early warning events at the construction site. Risk levels are determined for primary early warning events, and verification packages are generated based on preset early warning message templates to obtain coordinated early warning information for the construction site.

[0044] The method, based on time-coupled and spatial-coupled criteria, accumulates risks for spatiotemporal anchor points of environmental events and behavioral events to obtain primary early warning events at the construction site, including: The spatiotemporal correlation strength between environmental event spatiotemporal anchor points and behavioral event spatiotemporal anchor points is quantitatively characterized, and the calculation formula for the spatiotemporal correlation strength is as follows: ; In the formula, For spatiotemporal correlation strength, The time of occurrence of the abnormal environmental event within the spatiotemporal anchor point of the environmental event. This refers to the midpoint of the time window for dangerous operational behaviors within the spatiotemporal anchor point of the behavioral event. It is the shortest spatial distance from the hazardous operation behavior trajectory point in the spatiotemporal anchor point of the behavioral event to the location of the abnormal environmental event in the spatiotemporal anchor point of the environmental event. As a preset time tolerance, The preset space tolerance; When the spatiotemporal correlation strength is not less than the preset coupling activation threshold, abnormal environmental events and dangerous operational behaviors are fused to obtain primary early warning events at the construction site.

[0045] The first spatiotemporal positioning information is parsed in its entirety according to the protocol. The three-dimensional spatial coordinates and timestamp sequence are decomposed according to the preset data encapsulation protocol. The core time information and spatial location information of the abnormal environmental state are extracted and anchored. The marked time and spatial core information are integrated into an integrated environmental event identification data to obtain the spatiotemporal anchor point of the environmental event at the construction site.

[0046] The three-dimensional spatial coordinates of the start time, end time, and triggering position of dangerous operation behavior in the second spatiotemporal positioning information are completely extracted. The three-dimensional spatial coordinates of the triggering position are continuously delineated according to the chronological order. The core time window information and key spatial trajectory point information in the process of dangerous operation behavior are extracted and anchored. The marked time and spatial core information are integrated into integrated behavior event identification data to obtain the spatiotemporal anchor points of behavior events at the construction site.

[0047] The time stamp code stored in the preset fields of the spatiotemporal anchor point of the environmental event is extracted as the time reference. The start and end time nodes of the dangerous operation behavior time window corresponding to the spatiotemporal anchor point of the behavioral event are retrieved. The time interval of the time window is compared with the time stamp code of the environmental event one by one to determine the correlation between the behavioral event time window and the occurrence time of the environmental event and generate the corresponding time judgment result, thus obtaining the time coupling criterion of the construction site.

[0048] The spatial coordinates of the points bound in the spatiotemporal anchor points of the environmental event are extracted as the spatial reference. All spatial trajectory points of dangerous operation behaviors in the spatiotemporal anchor points of the behavioral event are retrieved. The proximity of each spatial coordinate in the trajectory point series with the spatial coordinates of the environmental event is searched. The criterion for the search is a preset spatial proximity threshold. Based on the search results, the spatial association judgment result between the behavioral event and the environmental event is generated, and the spatial coupling criterion of the construction site is obtained.

[0049] The system retrieves a preset risk accumulation rule, which is an event fusion standard based on the correlation judgment results of time coupling criteria and spatial coupling criteria. The judgment results of time coupling criteria and spatial coupling criteria are substituted into the rule for comprehensive judgment. When the judgment result reaches the preset risk accumulation trigger condition, the corresponding environmental event spatiotemporal anchor point and behavioral event spatiotemporal anchor point are fully fused to obtain the primary early warning event of the construction site.

[0050] The system retrieves a preset risk grading standard, which is a multi-level risk classification system based on construction safety management requirements and corresponding grading conditions. It then matches the spatiotemporal correlation characteristics of primary warning events with this standard one by one to determine the risk level corresponding to the primary warning event. Next, it retrieves a preset warning message template and fills in the data such as the risk level, spatiotemporal information, and event type of the primary warning event according to the template's field requirements and completes message verification. After the verification is passed, the message is packaged to obtain the linkage warning information of the construction site.

[0051] This method extracts the core temporal and spatial information of abnormal environmental events from the spatiotemporal anchor points of environmental events, and simultaneously extracts the shortest spatial distance information from the time window point and trajectory point of dangerous operation behavior to the location of the abnormal environment from the spatiotemporal anchor points of behavioral events. Combining preset time tolerance and spatial tolerance, it comprehensively quantifies the degree of deviation between environmental events and behavioral events in the time dimension and the degree of proximity in the spatial dimension, forming a single representational value that can intuitively reflect the closeness of the relationship between the two, thus completing the quantitative representation of the spatiotemporal correlation strength between the spatiotemporal anchor points of environmental events and the spatiotemporal anchor points of behavioral events.

[0052] The system retrieves the preset coupling activation threshold at the construction site. This threshold is a critical value of spatiotemporal correlation strength set according to the construction safety management level and risk prevention and control requirements. The spatiotemporal correlation strength value obtained by quantitative characterization is directly compared with this coupling activation threshold. When the spatiotemporal correlation strength value is not less than the preset coupling activation threshold, all information of the corresponding abnormal environmental event and all information of the dangerous operation behavior are completely integrated and combined. The core spatiotemporal information, event characteristics and correlation of the two types of events are retained to obtain the primary early warning event at the construction site.

[0053] The specific moment when an abnormal environmental event actually occurs is directly extracted from the spatiotemporal anchor points of the environmental event, serving as the core information representing the time dimension of the environmental event.

[0054] The start and end times of the time window corresponding to the dangerous operation behavior are extracted from the spatiotemporal anchor points of the behavioral event. By taking the midpoint between the two times, the midpoint of the time window of the dangerous operation behavior is obtained, which serves as the core reference information representing the time dimension of the behavioral event.

[0055] The coordinates of all spatial trajectory points throughout the entire process of dangerous operational behavior are extracted from the spatiotemporal anchor points of behavioral events. At the same time, the spatial location coordinates of abnormal environmental events are extracted from the spatiotemporal anchor points of environmental events. The distance between all trajectory point coordinates and the spatial location coordinates of environmental events is calculated, and the minimum value in the calculation results is selected as the core correlation information between behavioral events and environmental events in the spatial dimension.

[0056] Based on the safety management level and risk prevention and control requirements of the construction site, time tolerance and space tolerance are set in advance. These two pieces of information are fixed reference information determined in advance based on safety management specifications for the construction site, and are directly used as the basic reference for spatiotemporal correlation analysis.

[0057] By comprehensively quantifying the temporal deviation and spatial proximity of environmental events and behavioral events, a precise characterization of the spatiotemporal correlation strength between the two can be achieved. The quantification results can intuitively reflect the close correlation between abnormal environmental states and dangerous operational behaviors in the temporal and spatial dimensions.

[0058] When the characterization result reaches the preset coupling activation threshold at the construction site, it can be determined that there is a significant spatiotemporal correlation between abnormal environmental events and dangerous operational behaviors. This provides a clear and unique basis for the subsequent fusion of the two types of events into a primary warning event, enabling the risk accumulation judgment of environmental events and behavioral events to have a unified quantitative standard, and ensuring the objectivity and accuracy of the judgment results.

[0059] The beneficial effects are as follows: by analyzing and delineating the first and second spatiotemporal positioning information respectively, the core spatiotemporal anchor points of environmental and behavioral events can be accurately extracted. Based on time stamp codes and spatial point coordinates, the coupling criteria of time and space dimensions are determined. Then, through quantitative representation, the spatiotemporal correlation strength of environmental and behavioral events is accurately calculated. Combined with preset coupling activation thresholds, risk accumulation and event fusion are completed to form primary early warning events. Finally, risk classification is carried out on the primary early warning events, and the packages are verified according to preset templates. This achieves precise spatiotemporal coupling analysis of abnormal environmental states and dangerous operational behaviors. The resulting linked early warning information fully includes core content such as risk level, spatiotemporal correlation characteristics, and event type, enabling safety early warnings at construction sites to possess spatiotemporal correlation and risk hierarchy, significantly improving the accuracy and practicality of early warning information. This provides a scientific and comprehensive early warning basis for subsequent graded intervention operations, enhancing the pertinence and effectiveness of intelligent construction supervision from the early warning level.

[0060] S5. Based on violation records and linkage early warning information, the graded intervention instructions at the construction site are coded and delivered to obtain intervention feedback data at the construction site. In this embodiment of the invention, the step of encoding and delivering tiered intervention instructions for the construction site based on violation records and linked early warning information to obtain intervention feedback data from the construction site includes: Risk level assessment is performed on records of violations to determine the first intervention priority indicator for the construction site. By analyzing the event type of the linked early warning information, the second intervention priority identifier of the construction site is obtained; Arbitrate the first intervention priority identifier and the second intervention priority identifier to obtain the intervention instruction header at the construction site; Based on the identification of the personnel involved and the location of the incident in the violation record, the target address field and location field of the intervention command header are loaded to obtain the intervention command frame at the construction site; The intervention command frame is encapsulated and then delivered in a targeted manner to obtain intervention feedback data from the construction site.

[0061] The pre-set risk level assessment criteria for violations at the construction site are retrieved. These criteria, based on construction safety management regulations, classify different types of dangerous operations into fixed risk levels and assign unique priority identifiers. The types of dangerous operations, locations, and scope of impact in the violation records are analyzed in all dimensions. The analysis results are then precisely matched with the pre-set risk level assessment criteria for violations, and the priority identifiers corresponding to the matching results are extracted to obtain the first intervention priority identifier for the construction site.

[0062] The pre-set grading standard for early warning event types at the construction site is retrieved. This standard classifies different types of linked early warning events into fixed handling levels and assigns unique priority identifiers based on construction safety management requirements. The risk level, event type, and spatiotemporal correlation characteristics in the linked early warning information are analyzed in all dimensions. The analysis results are accurately matched with the pre-set grading standard for early warning event types, and the priority identifiers corresponding to the matching results are extracted to obtain the second intervention priority identifier for the construction site.

[0063] The pre-set intervention priority arbitration rules at the construction site are retrieved. These rules set different arbitration weights for priority indicators based on the urgency of handling construction safety risks. The first intervention priority indicator and the second intervention priority indicator are substituted into the rules for weight comparison. The priority indicator with the higher weight in the comparison result is selected as the core intervention priority. The core intervention priority and the corresponding handling requirements are integrated and encapsulated to obtain the intervention instruction header at the construction site.

[0064] Extract the identification of the person involved and the three-dimensional spatial coordinates of the incident location from the violation record. Retrieve the field format requirements of the preset target address field and location field in the intervention command header. Fill the identification of the person involved into the target address field of the intervention command header according to the format requirements. Convert the three-dimensional spatial coordinates of the incident location into on-site identifiable location information according to the format requirements and fill it into the location field of the intervention command header. Complete the information loading and format verification of each field of the intervention command header to obtain the intervention command frame of the construction site.

[0065] The pre-defined intervention command message encapsulation specification at the construction site is retrieved. This specification clarifies the unified requirements for field sorting, check code generation, and data encapsulation format of the intervention command frame. The intervention command frame is sorted and organized according to this specification, and the corresponding check code is generated and appended to the end of the command frame to complete the standardized message encapsulation of the intervention command frame. Based on the target address field and location field information in the encapsulated message, the message is sent to the corresponding control terminal and on-site operation terminal at the construction site. The command reception status and execution feedback information of each receiving terminal are received and recorded to obtain the intervention feedback data at the construction site.

[0066] The beneficial effects are as follows: by conducting risk level assessment and event type analysis on violation records and linkage early warning information respectively, the intervention priority identifiers corresponding to the two types of information can be accurately obtained. Based on the level arbitration, the core intervention and handling priority is determined and an intervention instruction header is formed. Then, by combining the personnel and location information in the violation record, the fields of the instruction header are loaded to form a standardized intervention instruction frame. After message encapsulation and targeted delivery, intervention feedback data is obtained. This realizes the accurate coding and targeted delivery of graded intervention instructions at the construction site, making the generation of intervention instructions fit the actual risk situation on site, and the delivery has a clear target orientation. At the same time, the execution status of instructions can be grasped through feedback data, which improves the timeliness, pertinence and traceability of construction safety risk intervention, and makes the risk handling operation at the construction site form a closed-loop management, effectively enhancing the execution efficiency and handling effect of intelligent construction supervision.

[0067] S6. Clean and archive records of violations, linked early warning information, and intervention feedback data to build individual safety record files for construction sites.

[0068] In this embodiment of the invention, the step of cleaning and archiving violation records, linked early warning information, and intervention feedback data to construct an individual safety record file at the construction site includes: By collecting and summarizing records of violations, linked early warning information, and intervention feedback data, the original event pool of the construction site is obtained; Perform field integrity checks on the original event pool, and reformat the data that passes the checks to obtain a standardized event record set for the construction site; The standardized event record set is deduplicated and compared, and the merged event records are correlated and mapped with the intervention feedback data to obtain event intervention correlation pairs at the construction site; Based on the identification of the personnel involved in the event intervention association pair, the event records of the operators are classified and aggregated to obtain the individual event sequence of the operators; Key fields are extracted from event records in individual event sequences, and the extracted key fields are arranged into documents according to a preset file template to obtain individual safety record files for construction sites.

[0069] All records of violations, alerts, and intervention feedback generated at the construction site throughout the entire time period are collected. The three types of data are then uniformly aggregated according to the order of their generation. All collected data are integrated into the same data storage area and preliminarily classified and labeled to obtain the original event pool of the construction site.

[0070] The pre-defined construction safety supervision data field integrity verification standard is retrieved. This standard clarifies the mandatory and optional fields that must be included in violation records, linkage early warning information, and intervention feedback data. Each data entry in the original event pool is checked field by field according to this standard. Data entries with missing mandatory fields are removed. All data entries that pass the verification are sorted, data types are converted, and content is standardized according to the pre-defined unified data format requirements to obtain a standardized event record set of the construction site.

[0071] Content features are extracted from all data entries in the standardized event log set. Duplicate data entries containing the same person involved, the same time of the incident, and the same location of the incident are compared one by one. The single entry with the most complete information after comparison is retained and the remaining duplicate entries are deleted. All event records after deduplication and merging are bound one-to-one with the corresponding intervention feedback data according to the unique event identifier, so as to achieve accurate association between event records and handling feedback information and obtain event intervention association pairs at the construction site.

[0072] Extract the identity identifier of the involved personnel bound to each data in the event intervention association pair, classify all event intervention association pairs corresponding to the same identity identifier, and arrange the classified event intervention association pairs in order of the chronological order of the events to form a complete event handling record sequence with a single operator as the main body, thus obtaining the individual event sequence of the operator.

[0073] Key fields are extracted from each record in the individual event sequence of the workers. The extracted key fields include information about the personnel involved, the time and space information of the event, the event type, the risk level, the intervention and handling measures, and the feedback results. The pre-set individual safety record file template of the construction site is retrieved. The template clarifies the content structure, field layout and document format requirements of the file. The extracted key fields are filled into the corresponding positions in accordance with the template requirements and the document format is standardized to obtain the individual safety record file of the construction site.

[0074] The beneficial effects are as follows: by fully collecting and summarizing violation records, linked early warning information, and intervention feedback data to form a raw event pool, and then standardizing the data through field integrity verification and format reorganization, the event records and intervention feedback data are accurately bound by deduplication comparison and association mapping. Based on the identity of the involved personnel, the relevant events of the workers are classified and aggregated. Finally, key fields are extracted and arranged according to preset templates to build individual safety history files. This achieves systematic cleaning, archiving, and refined management of construction site supervision data. The individual safety history files fully cover the entire process of workers' violations, related early warnings, and intervention and handling information. This provides comprehensive and accurate archival data support for personnel safety assessment, risk tracing, and targeted safety training at construction sites. At the same time, it promotes the closed-loop utilization of construction safety supervision data and improves the refinement and systematization level of intelligent construction supervision.

[0075] like Figure 2 The diagram shown is a functional block diagram of a construction intelligent supervision system based on multimodal perception provided in an embodiment of the present invention.

[0076] The construction intelligent monitoring system 100 based on multimodal perception described in this invention can be installed in an electronic device. Depending on the functions implemented, the construction intelligent monitoring system 100 based on multimodal perception may include a data decoupling processing module 101, a spatiotemporal alignment determination module 102, a behavior matching and attribution module 103, a coupling analysis and early warning module 104, an instruction encoding and delivery module 105, and an archive construction and archiving module 106. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, stored in the memory of the electronic device.

[0077] In this embodiment, the functions of each module / unit are as follows: The data decoupling processing module 101 is used to decouple the features of the original sensing data of the construction site to obtain the positioning data stream, image frame sequence and environmental monitoring data set of the construction site. The spatiotemporal alignment determination module 102 is used to perform spatiotemporal alignment of the position and posture of workers in the construction site based on the positioning data stream and image frame sequence, to obtain the worker's identity and continuous action frame sequence, and to determine the safety threshold of abnormal environmental conditions in the construction site based on the environmental monitoring data set, so as to obtain the first spatiotemporal positioning information of the construction site. The behavior matching and attribution module 103 is used to perform feature matching on a continuous action frame sequence based on a preset standard action feature library, and to perform event attribution mapping between the second spatiotemporal location information of the identified dangerous operation behavior and the identity identifier to obtain the record of the violation behavior at the construction site. The coupling analysis and early warning module 104 is used to perform spatiotemporal coupling analysis on abnormal environmental conditions and dangerous operation behaviors based on the first spatiotemporal positioning information and the second spatiotemporal positioning information to obtain linkage early warning information of the construction site. The instruction encoding and delivery module 105 is used to encode and deliver graded intervention instructions at the construction site based on violation records and linkage early warning information, so as to obtain intervention feedback data at the construction site. The archive construction module 106 is used to clean and archive records of violations, linkage early warning information and intervention feedback data to build individual safety history archives at the construction site.

[0078] In the several embodiments provided by this invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0079] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0080] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0081] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0082] This application embodiment can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A construction intelligent supervision method based on multimodal perception, characterized in that, The method includes: S1. Decouple the original sensing data of the construction site by feature decoupling to obtain the location data stream, image frame sequence and environmental monitoring data set of the construction site; S2. Based on the positioning data stream and image frame sequence, the position and posture of the workers in the construction site are spatiotemporally aligned to obtain the workers' identification and continuous action frame sequence. Based on the environmental monitoring data set, the safety threshold of the abnormal environmental state of the construction site is determined to obtain the first spatiotemporal positioning information of the construction site. S3. Based on the preset standard action feature library, perform feature matching on the continuous action frame sequence, and perform event attribution mapping between the second spatiotemporal positioning information of the identified dangerous operation behavior and the identity identifier to obtain the record of the violation behavior at the construction site. S4. Based on the first and second spatiotemporal positioning information, a spatiotemporal coupling analysis is performed on the abnormal environmental state and dangerous operation behavior to obtain the linkage early warning information of the construction site. S5. Based on violation records and linkage early warning information, the graded intervention instructions at the construction site are coded and delivered to obtain intervention feedback data at the construction site. S6. Clean and archive records of violations, linked early warning information, and intervention feedback data to build individual safety record files for construction sites.

2. The intelligent construction supervision method based on multimodal perception as described in claim 1, characterized in that, The process of decoupling the original sensing data from the construction site to obtain the location data stream, image frame sequence, and environmental monitoring data set of the construction site includes: The positioning pulse signal at the construction site is demodulated by time difference to obtain the positioning data stream at the construction site; Motion contour extraction is performed on the real-time video stream of the construction site to obtain the image frame sequence of the construction site; The dimensions of the analog signals at the construction site are restored to obtain the environmental monitoring data set of the construction site.

3. The intelligent construction supervision method based on multimodal perception as described in claim 1, characterized in that, The method involves spatiotemporally aligning the position and posture of workers at the construction site based on positioning data streams and image frame sequences to obtain worker identification and continuous action frame sequences, including: The location beacons in the location data stream are timestamped to obtain the set of location points of the workers at the construction site; Joint spatial coordinate detection is performed on the image frame sequence to obtain the posture parameter set of the workers at the construction site; Based on the time tags of the worker location point set, the worker posture parameter set is time-axis registered to obtain the position and posture association pairs at the construction site. By interpolating the position and attitude correlation pairs, a continuous sequence of action frames at the construction site is obtained. The facial region of the first frame in a continuous action frame sequence is feature-encoded, and the encoded facial feature vector is matched with a pre-stored personnel information database to obtain the identity identifier of the construction site.

4. The intelligent construction supervision method based on multimodal perception as described in claim 1, characterized in that, The method of determining safety thresholds for abnormal environmental conditions at the construction site based on environmental monitoring data sets to obtain the first spatiotemporal location information of the construction site includes: The threshold intervals of the monitoring values ​​in the environmental monitoring data set are traversed to obtain the abnormal trigger values ​​at the construction site; By attributing and tracing the abnormal trigger values, the physical addresses and timestamp sequences of the sensors at the construction site are obtained; Based on the physical addresses of the sensors, spatial coordinate mapping is performed on the sensor deployment point map of the construction site to obtain the three-dimensional spatial coordinates of the construction site location. By associating and encapsulating three-dimensional spatial coordinates with timestamp sequences, the first spatiotemporal positioning information of the construction site is obtained.

5. The intelligent construction supervision method based on multimodal perception as described in claim 1, characterized in that, The system, based on a preset standard action feature library, performs feature matching on continuous action frame sequences and maps the second spatiotemporal location information of identified dangerous operational behaviors with identity identifiers to obtain records of violations at the construction site, including: Trajectory fitting is performed on the motion data in a continuous sequence of motion frames to obtain the motion trajectory feature vector of the construction site; Based on a pre-set standard action feature library, template indexing is performed on the action trajectory feature vectors to obtain dangerous operational behaviors at the construction site. Spatial coordinate inversion is performed on dangerous operation behaviors to obtain the three-dimensional spatial coordinates of the start time, end time and trigger location of the dangerous operation behaviors, which constitute the second spatiotemporal positioning information of the construction site; Based on the identity identifier, the second spatiotemporal location information is associated and indexed, and the associated data after indexing is packaged into events to obtain the record of violations at the construction site.

6. The intelligent construction supervision method based on multimodal perception as described in claim 1, characterized in that, Based on the first and second spatiotemporal positioning information, a spatiotemporal coupling analysis of abnormal environmental conditions and dangerous operational behaviors is performed to obtain coordinated early warning information for the construction site, including: Protocol parsing is performed on the first spatiotemporal positioning information to obtain the spatiotemporal anchor points of environmental events at the construction site; Trajectory delineation is performed on the second spatiotemporal positioning information to obtain spatiotemporal anchor points for behavioral events at the construction site; Based on the time stamp code of the spatiotemporal anchor points of environmental events, the time windows of the spatiotemporal anchor points of behavioral events are compared between intervals to obtain the time coupling criteria of the construction site. Based on the spatial coordinates of spatiotemporal anchor points of environmental events, a proximity search is performed on the spatial trajectory point series of spatiotemporal anchor points of behavioral events to obtain the spatial coupling criterion of the construction site. Based on time-coupled criteria and spatial-coupled criteria, risk accumulation is performed on spatiotemporal anchor points of environmental events and spatiotemporal anchor points of behavioral events to obtain primary early warning events at the construction site. Risk levels are determined for primary early warning events, and verification packages are generated based on preset early warning message templates to obtain coordinated early warning information for the construction site.

7. The intelligent construction supervision method based on multimodal perception as described in claim 6, characterized in that, The method, based on time-coupled and spatial-coupled criteria, accumulates risks for spatiotemporal anchor points of environmental events and behavioral events to obtain primary early warning events at the construction site, including: The spatiotemporal correlation strength between environmental event spatiotemporal anchor points and behavioral event spatiotemporal anchor points is quantitatively characterized, and the calculation formula for the spatiotemporal correlation strength is as follows: ； In the formula, For spatiotemporal correlation strength, The time of occurrence of the abnormal environmental event within the spatiotemporal anchor point of the environmental event. This refers to the midpoint of the time window for dangerous operational behaviors within the spatiotemporal anchor point of the behavioral event. It is the shortest spatial distance from the hazardous operation behavior trajectory point in the spatiotemporal anchor point of the behavioral event to the location of the abnormal environmental event in the spatiotemporal anchor point of the environmental event. As a preset time tolerance, The preset space tolerance; When the spatiotemporal correlation strength is not less than the preset coupling activation threshold, abnormal environmental events and dangerous operational behaviors are fused to obtain primary early warning events at the construction site.

8. The intelligent construction supervision method based on multimodal perception as described in claim 1, characterized in that, The method of encoding and delivering tiered intervention instructions at the construction site based on violation records and linked early warning information to obtain intervention feedback data at the construction site includes: Risk level assessment is performed on records of violations to determine the first intervention priority indicator for the construction site. By analyzing the event type of the linked early warning information, the second intervention priority identifier of the construction site is obtained; Arbitrate the first intervention priority identifier and the second intervention priority identifier to obtain the intervention instruction header at the construction site; Based on the identification of the personnel involved and the location of the incident in the violation record, the target address field and location field of the intervention command header are loaded to obtain the intervention command frame at the construction site; The intervention command frame is encapsulated and then delivered in a targeted manner to obtain intervention feedback data from the construction site.

9. The intelligent construction supervision method based on multimodal perception as described in claim 1, characterized in that, The process involves cleaning and archiving records of violations, linked early warning information, and intervention feedback data to construct individual safety record files for construction sites, including: By collecting and summarizing records of violations, linked early warning information, and intervention feedback data, the original event pool of the construction site is obtained; Perform field integrity checks on the original event pool, and reformat the data that passes the checks to obtain a standardized event record set for the construction site; The standardized event record set is deduplicated and compared, and the merged event records are correlated and mapped with the intervention feedback data to obtain event intervention correlation pairs at the construction site; Based on the identity identifiers of the personnel involved in the event intervention association pair, the event records of the operators are classified and aggregated to obtain the individual event sequence of the operators; Key fields are extracted from event records in individual event sequences, and the extracted key fields are arranged into documents according to a preset file template to obtain individual safety record files for construction sites.

10. A construction intelligent monitoring system based on multimodal perception, characterized in that, The system is used to implement the intelligent construction monitoring method based on multimodal perception as described in claim 1, the system comprising: The data decoupling processing module is used to decouple the features of the raw sensing data at the construction site to obtain the location data stream, image frame sequence and environmental monitoring data set of the construction site. The spatiotemporal alignment determination module is used to perform spatiotemporal alignment of the position and posture of workers in the construction site based on the positioning data stream and image frame sequence, to obtain the worker's identity and continuous action frame sequence, and to determine the safety threshold of abnormal environmental conditions at the construction site based on the environmental monitoring data set, so as to obtain the first spatiotemporal positioning information of the construction site. The behavior matching and attribution module is used to perform feature matching on a continuous action frame sequence based on a preset standard action feature library, and to perform event attribution mapping between the second spatiotemporal location information of the identified dangerous operation behavior and the identity identifier to obtain the record of violations at the construction site. The coupling analysis and early warning module is used to perform spatiotemporal coupling analysis on abnormal environmental conditions and dangerous operational behaviors based on the first and second spatiotemporal positioning information to obtain linkage early warning information for the construction site. The instruction coding and delivery module is used to encode and deliver graded intervention instructions at the construction site based on violation records and linkage early warning information, so as to obtain intervention feedback data at the construction site. The archive building module is used to clean and archive records of violations, linkage early warning information, and intervention feedback data to build individual safety history archives at the construction site.

Images