A three-dimensional visualization safety monitoring method for a wind power hoisting site

Abstract

The application provides a three-dimensional visualization safety monitoring method for a wind power hoisting site, and relates to the technical field of safety monitoring, and comprises the following steps: constructing a three-dimensional construction map based on real-time monitoring of the wind power hoisting site; interacting with hoisting construction progress, positioning a rule violation identification area; presetting a video time domain, cutting and exporting a local video sequence; preconstructing a ground rule violation analysis model, identifying a ground rule violation behavior, and obtaining a first risk behavior set; extracting a hoisted component to be obtained, and determining a real-time construction equipment group; interacting with K operation monitoring video streams, identifying an operation rule violation behavior, and obtaining a second risk behavior set; performing visualization identification, and generating a safety visualization map. The application solves the technical problem that, in the prior art, safety monitoring data of the wind power hoisting site is often presented in a non-intuitive form, it is difficult to quickly and intuitively determine a safety risk position and degree, and the safety management level of a construction site is poor.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of safety monitoring, in particular to a three-dimensional visual safety monitoring method for a wind power hoisting site. BACKGROUND

[0002] The hoisting of a wind power generator needs to be performed in the air and involves the coordinated operation of multiple construction devices and personnel. Current technical means usually rely on manual supervision and traditional monitoring devices, which have some technical problems. Specifically, during the hoisting process, the illegal operation of construction personnel and devices can cause serious safety accidents. The traditional monitoring system is difficult to achieve comprehensive coverage of the entire construction area, is prone to monitoring blind spots, and the safety monitoring data of the hoisting site is often presented in a non-intuitive form, making it difficult to quickly and intuitively determine the location and degree of safety risks, resulting in poor targeting of construction safety monitoring, slow safety risk investigation, and thus poor safety management level of the construction site. SUMMARY

[0003] The present application provides a three-dimensional visual safety monitoring method for a wind power hoisting site, aiming to solve the technical problem that the safety monitoring data of the wind power hoisting site in the prior art is often presented in a non-intuitive form, making it difficult to quickly and intuitively determine the location and degree of safety risks, resulting in a poor safety management level of the construction site.

[0004] The three-dimensional visual safety monitoring method for a wind power hoisting site disclosed in the present application comprises the following steps: constructing a three-dimensional construction map based on real-time monitoring of the wind power hoisting site; interacting with the hoisting construction progress and positioning the illegal identification area in the three-dimensional construction map according to the hoisting construction progress; presetting a video time domain, and cutting and deriving a local video sequence from the three-dimensional construction map with the illegal identification area and the video time domain as constraints; pre-building a ground illegal analysis model, identifying ground illegal behaviors by synchronizing the local video sequence to the ground illegal analysis model, and obtaining a first risk behavior set; extracting the hoisting component based on the hoisting construction progress, determining the real-time construction device group based on the hoisting component, wherein the real-time construction device group operates in the illegal identification area, and the real-time construction device group comprises K real-time construction devices; interacting with K operation monitoring video streams in the operation room of the K real-time construction devices, identifying operation illegal behaviors based on the K operation monitoring video streams, and obtaining a second risk behavior set; visualizing and marking the first risk behavior set and the second risk behavior set on the three-dimensional construction map to generate a safety visualization map.

[0005] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0006] By combining real-time monitoring data with three-dimensional modeling, a three-dimensional construction map is constructed, realizing full-range, non-blind area monitoring of the hoisting site, improving the monitoring coverage range, reducing monitoring dead angles, and improving the comprehensiveness of on-site safety management; according to the hoisting construction progress, the illegal identification area is positioned, and the local video sequence is real-time cropped and exported for illegal behavior identification, through synchronous video to ground illegal analysis model, the risk behavior can be identified and marked in time, so as to reduce the potential safety accidents caused by illegal behavior not found in time; by obtaining the operation monitoring video stream of the construction equipment, and based on the operation illegal identification model for real-time analysis, the illegal behavior in the operation of the construction equipment can be effectively identified, further improving the overall safety of the construction site; by integrating the risk behavior into the construction three-dimensional map, a safety visualization map is generated, which not only helps the management personnel to intuitively understand the construction progress and risk distribution, but also clearly shows the specific location and scene of the illegal behavior, improving the decision-making efficiency and accuracy. In summary, the method significantly improves the safety monitoring capability of the wind power hoisting site through comprehensive, real-time monitoring, timely illegal behavior identification, and intuitive three-dimensional visualization display, providing strong technical support for the safety management of the construction site.

[0007] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the specific embodiments of the present application can be implemented in accordance with the content of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 A three-dimensional visualization safety monitoring method flow chart for a wind power hoisting site is provided for the embodiments of the present application.

[0009] Figure 2 A three-dimensional visualization safety monitoring method flow chart for a wind power hoisting site is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0010] The embodiments of the present application provide a three-dimensional visualization safety monitoring method for a wind power hoisting site, which solves the technical problem that the safety monitoring data of the wind power hoisting site in the prior art is often presented in a non-intuitive form, making it difficult to quickly and intuitively determine the safety risk position and degree, resulting in poor safety management level of the construction site.

[0011] After introducing the basic principles of the present application, the various non-limiting embodiments of the present application will be specifically introduced in combination with the drawings of the specification.

[0012] As Figure 1As shown, the embodiment of the present application provides a three-dimensional visualization safety monitoring method for wind power hoisting site, the method comprises:

[0013] Based on the real-time monitoring of the wind power hoisting site, a construction three-dimensional map is constructed.

[0014] By measuring the boundary data of the construction area with measuring instruments, the construction boundary of the wind power hoisting site is determined as the basis for constructing the construction three-dimensional map. According to the construction boundary, data collection is carried out to obtain hoisting site construction information. Based on the hoisting site construction information, three-dimensional modeling is carried out to generate a hoisting site twin model. This model contains all key elements of the construction site, such as hoisting equipment, wind turbine components, topography, etc. Through the monitoring camera array arranged in the wind power hoisting site, video stream data is collected in real time, the texture data in the video stream is mapped to the hoisting site twin model, the realism of the model is enhanced, and it is more intuitive. Obtain the construction three-dimensional map to provide basic support for subsequent construction progress positioning, illegal behavior identification and risk visualization.

[0015] Interact with the hoisting construction progress, and locate the illegal identification area in the construction three-dimensional map according to the hoisting construction progress.

[0016] Real-time hoisting construction progress data is obtained from the construction management system, including the working state of the hoisting equipment, the completion percentage of the current hoisting task, and the estimated completion time. The obtained construction progress data is associated with the construction three-dimensional map. Each progress node, such as a certain component of a wind turbine has been hoisted to a specified position, is positioned and labeled on the three-dimensional map, which can be realized by coordinate system conversion or direct mapping method. Combined with the construction progress and known construction risks, high-risk areas where illegal behavior may occur are identified, such as hoisting equipment overload and personnel entering dangerous areas during hoisting operations. In the three-dimensional map, the identified high-risk areas are marked as illegal identification areas, which will be monitored to ensure that possible illegal behavior can be discovered and handled in time during the actual construction process.

[0017] A video time domain is preset, and a local video sequence is derived from the construction three-dimensional map by cutting and deriving the illegal identification area and the video time domain as constraints.

[0018] The video time domain refers to a time span range, that is, the time period of the construction process that needs to be analyzed. The video data in this time period will be analyzed to identify illegal behavior. The time domain can be a fixed time period set according to the construction task duration, equipment working cycle, etc., such as 10 seconds, 30 seconds, or can be dynamically adjusted according to the specific requirements of the construction task.

[0019] In the positioned violation identification area, combined with the set video time domain, the time period and the area range that need to be monitored are determined, and the corresponding local video sequence is cropped from the construction three-dimensional map and the monitoring video, which reflects the construction process in a specific time period and a specific area.

[0020] A pre-constructed ground violation analysis model is used to identify ground violation behaviors by synchronizing the local video sequence to the ground violation analysis model, and a first risk behavior set is obtained.

[0021] A large amount of historical video data is collected, including labeled violation behavior videos and normal operation videos. The sample data covers various possible ground violation behaviors, such as personnel not wearing safety equipment, misoperation of equipment, and walking without following the specified route. According to the characteristics of the sample data, multiple static risk identification branches are constructed for identifying specific moment violation behaviors, such as improper equipment position, and multiple dynamic risk identification branches are constructed for identifying behaviors within a period of time, such as non-compliance with the operation process. The multiple static risk identification branches and the multiple dynamic risk identification branches are integrated to complete the construction of the ground violation analysis model.

[0022] The cropped local video sequence is video-stripped, and the video including the behaviors of multiple persons is split into multiple behavior videos of multiple persons. The obtained multiple behavior videos are synchronized to the ground violation analysis model, key frames are extracted from the video sequence, and feature information is extracted therefrom. Through multiple static risk identification branches, static picture violation identification is performed to obtain a static violation behavior set. Through multiple dynamic risk identification branches, dynamic picture violation identification is performed on the multiple behavior videos to obtain a dynamic violation behavior set. The static violation behavior set and the dynamic violation behavior set are integrated to obtain the first risk behavior set.

[0023] Based on the hoisting construction progress, a to-be-hoisted component is extracted, and a real-time construction equipment group is determined based on the to-be-hoisted component, wherein the real-time construction equipment group operates in the violation identification area, and the real-time construction equipment group includes K real-time construction equipment.

[0024] From the construction progress, the specific components currently required to be hoisted, such as the blades, tower, generator, etc. of the wind turbine, are parsed, the hoisted components are obtained, the basic information about the components is obtained from the project database, including the basic characteristics of the components, the transportation path, the hoisting requirements, etc., and according to the component basic information, it is determined which construction equipment is required to complete the hoisting task. According to the real-time state of the equipment in the illegal identification area, such as whether it is in an idle state, whether it is performing other tasks, K real-time construction equipment is dispatched to form a real-time construction equipment group, K represents the number of construction equipment in the real-time construction equipment group, which is a positive integer, which is determined according to the task complexity, equipment characteristics, construction area size and other factors. The construction equipment group usually includes cranes, hoists, lifting tools, balancing equipment, auxiliary vehicles and other equipment, which work together to complete a specific hoisting task.

[0025] Interactively obtain K operation monitoring video streams in the operation rooms of the K real-time construction equipment, and perform operation violation behavior identification based on the K operation monitoring video streams to obtain a second risk behavior set.

[0026] A camera is configured in the operation room of each real-time construction equipment to capture the behavior of the operator and the environment in the operation room, and K operation monitoring video streams in the operation rooms of the K real-time construction equipment are obtained through camera acquisition.

[0027] A sample operation violation video set of a plurality of historical operation violation behaviors, such as excessive manipulation, dangerous operation, and neglect of safety regulations, is obtained, an operation violation identification model is trained using historical video data, which can identify a plurality of possible violation behaviors and adapt to different equipment and operating environments, the trained operation violation identification model is deployed to the local server of the K real-time construction equipment, and the K operation monitoring video streams are analyzed to identify possible violation behaviors, including violation of safety operation procedures, operation errors, and fatigue driving, etc. The identified violation behaviors are marked on the corresponding video frames to generate a second risk behavior set.

[0028] The first risk behavior set and the second risk behavior set are visually identified on the construction three-dimensional map to generate a safety visualization map.

[0029] The first risk behavior set includes the illegal behavior data in the ground operation of the construction site identified by the ground illegal behavior identification model, and the second risk behavior set includes the illegal behavior data in the real-time construction equipment operation identified by the operation illegal behavior identification model. The data of the first risk behavior set and the second risk behavior set are arranged into a format suitable for three-dimensional map visualization, associated with the corresponding positions in the construction three-dimensional map, the coordinate information of each risk behavior is mapped to the specific positions in the three-dimensional map, a risk identification layer is superimposed on the three-dimensional map, and the identification points or areas of the risk behaviors are displayed on the three-dimensional model. For high-risk areas, an enlarged image or a detailed view is designed so that the user can more clearly see the specific risk situation. The construction three-dimensional map and the risk identification layer are synthesized to generate a final safety visualization map. The visualization map provides intuitive data support for risk assessment and helps managers make more accurate safety decisions and take measures.

[0030] Further, as shown in Figure 2 Further, as shown in

[0031] The construction boundary of the wind power hoisting site is interactively obtained, and hoisting site construction information is obtained by data collection according to the construction boundary. Based on the hoisting site construction information, a three-dimensional model is established based on the constraint of the construction boundary to obtain a hoisting site twin model. The real-time video stream array obtained by the monitoring camera array pre-configured in the wind power hoisting site is synchronized to the hoisting site twin model to obtain a construction three-dimensional map.

[0032] The boundary data of the construction area is measured by a measuring instrument to determine the construction boundary of the wind power hoisting site. Data collection is performed within the construction boundary, including site facility collection such as buildings, infrastructure and other structures, construction planning collection including construction planning, equipment placement position, and environmental data collection including terrain, weather conditions, soil type, etc. The collected information is arranged to obtain hoisting site construction information.

[0033] An appropriate three-dimensional modeling tool such as AutoCAD is selected for modeling. The construction boundary data and construction information are imported into the modeling software as the basis data for modeling. Specifically, the outer contour of the construction area is drawn according to the construction boundary data to establish a three-dimensional boundary model of the construction area. The buildings, equipment, roads and other facilities of the site are added to the three-dimensional boundary model according to the construction information. Finally, model details such as equipment position, construction area, material storage, etc. are added to generate a hoisting site twin model. The twin model is a digital simulation model of the construction site for further analysis.

[0034] Obtain the pre-configured monitoring camera array at the wind power hoisting site, which continuously captures the construction dynamics of the site, including equipment operation, personnel activities, etc. The real-time video streams of each camera are aggregated to form a real-time video stream array, which contains real-time video information of each area of the hoisting site.

[0035] Synchronize the video stream with the time and spatial coordinates of the twin model, so that each video stream can be displayed at the correct location and time point in the twin model. After synchronization, each video stream is embedded into the corresponding monitoring camera position and angle in the twin model, so that the content in the video stream is consistent with the scene of the twin model, and the real-time situation of the site can be dynamically displayed. Obtain the construction three-dimensional map, which not only shows the three-dimensional layout of the site, but also can display the construction dynamics of each monitoring area in real time.

[0036] Further, synchronize the real-time video stream array obtained by the pre-configured monitoring camera array at the wind power hoisting site to the hoisting site twin model to obtain the construction three-dimensional map. The method further comprises:

[0037] Interactively obtain the original monitoring layout of the wind power hoisting site, and call the historical video streams of multiple initial cameras in the original monitoring layout. Fit the multiple historical video streams to the hoisting site twin model to obtain N monitoring blind areas. Based on the hoisting site construction information, call the N construction height information of the N monitoring blind areas. According to the N construction height information, perform monitoring coverage analysis on the N monitoring blind areas to locate N new monitoring sites. Based on the N new monitoring sites, update the original monitoring layout to obtain the monitoring camera array.

[0038] By interacting with the monitoring system of the wind power hoisting site, obtain the original monitoring layout information of the site, including the positions, installation angles, coverage ranges, etc. of all initial cameras. According to the original monitoring layout, obtain multiple initial cameras and call the historical video streams of these cameras, which record the site situation in the past time period.

[0039] Convert the spatial information in the video frame to the coordinates in the twin model, and match it with the three-dimensional structure of the twin model to determine the position and angle of each video frame in the twin model. In the fitting process, identify the areas that are not covered by video, which are the monitoring blind areas. According to the fitting result, identify N monitoring blind areas of the hoisting site, where N is the number of monitoring blind areas and is a positive integer. These blind areas are areas in the twin model that cannot be effectively monitored by historical video streams.

[0040] Call the collected hoisting site construction information, especially the height data related to the construction area, including the construction height of buildings, equipment, structures, etc., correlate the obtained N monitoring blind areas with their corresponding construction height information, determine the vertical height range of each blind area in the twin model, and obtain N construction height information.

[0041] Perform monitoring coverage analysis. Specifically, according to the construction height information of the blind area, simulate the coverage effect after installing the monitoring device at different heights or positions, consider the installation height, angle of the device, and the interference of adjacent devices, find the best monitoring point, evaluate whether full coverage of the blind area can be achieved by calculating the monitoring range of each proposed new device, ensure that all blind areas can be completely monitored by at least one device, determine the best monitoring device layout position for each blind area according to the monitoring coverage analysis result, and obtain N new monitoring sites. These new monitoring sites can make up for the shortcomings of the existing monitoring system and achieve full coverage.

[0042] Add the N new monitoring sites positioned to the original monitoring layout, fine-tune the original monitoring devices based on the device layout of the new sites, such as angle adjustment and position fine-tuning, to avoid mutual interference between devices and optimize monitoring coverage effect. Integrate all updated monitoring devices to generate the final monitoring camera array, which covers all original blind areas and improves the overall monitoring capability and safety of the wind power hoisting site.

[0043] Further, a pre-constructed ground violation analysis model is constructed, and the method further comprises:

[0044] Interactively obtain multiple sample static behavior image sets of multiple static construction risk behaviors; interactively obtain multiple sample dynamic behavior video sets of multiple dynamic construction risk behaviors with the video time domain as a constraint; construct multiple static risk identification branches based on the multiple static construction risk behaviors and multiple sample static behavior image sets; construct multiple dynamic risk identification branches based on the multiple dynamic construction risk behaviors and multiple sample dynamic behavior video sets; parallel the multiple static risk identification branches to obtain a static violation analysis channel; parallel the multiple dynamic risk identification branches to obtain a dynamic violation analysis channel; parallel the static violation analysis channel and the dynamic violation analysis channel to complete the construction of the ground violation analysis model.

[0045] Static construction risk behaviors refer to risk behaviors that do not involve motion in the construction site. These behaviors can be identified through single-frame images, such as workers not wearing safety helmets, non-standard stacking of construction site items, and missing warning signs in dangerous areas. Image data of static construction risk behaviors is collected from multiple construction scenes, different construction time periods, and different camera perspectives to ensure the diversity and comprehensiveness of sample data, and multiple sample static behavior image sets are obtained.

[0046] Dynamic construction risk behavior refers to the risk behavior involving movement in the construction site, which needs to be identified through video sequences, such as illegal operation of hoisting equipment, abnormal driving of vehicles in the construction area, and workers climbing over guardrails, etc. Video clips of dynamic construction risk behavior are collected from monitoring videos of different camera perspectives and different time periods. The video duration should meet the video time domain to ensure the integrity of the behavior process and obtain multiple sample dynamic behavior video sets.

[0047] A multi-branch convolutional neural network architecture is constructed, including an input layer for receiving pre-processed image data and inputting into the convolutional neural network, a convolutional layer, each static risk identification branch using an independent convolutional layer to extract features of different types of static risk behavior, such as a branch dedicated to identifying the risk of not wearing a safety helmet and another branch for identifying the risk of non-standard equipment placement. According to the characteristics of specific risk behavior, appropriate convolution kernel size is designed to extract low-level and high-level image features. A pooling layer is used to reduce the size of the feature map, extract key features, and suppress noise. After the convolution and pooling layers, one or more fully connected layers are connected to map the extracted features to various risk behavior categories.

[0048] The multiple sample static behavior image sets are pre-processed, including image normalization, data enhancement, and labeling processing, etc. to facilitate training in the supervised learning process. For each static risk identification branch, the corresponding sample static behavior image set is used for independent training to ensure that the model has optimal recognition ability in each category. During the training process, the validation set is used for model verification, the hyperparameters are adjusted to avoid overfitting, and finally the model performance is evaluated on the test set to obtain multiple static risk identification branches that meet the requirements. Each branch independently identifies a specific category of static construction risk behavior, thereby realizing real-time and efficient identification of potential risks in the construction site.

[0049] A model combining convolutional neural network and long short-term memory network (CNN-LSTM) is used to build a branch that can process sequence data with spatio-temporal correlation and effectively identify behaviors with time sequence. Specifically, multiple sample dynamic behavior video sets of various dynamic construction risk behaviors are pre-processed, including video editing, frame extraction, data normalization, etc. to ensure uniform data format. Convolutional neural network (CNN) is used to extract features from pre-processed video data. CNN can automatically learn spatial features in video frames through convolutional layers, pooling layers, and activation functions, such as object shape, edge, texture, etc. These spatial features are used to identify potential risk behaviors in construction.

[0050] The features extracted from the CNN are input into a long short-term memory network (LSTM), which is a special type of recurrent neural network that is good at processing and predicting time-dependent relationships in sequence data. Through the memory unit of the LSTM, the model can capture the temporal patterns and behavior evolution in the sequence of video frames, thereby identifying time-series construction risk behaviors.

[0051] By inputting the processed sample data into the CNN-LSTM model for training, during the training process, the model continuously adjusts its parameters to minimize the loss function and optimize its ability to identify dynamic construction risk behaviors. Based on the trained CNN-LSTM model, multiple dynamic risk identification branches are constructed, each branch focusing on identifying different types of dynamic construction risk behaviors. For example, one branch focuses on identifying violations in hoisting equipment operations, while another branch focuses on the risk behaviors of ground construction personnel.

[0052] The multiple static risk identification branches are connected in parallel, each branch focusing on identifying a specific type of static construction risk behavior. Based on the parallel static risk identification branches, a static violation analysis channel is obtained, which focuses on identifying possible static risk behaviors or states in the images captured at a certain moment, such as whether workers are wearing safety helmets or reflective vests.

[0053] The multiple dynamic risk identification branches are connected in parallel to obtain a dynamic violation analysis channel, which focuses on detecting dynamic risk behaviors occurring in the video, such as whether the monitor personnel are climbing over the guardrails or performing dangerous operations.

[0054] The static violation analysis channel and the dynamic violation analysis channel are connected in parallel to form a complete ground violation analysis model, which can simultaneously process static and dynamic violation behaviors in the construction site, providing technical support for the safety management of the construction site.

[0055] Further, by synchronizing the local video sequence to the ground violation analysis model for ground violation behavior identification, a first risk behavior set is obtained, and the method further includes:

[0056] The local video sequence is stripped to obtain M behavioral videos of M ground operation users, wherein the video duration of each behavioral video in the M behavioral videos satisfies the video time domain, and M is a positive integer; key frames are extracted from the M behavioral videos to obtain M risk identification images; a first behavioral video and a first risk identification image of a first ground operation user are extracted based on the M behavioral videos and the M risk identification images, wherein the first ground operation user is any one of the M ground operation users; the first behavioral video and the first risk identification image are synchronized to the static violation analysis channel and the dynamic violation analysis channel of the ground violation analysis model respectively, and a multivariate identification analysis of violation risks is performed to obtain a first violation analysis result; the information of the first risk behavior set is updated according to the first violation analysis result; and so on, the first risk behavior set is obtained by synchronizing the local video sequence to the ground violation analysis model to perform ground violation behavior identification.

[0057] Video stripping is to split a long video containing multiple user behaviors into multiple shorter behavior videos. Each video focuses on the behavior of a single ground operation user, which can more effectively perform behavior analysis. Specifically, the local video sequence containing the behavior data of multiple operation users in a specific area is analyzed to identify the behavior of different ground operation users. Target detection or object tracking technology can be used to identify and track individuals. After detecting the behavior of different users, the original video is divided into M shorter behavior videos according to the set time range of the video time domain. Each behavior video contains the behavior of a user within a specified time period, and the duration of each behavior video meets the preset video time domain to ensure data consistency.

[0058] A keyframe is a single frame image in a behavioral video that is highly representative and contains important information. The purpose of extracting keyframes is to extract representative images from each behavioral video for further risk identification and analysis. Specifically, based on the duration and content of the behavioral video, an appropriate time interval or frame rate is selected to uniformly extract keyframes from the video. For example, a keyframe can be extracted every few seconds or a fixed number of frames, focusing on points of action change in the video, such as the start and end of fast movements or behaviors, which often contain useful information. Keyframes are extracted from M behavioral videos and saved as image files to form M risk identification images.

[0059] A user is randomly selected from M ground operation users as the first ground operation user, a behavior video corresponding to the first ground operation user is screened out from the M behavior videos as the first behavior video, and an image corresponding to the first ground operation user is screened out from the M risk identification images as the first risk identification image.

[0060] The first risk identification image is input into a static violation analysis channel, which mainly focuses on identifying static violations from single-frame images, such as whether a safety helmet is worn or not, and the first risk identification image is analyzed by using an identification model in the static violation analysis channel to detect whether there is a static violation in the image. The model extracts image features and classifies them to identify whether there is a violation of safety regulations.

[0061] The first behavior video is input into a dynamic violation analysis channel, which analyzes the action sequence in the video and identifies violations in dynamic behavior, such as jumping over a guardrail, and the first behavior video is analyzed by using an identification model in the dynamic violation analysis channel. The model processes time series data, extracts time series features through a convolutional neural network, and captures time-dependent relationships through a long short-term memory network to identify dynamic violation behavior in the video.

[0062] The results of static and dynamic violation analysis are integrated to generate the first violation analysis result, which is a comprehensive violation behavior analysis result for the user, effectively improving the accuracy and comprehensiveness of violation behavior identification.

[0063] If the first violation analysis result shows new violations or updates existing violations, these information is added to the first risk behavior set, such as adding newly detected violations, updating specific circumstances of violations or correcting previous records, to complete the information update of the first risk behavior set and ensure the accuracy and timeliness of the first risk behavior set.

[0064] The above steps are repeated to continuously identify violations in local video sequences and update the first risk behavior set, ensuring that the safety monitoring system can reflect the latest risk situation of the construction site in real time, thereby improving the safety management level of the construction site.

[0065] Further, according to the first violation analysis result, the information of the first risk behavior set is updated, and the method further comprises:

[0066] If the first violation analysis result is an empty set, the first violation analysis result is formatted. If the first violation analysis result is a non-empty set, the first behavior video, the first risk identification image and the first violation analysis result are stored in association, and the first ground operation user is stored as a storage identifier as a first ground violation behavior. The first ground violation behavior is updated to the first risk behavior set.

[0067] It is verified whether the first violation analysis result is an empty set, which means that no violation behavior is detected in the first violation analysis result. When the first violation analysis result is an empty set, the first violation analysis result is formatted.

[0068] When the first violation analysis result is not empty, it means that a violation behavior is detected in the first violation analysis result. In this case, the first behavior video, the first risk identification image and the first violation analysis result are stored in association, and the first ground operation user is taken as the unique storage identifier, which helps to track the specific person responsible for the violation behavior or related personnel in the future. All related violation information is recorded as the first ground violation behavior. The first ground violation behavior is added to the first risk behavior set to reflect the latest safety situation in real time.

[0069] Further, the K operation monitoring video streams in the operation rooms of the K real-time construction equipment are obtained interactively, and operation violation behavior identification is performed based on the K operation monitoring video streams to obtain a second risk behavior set. The method further comprises:

[0070] With the video time domain as a constraint, K sample operation violation video sets of the K real-time construction equipment are obtained interactively; K operation violation identification models are constructed based on the K sample operation violation video sets, and the K operation violation identification models are deployed on K local servers of the K real-time construction equipment; the K operation monitoring video streams obtained interactively are frame-disassembled with the video time domain as a constraint to obtain K sets of time sequence operation monitoring videos; the K sets of time sequence operation monitoring videos are mapped and synchronized to the K local servers, and behavior identification is performed in the K local servers via the K operation violation identification models to obtain the second risk behavior set.

[0071] With the video time domain as a constraint, K operation video data of the K real-time construction equipment in the past time are obtained by interacting with a real-time monitoring system of a construction site, the lengths of the operation video data meet the video time domain, the obtained operation video data are sample-labeled to obtain K sample operation violation video sets.

[0072] According to the K sample operation violation video sets, K operation violation identification models are constructed using appropriate deep learning algorithms such as convolutional neural networks (CNN) or convolutional neural networks combined with long short-term memory networks (CNN-LSTM), and the specific structure is the same as that of the model in the foregoing steps. Each model is optimized for a specific construction equipment and operation type. Each model is trained using the K sample operation violation video sets to ensure that the model can accurately identify different types of operation violation behaviors. The accuracy and robustness of the model are evaluated through cross-validation or an independent test set to ensure that the model performs well in a real environment.

[0073] The trained K operation violation identification models are deployed on K local servers of the K real-time construction equipment, and the models are integrated with the monitoring system of the construction equipment to ensure that the models can process video streams in real time and perform violation behavior identification tasks.

[0074] According to the pre-set video time domain, each of the K operation monitoring video streams is time period split, so that the length of each video segment matches the video time domain, each split video segment is processed frame by frame, key frames are extracted, the selection of the key frames should fully reflect the changes of the operation behavior, the split frames are recombined in time sequence to generate K sets of time sequence operation monitoring videos, each set of time sequence video corresponds to the operation monitoring of one real-time construction equipment.

[0075] The generated K sets of time sequence operation monitoring videos are synchronized to the corresponding K local servers, on each local server, according to the device source of the time sequence operation monitoring video, the video data is mapped into the corresponding identification model, the identification model analyzes the behavior in the operation video frame by frame, detects whether there is a behavior that violates the operation specification or has a potential risk, and generates a related identification result, the identified violation behavior result is summarized to form a second risk behavior set, and the result generated by each local server reflects the operation risk situation of the corresponding device.

[0076] Further, the K sets of time sequence operation monitoring videos are mapped and synchronized to the K local servers, and the behavior identification is performed in the K local servers via the K operation violation identification models to obtain the second risk behavior set, and the method further comprises:

[0077] The K sets of time sequence operation monitoring videos are mapped and synchronized to the K local servers, and the behavior identification is performed in the K local servers via the K operation violation identification models to obtain K violation identification sequences; the violation duration calculation is performed based on the K violation identification sequences to obtain K violation duration information; a violation risk threshold is preset, and the violation risk threshold and the K violation duration information are used for screening to obtain H real-time construction equipment, wherein H is a positive integer less than or equal to K; H violation operation dynamic pictures are extracted from the K sets of time sequence operation monitoring videos based on the H real-time construction equipment and the K violation identification sequences; the H violation operation dynamic pictures are identified by the H real-time construction equipment, and are used as the second risk behavior set.

[0078] The K sets of time sequence operation monitoring videos are respectively mapped to the corresponding K local servers, so that the video stream corresponds to the operation violation identification model deployed in the local server one by one, on each local server, the corresponding operation violation identification model is started, the model analyzes the operation behavior in the video frame by frame, detects whether there is a violation operation behavior, such as non-standard device use and violation of safety operation regulations, for each operation monitoring video, the identification model generates a violation identification sequence according to the identification result, and the sequence contains detailed information of the identified violation behavior at each time point.

[0079] The time point and duration of the identified violation behavior are extracted from each violation identification sequence, and the duration of each violation behavior is calculated according to the analysis result. For multiple violation behaviors in the same violation identification sequence, the total duration is combined to obtain the violation duration information. The violation duration information of each local server is summarized to form K pieces of violation duration information.

[0080] According to the safety requirements of the construction site, a violation risk threshold is preset, which can be set based on empirical data, industry standards or safety assessment. For example, if the violation duration of a device exceeds the set threshold, the device is high-risk.

[0081] The K pieces of violation duration information are compared with the preset violation risk threshold one by one. During the comparison process, devices exceeding the threshold are identified and screened to obtain H real-time construction devices. H is a positive integer less than or equal to K, indicating that the screened devices have too long violation duration, and these devices have a higher risk of violation during construction.

[0082] From the K sets of time sequence operation monitoring videos, the violation video clips associated with the H real-time construction devices are extracted, which include explicit violation operation behaviors. These violation video clips are converted into GIF or other format animation pictures to quickly review and identify specific violation operations in subsequent analysis. The animation pictures highlight the key violation actions.

[0083] The generated H violation operation animation pictures are associated with the corresponding H real-time construction device identifiers, so that each animation picture clearly identifies its corresponding device. The identified H violation operation animation pictures are integrated into a second risk behavior set, which is used for further safety assessment and processing.

[0084] In summary, the three-dimensional visual safety monitoring method for wind power hoisting sites provided by the embodiments of the present application has the following technical effects:

[0085] By combining real-time monitoring data with three-dimensional modeling, a three-dimensional construction map is constructed, realizing full-range and non-blind area monitoring of the hoisting site, improving the monitoring coverage range, reducing monitoring dead angles, and improving the comprehensiveness of on-site safety management; according to the hoisting construction progress, the illegal identification area is positioned, and the local video sequence is real-time cropped and exported for illegal behavior identification, through synchronous video to ground illegal analysis model, the risk behavior can be identified and marked in time, so as to reduce the potential safety accidents caused by illegal behavior not found in time; by obtaining the operation monitoring video stream of the construction equipment, and based on the operation illegal identification model for real-time analysis, the illegal behavior in the operation of the construction equipment can be effectively identified, further improving the overall safety of the construction site; by integrating the risk behavior into the three-dimensional construction map, a safety visual map is generated, which not only helps the management personnel to intuitively understand the construction progress and risk distribution, but also clearly shows the specific location and scene of the illegal behavior, improving the decision-making efficiency and accuracy. In summary, the method significantly improves the safety monitoring capability of the wind power hoisting site through comprehensive, real-time monitoring, timely illegal behavior identification, and intuitive three-dimensional visual display, providing strong technical support for the safety management of the construction site.

[0086] The above description of disclosed embodiments enables one of ordinary skill in the art to make or use the application. Various modifications to these embodiments will be readily apparent to those of ordinary skill in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Accordingly, the application is not to be limited to the embodiments shown herein but is to be accorded the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for three-dimensional visualized safety monitoring for a wind power installation site, characterized in that, The method comprises: constructing a construction three-dimensional map based on real-time monitoring of a wind power hoisting site; interacting with hoisting construction progress and positioning a violation identification area in the construction three-dimensional map according to the hoisting construction progress; presetting a video time domain, and deriving a local video sequence from the construction three-dimensional map by taking the violation identification area and the video time domain as constraints; preconstructing a ground violation analysis model, and identifying ground violation behaviors by synchronizing the local video sequence to the ground violation analysis model to obtain a first risk behavior set; extracting a hoisting component to be hoisted based on the hoisting construction progress, and determining a real-time construction equipment group based on the hoisting component to be hoisted, wherein the real-time construction equipment group operates in the violation identification area, and the real-time construction equipment group comprises K real-time construction equipment; interacting with operation monitoring video streams in operation rooms of the K real-time construction equipment, and identifying operation violation behaviors based on the operation monitoring video streams to obtain a second risk behavior set; visualizing and marking the first risk behavior set and the second risk behavior set on the construction three-dimensional map to generate a safety visual map; preconstructing a ground violation analysis model, and the method further comprises: interactively obtaining a plurality of sample static behavior image sets of a plurality of static construction risk behaviors; interactively obtaining a plurality of sample dynamic behavior video sets of a plurality of dynamic construction risk behaviors by taking the video time domain as a constraint; constructing a plurality of static risk identification branches based on the plurality of static construction risk behaviors and the plurality of sample static behavior image sets; constructing a plurality of dynamic risk identification branches based on the plurality of dynamic construction risk behaviors and the plurality of sample dynamic behavior video sets; parallel connecting the plurality of static risk identification branches to obtain a static violation analysis channel; parallel connecting the plurality of dynamic risk identification branches to obtain a dynamic violation analysis channel; parallel connecting the static violation analysis channel and the dynamic violation analysis channel to complete the construction of the ground violation analysis model.

2. The method of claim 1, wherein, Constructing a construction three-dimensional map based on real-time monitoring of a wind power hoisting site, the method further comprises: interactively obtaining a construction boundary of a wind power hoisting site, and collecting data according to the construction boundary to obtain hoisting site construction information; taking the construction boundary as a constraint, performing three-dimensional modeling based on the hoisting site construction information to obtain a hoisting site twin model; synchronizing an array of real-time video streams collected by an array of monitoring cameras pre-configured in the wind power hoisting site to the hoisting site twin model to obtain a construction three-dimensional map.

3. The method of claim 2, wherein, Synchronizing an array of real-time video streams collected by an array of monitoring cameras pre-configured in the wind power hoisting site to the hoisting site twin model to obtain a construction three-dimensional map, the method further comprises: interactively obtaining an original monitoring layout of the wind power hoisting site, and calling a plurality of historical video streams of a plurality of initial cameras in the original monitoring layout; fitting the plurality of historical video streams to the hoisting site twin model to obtain N monitoring blind areas; calling construction height information of the N monitoring blind areas based on the hoisting site construction information; performing monitoring coverage analysis on the N monitoring blind areas according to the construction height information to locate new monitoring sites; Update the original monitoring layout based on the new monitoring site to obtain the monitoring camera array.

4. The method of claim 1, wherein, Identify ground violation behaviors by synchronizing the local video sequence to the ground violation analysis model to obtain a first risk behavior set, and the method further comprises: Video segmentation is performed on the local video sequence to obtain behavior videos of M ground operation users, wherein the video duration of each behavior video in the behavior video satisfies the video time domain, and M is a positive integer; Extract key frames from the behavior videos to obtain risk identification images; Extract the first behavior video and the first risk identification image of the first ground operation user based on the behavior video and the risk identification image, wherein the first ground operation user is any one of the M ground operation users; Synchronize the first behavior video and the first risk identification image to the static violation analysis channel and the dynamic violation analysis channel of the ground violation analysis model respectively, perform multi-element identification analysis of violation risk, and obtain a first violation analysis result; Update the information of the first risk behavior set according to the first violation analysis result; Identify ground violation behaviors by synchronizing the local video sequence to the ground violation analysis model to obtain the first risk behavior set.

5. The method of claim 4, wherein, Update the information of the first risk behavior set according to the first violation analysis result, and the method further comprises: If the first violation analysis result is an empty set, format the first violation analysis result; If the first violation analysis result is a non-empty set, store the first behavior video, the first risk identification image and the first violation analysis result in association, and store the first ground operation user as a storage identifier, and store the first ground violation behavior; Update the first ground violation behavior to the first risk behavior set.

6. The method of claim 1, wherein, Interactively obtain operation monitoring video streams in the operation rooms of the K real-time construction equipment, and identify operation violation behaviors based on the operation monitoring video streams to obtain a second risk behavior set, and the method further comprises: Interactively obtain a sample operation violation video set of the K real-time construction equipment with the video time domain as a constraint; Construct an operation violation identification model based on the sample operation violation video set, and deploy the operation violation identification model on a local server of the K real-time construction equipment; Frame the interactively obtained operation monitoring video stream with the video time domain as a constraint to obtain a time sequence operation monitoring video; Map and synchronize the time sequence operation monitoring video to the local server, and identify behaviors in the local server via the operation violation identification model to obtain the second risk behavior set.

7. The method of claim 6, wherein, Map and synchronize the time sequence operation monitoring video to the local server, and identify behaviors in the local server via the operation violation identification model to obtain the second risk behavior set, and the method further comprises: Map and synchronize the time sequence operation monitoring video to the local server, and identify behaviors in the local server via the operation violation identification model to obtain a violation identification sequence; Perform a violation duration calculation based on the violation identification sequence to obtain violation duration information; A violation risk threshold is preset, and the violation risk threshold and the violation duration information are used to filter to obtain H real-time construction equipment, where H is a positive integer less than or equal to K; Based on the H real-time construction equipment and the violation identification sequence, a violation operation animation is extracted from the time sequence operation monitoring video; The H real-time construction equipment is used to identify the violation operation animation as the second risk behavior set.

Images