Three-dimensional visual safety monitoring method for wind power hoisting site

Abstract

The invention provides a three-dimensional visual safety monitoring method for a wind power hoisting site, and relates to the technical field of safety monitoring, and the method comprises the steps: building a construction three-dimensional map based on the real-time monitoring of the wind power hoisting site; interacting the hoisting construction progress, and positioning a violation identification area; presetting a video time domain, and cutting and exporting a local video sequence; pre-constructing a ground violation analysis model, and performing ground violation behavior identification to obtain a first risk behavior set; extracting to obtain a to-be-hoisted component, and determining a real-time construction equipment group; k operation monitoring video streams are obtained through interaction, operation violation behavior recognition is carried out, and a second risk behavior set is obtained; and carrying out visual identification to generate a safe visual map. The technical problem that in the prior art, the safety management level of a construction site is poor due to the fact that safety monitoring data of a wind power hoisting site are always displayed in a non-visual mode and the safety risk position and degree are difficult to judge rapidly and visually is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of safety monitoring, and in particular to a three-dimensional visual safety monitoring method for a wind power installation site. Background Art

[0002] The hoisting of wind turbines needs to be carried out at high altitudes and involves the coordinated operation of multiple construction equipment and personnel. Current technical means usually rely on human supervision and traditional monitoring equipment, and there are some technical problems. Specifically, during the hoisting process, illegal operations of construction personnel and equipment may lead to serious safety accidents. Traditional monitoring systems are difficult to achieve comprehensive coverage of the entire construction area, and are prone to monitoring blind spots. In addition, the safety monitoring data of the hoisting site is often presented in a non-intuitive form, making it difficult to quickly and intuitively determine the location and degree of safety risks, resulting in poor pertinence of construction safety monitoring and slow safety risk investigation, which in turn leads to poor safety management level at the construction site. Summary of the invention

[0003] The present application provides a three-dimensional visual safety monitoring method for wind turbine hoisting sites, aiming to solve the technical problem in the prior art that safety monitoring data of wind turbine hoisting sites are often presented in a non-intuitive form, making it difficult to quickly and intuitively determine the location and degree of safety risks, resulting in poor safety management level at the construction site.

[0004] The present application discloses a three-dimensional visualization safety monitoring method for a wind power hoisting site, the method comprising: constructing a three-dimensional construction map based on real-time monitoring of the wind power hoisting site; interactively hoisting construction progress, and locating a violation identification area in the three-dimensional construction map according to the hoisting construction progress; presetting a video time domain, taking the violation identification area and the video time domain as constraints, cropping and deriving a local video sequence from the three-dimensional construction map; pre-constructing a ground violation analysis model, and performing ground violation behavior identification by synchronizing the local video sequence to the ground violation analysis model to obtain a first risk behavior set; extracting and obtaining components to be hoisted based on the hoisting construction progress, and determining a real-time construction equipment group based on the components to be hoisted, wherein the real-time construction equipment group operates in the violation identification area, and the real-time construction equipment group includes K real-time construction equipment; interactively obtaining K operation monitoring video streams in the operation rooms of the K real-time construction equipment, and performing operation violation behavior identification based on the K operation monitoring video streams to obtain a second risk behavior set; visually marking the first risk behavior set and the second risk behavior set in the three-dimensional construction map to generate a safety visualization map.

[0005] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0006] By combining real-time monitoring data with three-dimensional modeling, a three-dimensional construction map is constructed, which realizes all-round and blind-spot monitoring of the hoisting site, improves the monitoring coverage, reduces the monitoring blind spots, and improves the comprehensiveness of on-site safety management; locates the violation identification area according to the hoisting construction progress, and cuts and exports the local video sequence in real time for violation identification. By synchronizing the video to the ground violation analysis model, it can instantly identify and mark risk behaviors, thereby reducing potential safety accidents caused by the failure to timely discover violations; by obtaining the operation monitoring video stream of the construction equipment and performing real-time analysis based on the operation violation identification model, it can effectively identify violations in the operation of the construction equipment, further improving the overall safety of the construction site; by integrating risk behaviors into the construction three-dimensional map, a safety visualization map is generated, which not only helps managers intuitively understand the construction progress and risk distribution, but also can clearly show the specific location and scene of the violation, improving the decision-making efficiency and accuracy. In summary, this method significantly improves the safety monitoring capability of the wind power hoisting site through comprehensive and real-time monitoring, timely violation identification, and intuitive three-dimensional visualization, and provides strong technical support for the safety management of the construction site.

[0007] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A schematic flow chart of a three-dimensional visual safety monitoring method for a wind power installation site is provided for an embodiment of the present application;

[0009] Figure 2 A schematic diagram of the process of constructing a three-dimensional construction map in a three-dimensional visual safety monitoring method for a wind power installation site is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0010] The embodiment of the present application provides a three-dimensional visual safety monitoring method for wind turbine hoisting sites, thereby solving the technical problem in the prior art that safety monitoring data of wind turbine hoisting sites are often presented in a non-intuitive form, making it difficult to quickly and intuitively determine the location and degree of safety risks, resulting in a poor level of safety management at the construction site.

[0011] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application will be specifically described below in conjunction with the drawings in the specification.

[0012] like Figure 1As shown, the embodiment of the present application provides a three-dimensional visual safety monitoring method for a wind power installation site, the method comprising:

[0013] Build a three-dimensional construction map based on real-time monitoring of the wind turbine installation site.

[0014] The boundary data of the construction area is measured by measuring instruments to determine the construction boundary of the wind power hoisting site. This is used as the basis for building a three-dimensional construction map. Data is collected according to the construction boundary to obtain the construction information of the hoisting site. Three-dimensional modeling is performed based on the construction information of the hoisting site to generate a twin model of the hoisting site. This model contains all the key elements of the construction site, such as hoisting equipment, wind turbine components, topography, etc. The surveillance camera array deployed at the wind power hoisting site collects video stream data in real time, and maps the texture data in the video stream to the twin model of the hoisting site to enhance the realism of the model and make it more intuitive, so as to obtain a three-dimensional construction map and provide basic support for subsequent construction progress positioning, violation identification and risk visualization.

[0015] Interactively monitor the progress of the hoisting construction, and locate the illegal identification area in the three-dimensional construction map according to the progress of the hoisting construction.

[0016] Obtain real-time hoisting construction progress data from the construction management system, including the working status of the hoisting equipment, the completion percentage of the current hoisting task, and the estimated completion time. Associate the obtained construction progress data with the construction three-dimensional map. Each progress node, such as a component of a wind turbine that has been hoisted to a specified position, is located and marked on the three-dimensional map. This can be achieved through the conversion of the coordinate system or direct mapping. Combined with the construction progress and known construction risks, identify high-risk areas where violations may occur. For example, during hoisting operations, there may be risks such as overloading of hoisting equipment and personnel mistakenly entering dangerous areas. On the three-dimensional map, the identified high-risk areas are marked as violation identification areas. These areas will be subject to key monitoring to ensure that possible violations are discovered and handled in a timely manner during the actual construction process.

[0017] The video time domain is preset, and a local video sequence is cut and derived from the three-dimensional construction map with the violation identification area and the video time domain as constraints.

[0018] The video time domain refers to a time span range, that is, the time period of the construction process that needs to be analyzed. The video data within this time period will be analyzed in detail to identify violations. The time domain can be a fixed time period set according to the duration of the construction task, the equipment working cycle, etc., such as 10 seconds, 30 seconds, or it can be dynamically adjusted according to the specific requirements of the construction task.

[0019] In the located violation identification area, combined with the set video time domain, the time period and area range that need to be monitored are determined, and the corresponding local video sequences are cut out from the construction three-dimensional map and monitoring video. These sequences focus on reflecting the construction process within a specific time period and a specific area.

[0020] A ground violation analysis model is pre-built, and ground violation behavior is identified by synchronizing the local video sequence to the ground violation analysis model to obtain a first risk behavior set.

[0021] A large amount of historical video data is collected, including videos of annotated violations and normal operations. The sample data covers various possible ground violations, such as personnel not wearing safety equipment, misoperating equipment, not walking along the prescribed route, etc. According to the characteristics of the sample data, multiple static risk identification branches are constructed to identify violations at specific moments, such as improper equipment position, and multiple dynamic risk identification branches are constructed to identify behaviors over a period of time, such as non-compliant operating procedures. Multiple static risk identification branches and multiple dynamic risk identification branches are integrated to complete the construction of the ground violation analysis model.

[0022] The cropped partial video sequence is split into video strips, and the video including the behavior of multiple people is split into multiple behavior videos of multiple people. The obtained multiple behavior videos are synchronized to the ground violation analysis model, and key frames are extracted from the video sequence to extract the feature information therein. Violation identification of static images is performed through multiple static risk identification branches to obtain a static violation behavior set; violation identification of dynamic images of multiple behavior videos is performed through multiple dynamic risk identification branches to obtain a dynamic violation behavior set. The static violation behavior set and the dynamic violation behavior set are integrated to obtain the first risk behavior set.

[0023] The components to be hoisted are obtained based on the hoisting construction progress extraction, and a real-time construction equipment group is determined based on the components to be hoisted, wherein the real-time construction equipment group operates in the violation identification area, and the real-time construction equipment group includes K real-time construction equipment.

[0024] Analyze the specific components that need to be hoisted from the construction progress, such as the blades, towers, generators, etc. of the wind turbine, obtain the components to be hoisted, and obtain basic information about the components from the project database, including the basic characteristics of the components, transportation routes, hoisting requirements, etc. According to the basic information of the components, determine which construction equipment is needed to complete the hoisting task. According to the real-time status of the equipment in the violation identification area, such as whether it is idle or whether other tasks are being performed, schedule appropriate K real-time construction equipment to form a real-time construction equipment group. K represents the number of construction equipment in the real-time construction equipment group, which is a positive integer determined according to factors such as task complexity, equipment characteristics, and construction area size. The construction equipment group usually includes cranes, hoists, slings, balancing equipment, auxiliary vehicles, and other equipment. These equipment work together to complete specific hoisting tasks.

[0025] Interactively obtain K operation monitoring video streams in the operation rooms of the K real-time construction equipment, and identify operation violations based on the K operation monitoring video streams to obtain a second risk behavior set.

[0026] A camera is configured in the operation room of each real-time construction equipment to capture the operator's behavior and the environment in the operation room. K operation monitoring video streams in the operation rooms of K real-time construction equipment are obtained through camera collection.

[0027] Obtain a sample operation violation video set of various operation violation behaviors recorded in history, such as excessive manipulation, dangerous operation, and disregard for safety regulations. Use historical video data to train an operation violation recognition model so that it can recognize a variety of possible violations and adapt to different equipment and operating environments. Deploy the trained operation violation recognition model to the local servers of K real-time construction equipment, analyze the K operation monitoring video streams, and identify possible violations, including violations of safety operating procedures, operating errors, fatigue driving, etc. Mark the identified violations on the corresponding video frames to generate a second risk behavior set.

[0028] The first risk behavior set and the second risk behavior set are visually marked on the three-dimensional construction map to generate a safety visualization map.

[0029] The first risk behavior set includes violation data in ground operations at the construction site identified by a ground violation identification model, and the second risk behavior set includes violation data in real-time construction equipment operations identified by an operation violation identification model. The data of the first risk behavior set and the second risk behavior set are organized into a format suitable for three-dimensional map visualization and associated with corresponding positions in the construction three-dimensional map. The coordinate information of each risk behavior is mapped to a specific position in the three-dimensional map, and a risk identification layer is superimposed on the three-dimensional map. The identification points or areas of the risk behavior are displayed on the three-dimensional model. For high-risk areas, enlarged images or detailed views are designed so that users can see the specific risk situation more clearly. The construction three-dimensional map is synthesized with the risk identification layer to generate a final safety visualization map. The visualization map provides intuitive data support for risk assessment and helps managers make more accurate safety decisions and take measures.

[0030] Furthermore, if Figure 2 As shown, a three-dimensional construction map is constructed based on real-time monitoring of the wind power installation site, and the method further includes:

[0031] Interactively obtain the construction boundary of the wind power hoisting site, and collect data based on the construction boundary to obtain the construction information of the hoisting site; use the construction boundary as a constraint, perform three-dimensional modeling based on the construction information of the hoisting site, and obtain a twin model of the hoisting site; synchronize the real-time video stream array collected by the monitoring camera array pre-configured at the wind power hoisting site to the twin model of the hoisting site to obtain a three-dimensional construction map.

[0032] The boundary data of the construction area is measured by measuring instruments to determine the construction boundary of the wind power hoisting site. Data is collected within the construction boundary, including on-site facility collection, such as buildings, infrastructure and other structures, construction planning collection, including construction planning, equipment placement, etc., and environmental data collection, including terrain, meteorological conditions, soil type, etc. The collected information is sorted to obtain the construction information of the hoisting site.

[0033] Select a suitable 3D modeling tool, such as AutoCAD, for modeling, and import the construction boundary data and construction information into the modeling software as the basic data for modeling. Specifically, draw the outer contour of the construction area according to the construction boundary data, and establish a 3D boundary model of the construction area. Add on-site buildings, equipment, roads and other facilities to the 3D boundary model based on the construction information. Finally, add model details, such as equipment location, construction area, material stacking, etc., to generate a twin model of the lifting site. The twin model is a digital simulation model of the construction site for further analysis.

[0034] Obtain the surveillance camera array pre-configured at the wind turbine hoisting site. These cameras continuously capture the construction dynamics on site, including equipment operation, personnel activities, etc. The real-time video streams of each camera are aggregated to form a real-time video stream array. This array contains real-time video information of each area of ​​the hoisting site.

[0035] The video stream is synchronized with the time and space coordinates of the twin model so that each video stream can be displayed at the exact position and time point in the twin model. After synchronization, each video stream is embedded in the corresponding surveillance camera position and angle in the twin model so that the content in the video stream is consistent with the scene of the twin model, and the real-time situation on site can be dynamically displayed to obtain a three-dimensional construction map. This map not only shows the three-dimensional layout of the site, but also can display the construction dynamics of each monitoring area in real time.

[0036] Furthermore, a real-time video stream array acquired by a monitoring camera array pre-configured at the wind turbine installation site is synchronized to the twin model of the installation site to obtain a three-dimensional construction map. The method further includes:

[0037] Interactively obtain the original monitoring layout of the wind power hoisting site, and call multiple historical video streams of multiple initial cameras in the original monitoring layout; fit the multiple historical video streams to the twin model of the hoisting site to obtain N monitoring blind spots; obtain N construction height information of the N monitoring blind spots based on the construction information of the hoisting site; perform monitoring coverage analysis on the N monitoring blind spots according to the N construction height information, and locate N newly added monitoring sites; update the original monitoring layout based on the N newly added monitoring sites to obtain the monitoring camera array.

[0038] By interacting with the monitoring system of the wind power installation site, the original monitoring layout information of the site is obtained, including the location, installation angle, coverage and other detailed configurations of all initial cameras. Multiple initial cameras are obtained according to the original monitoring layout, and the historical video streams of these cameras are retrieved. These video streams record the on-site conditions in the past time period.

[0039] The spatial information in the video frame is converted into coordinates in the twin model and matched with the three-dimensional structure of the twin model to determine the position and angle of each video frame in the twin model. During the fitting process, the areas without video coverage are identified, which are monitoring blind spots. According to the fitting results, N monitoring blind spots at the lifting site are identified, where N is the number of monitoring blind spots, which is a positive integer. These blind spots are areas in the twin model that cannot be effectively monitored through historical video streams.

[0040] Call up the collected construction information of the lifting site, especially the height data related to the construction area, including the construction height of buildings, equipment, structures, etc., associate the N monitoring blind spots obtained with their corresponding construction height information, clarify the vertical height range of each blind spot in the twin model, and obtain N construction height information.

[0041] Conduct monitoring coverage analysis. Specifically, based on the construction height information of the blind spots, simulate the coverage effect after installing the monitoring equipment at different heights or positions, consider the installation height and angle of the equipment, and the interference of adjacent equipment, find the best monitoring point, and evaluate whether full coverage of the blind spots can be achieved by calculating the monitoring range of each proposed new equipment. Ensure that all blind spots can be fully monitored by at least one device. According to the monitoring coverage analysis results, determine the best monitoring equipment layout position for each blind spot, and obtain N new monitoring sites. These new monitoring sites can make up for the deficiencies of the existing monitoring system and achieve full coverage.

[0042] The N newly located monitoring sites are added to the original monitoring layout. Based on the equipment layout of the newly added sites, the original monitoring equipment is fine-tuned, such as angle adjustment and position adjustment, to avoid mutual interference between equipment and optimize the monitoring coverage effect. All updated monitoring equipment is integrated to generate the final monitoring camera array, which covers all the original blind spots and improves the overall monitoring capability and safety of the wind power hoisting site.

[0043] Furthermore, a ground violation analysis model is pre-built, and the method further comprises:

[0044] Interactively obtain multiple sample static behavior image sets of multiple static construction risk behaviors; using the video time domain as a constraint, interactively obtain multiple sample dynamic behavior video sets of multiple dynamic construction risk behaviors; construct multiple static risk identification branches based on the multiple static construction risk behaviors and multiple sample static behavior image sets; construct multiple dynamic risk identification branches based on the multiple dynamic construction risk behaviors and multiple sample dynamic behavior video sets; connect the multiple static risk identification branches in parallel to obtain a static violation analysis channel; connect the multiple dynamic risk identification branches in parallel to obtain a dynamic violation analysis channel; connect the static violation analysis channel and the dynamic violation analysis channel in parallel to complete the construction of the ground violation analysis model.

[0045] Static construction risk behaviors refer to risk behaviors that do not involve movement at the construction site. These behaviors can be identified through a single-frame image. For example, workers not wearing safety helmets, items on the construction site are not stacked in a standardized manner, warning signs in dangerous areas are missing, etc. Image data of static construction risk behaviors are collected from multiple construction scenes, different construction time periods, and different camera perspectives to ensure the diversity and comprehensiveness of the sample data and obtain multiple sample static behavior image sets.

[0046] Dynamic construction risk behaviors refer to risk behaviors involving movement at the construction site. These behaviors need to be identified through video sequences, such as illegal operation of lifting equipment, abnormal driving of vehicles in the construction area, workers climbing over guardrails, etc. Video clips of dynamic construction risk behaviors are collected from surveillance videos of different camera perspectives and different time periods. The video length during collection should meet the video time domain to ensure the integrity of the behavior process and obtain multiple sample dynamic behavior video sets.

[0047] Construct a multi-branch convolutional neural network architecture, including an input layer for receiving preprocessed image data and inputting it into the convolutional neural network; a convolutional layer, in which each static risk identification branch uses an independent convolutional layer to extract the features of different categories of static risk behaviors. For example, one branch is specifically used to identify the risk behavior of not wearing a safety helmet, and the other branch is used to identify the behavior of improperly placed equipment. According to the characteristics of specific risk behaviors, a suitable convolution kernel size is designed to extract low-level and high-level image features; a pooling layer, which uses maximum pooling or average pooling to reduce the size of the feature map, extract key features, and suppress noise; after the convolution and pooling layers, connect to one or more fully connected layers to map the extracted features to each risk behavior category.

[0048] Preprocess multiple sample static behavior image sets, including image normalization, data enhancement, labeling, etc., to facilitate training in the supervised learning process. For each static risk identification branch, use the corresponding sample static behavior image set for independent training to ensure that the model has the best recognition ability in its respective category. During the training process, use the validation set to verify the model, adjust the hyperparameters to avoid overfitting, and finally evaluate the model performance on the test set to obtain multiple static risk identification branches that meet the requirements. Each branch independently identifies a specific category of static construction risk behavior, thereby achieving real-time and efficient identification of potential risks on the construction site.

[0049] A model combining convolutional neural network and long short-term memory network (CNN-LSTM) is used to build a branch. This branch can process sequence data with spatiotemporal correlation and effectively identify temporal behaviors. Specifically, multiple sample dynamic behavior video sets of various dynamic construction risk behaviors are preprocessed, including video editing, frame extraction, data normalization and other steps to ensure the uniformity of data format. Convolutional neural network (CNN) is used to extract features of the preprocessed video data. CNN can automatically learn spatial features in video frames, such as object shape, edge, texture and other information in the image through structures such as convolutional layers, pooling layers and activation functions. These spatial features are used to identify potential risk behaviors in construction.

[0050] The features extracted from CNN are input into the long short-term memory network (LSTM). LSTM is a special recursive neural network that is good at processing and predicting temporal dependencies in sequence data. Through the memory units of LSTM, the model can capture the temporal patterns and behavioral evolution in video frame sequences, thereby identifying construction risk behaviors with temporal characteristics.

[0051] The processed sample data is input into the CNN-LSTM model for training. During the training process, the model continuously adjusts parameters to minimize the loss function and optimizes its ability to identify dynamic construction risk behaviors. Based on the trained CNN-LSTM model, multiple dynamic risk identification branches are constructed. Each branch performs special identification on different types of dynamic construction risk behaviors. For example, one branch focuses on identifying violations in lifting equipment operations, while another branch focuses on the risk behaviors of ground construction workers.

[0052] The constructed multiple static risk identification branches are connected in parallel, and each branch focuses on identifying a specific type of static construction risk behavior. According to the parallel static risk identification branches, a static violation analysis channel is obtained, which focuses on identifying possible static risk behaviors or states in the image captured at a certain moment, such as whether the workers are wearing safety helmets or reflective clothing.

[0053] The constructed multiple dynamic risk identification branches are connected in parallel to obtain a dynamic violation analysis channel, which focuses on detecting dynamic risk behaviors occurring in the video, for example, whether the monitoring personnel climb over the guardrail or perform dangerous operations.

[0054] The static violation analysis channel and the dynamic violation analysis channel are connected in parallel to form a complete ground violation analysis model. This model can handle both static and dynamic violations at the construction site, providing technical support for safety management of the construction site.

[0055] Furthermore, by synchronizing the local video sequence to the ground violation analysis model to identify ground violation behaviors, a first risk behavior set is obtained, and the method further includes:

[0056] The local video sequence is stripped to obtain M behavior videos of M ground operation users, wherein the video duration of each of the M behavior videos satisfies the video time domain, and M is a positive integer; key frames are extracted from the M behavior videos to obtain M risk identification images; a first behavior video and a first risk identification image of a first ground operation user are extracted based on the M behavior videos and the M risk identification images, wherein the first ground operation user is any one of the M ground operation users; the first behavior video and the first risk identification image are synchronized to the static violation analysis channel and the dynamic violation analysis channel of the ground violation analysis model respectively, and a multivariate identification analysis of violation risks is performed to obtain a first violation analysis result; information of the first risk behavior set is updated according to the first violation analysis result; and by analogy, the first risk behavior set is obtained by synchronizing the local video sequence to the ground violation analysis model to perform ground violation behavior identification.

[0057] Video stripping is to split a long video containing multiple user behaviors into multiple shorter behavior videos. Each video focuses on the behavior of a single ground operation user. This can make behavior analysis more effective. Specifically, the local video sequence containing the behavior data of multiple operation users in a specific area is analyzed to identify the behaviors of different ground operation users. Target detection or object tracking technology can be used to identify and track individuals. After detecting the behaviors of different users, the original video is divided into M shorter behavior videos according to the set time range of the video time domain. Each behavior video contains the behavior of a user within a specified time period, and the duration of each behavior video meets the preset video time domain to ensure data consistency.

[0058] A key frame refers to a single frame image that is highly representative and contains important information in a behavior video. The purpose of extracting key frames is to extract representative images from each behavior video for further risk identification and analysis. Specifically, according to the duration and content of the behavior video, select an appropriate time interval or frame rate, and evenly extract key frames from the video. For example, a key frame can be extracted every few seconds or every fixed number of frames, focusing on the action change points in the video, such as the start and end of fast actions or behaviors, which usually contain useful information. Extract key frames from M behavior videos, save the extracted key frames as image files, and form M risk identification images.

[0059] A ground operation user is randomly selected from M ground operation users as the first ground operation user, a behavior video corresponding to the first ground operation user is screened out from the M behavior videos as the first behavior video, and an image corresponding to the first ground operation user is screened out from the M risk identification images as the first risk identification image.

[0060] The first risk identification image is input into the static violation analysis channel, which mainly focuses on identifying static violations from a single frame image, such as whether a helmet is worn, etc. The first risk identification image is analyzed using the recognition model in the static violation analysis channel to detect whether there is static violation in the image. The model extracts image features and classifies them to identify whether there is any violation of safety regulations.

[0061] The first behavior video is input into the dynamic violation analysis channel, which analyzes the action sequence in the video and identifies violations in dynamic behaviors, such as climbing over guardrails. The first behavior video is analyzed using the recognition model in the dynamic violation analysis channel. The model processes time series data, extracts time series features through convolutional neural networks, and then captures time dependencies through long short-term memory networks, thereby identifying dynamic violations in the video.

[0062] The results of static and dynamic violation analysis are integrated to generate a first violation analysis result, which is a comprehensive violation behavior analysis result of the user, effectively improving the accuracy and comprehensiveness of violation behavior identification.

[0063] If the first violation analysis result shows a new violation or updates an existing violation, this information is added to the first risk behavior set, for example, adding a newly detected violation, updating the details of the violation, or correcting the previous record, thereby completing the information update of the first risk behavior set and ensuring the accuracy and timeliness of the first risk behavior set.

[0064] Repeat the above steps to continuously identify illegal behaviors in the local video sequence and update the first risk behavior set, ensuring that the safety monitoring system can reflect the latest risk situation of the construction site in real time, thereby improving the safety management level of the construction site.

[0065] Furthermore, updating the information of the first risk behavior set according to the first violation analysis result, the method further includes:

[0066] If the first violation analysis result is an empty set, format the first violation analysis result; if the first violation analysis result is a non-empty set, store the first behavior video, the first risk identification image and the first violation analysis result in association, and use the first ground operation user as a storage identifier as the first ground violation behavior; update the first ground violation behavior to the first risk behavior set.

[0067] Verify whether the first violation analysis result is an empty set, where an empty set means that no violation behavior is detected in the first violation analysis result. When the first violation analysis result is an empty set, format the first violation analysis result.

[0068] When the first violation analysis result is not an empty set, it means that a violation is detected in the first violation analysis result. In this case, the first behavior video, the first risk identification image and the first violation analysis result are stored in association, and the first ground operation user is used as a unique storage identifier, which helps to track the specific person responsible or related personnel of the violation in the future. All relevant violation information is recorded as the first ground violation. The first ground violation is added to the first risk behavior set to reflect the latest safety status in real time.

[0069] Further, interactively obtaining K operation monitoring video streams in the operation rooms of the K real-time construction equipment, and identifying operation violation behaviors based on the K operation monitoring video streams to obtain a second risk behavior set, the method further includes:

[0070] Taking the video time domain as a constraint, interactively obtain K sample operation violation video sets of the K real-time construction equipment; construct and obtain K operation violation recognition models based on the K sample operation violation video sets, and deploy the K operation violation recognition models on K local servers of the K real-time construction equipment; split the K operation monitoring video streams interactively obtained by taking the video time domain as a constraint to obtain K groups of time-sequential operation monitoring videos; map and synchronize the K groups of time-sequential operation monitoring videos to the K local servers, perform behavior recognition in the K local servers via the K operation violation recognition models, and obtain the second risk behavior set.

[0071] Taking the video time domain as a constraint, by interacting with the real-time monitoring system of the construction site, the operation video data of K real-time construction equipment in the past time are obtained. The length of these operation video data meets the video time domain. The obtained operation video data are sampled and K sample operation violation video sets are obtained.

[0072] Based on the K sample operation violation video sets, use appropriate deep learning algorithms, such as convolutional neural networks (CNN) or a combination of convolutional neural networks and long short-term memory networks (CNN-LSTM), to build K operation violation recognition models, the specific structure of which is the same as the model in the previous steps, and each model is optimized for specific construction equipment and operation types. Train each model with the K sample operation violation video sets to ensure that the model can accurately identify different types of operation violations, and evaluate the accuracy and robustness of the model through cross-validation or independent test sets to ensure that the model performs well in real environments.

[0073] Deploy the trained K operation violation recognition models to K local servers of K real-time construction equipment, and integrate the models with the monitoring system of the construction equipment to ensure that the models can process video streams in real time and perform violation recognition tasks.

[0074] According to the pre-set video time domain, each of the K operation monitoring video streams is split into time periods so that the length of each video segment matches the video time domain. Each split video segment is processed frame by frame to extract the key frames. The selection of key frames should be able to fully reflect the changes in operation behavior. The split frames are recombined in chronological order to generate K groups of time-series operation monitoring videos. Each group of time-series videos corresponds to the operation monitoring of a real-time construction equipment.

[0075] The generated K groups of time-series operation monitoring videos are synchronized to the corresponding K local servers. On each local server, the video data is mapped to the corresponding recognition model according to the device source of the time-series operation monitoring video. The recognition model analyzes the behavior in the operation video frame by frame to detect whether there is any violation of the operating specifications or any behavior with potential risks, and generates relevant recognition results. The identified violation results are summarized to form a second risk behavior set. The results generated by each local server reflect the corresponding equipment operation risk situation.

[0076] Further, the K groups of time-series operation monitoring video mappings are synchronized to the K local servers, and behavior recognition is performed in the K local servers via the K operation violation recognition models to obtain the second risk behavior set. The method also includes:

[0077] The K groups of sequential operation monitoring video mappings are synchronized to the K local servers, and behavior recognition is performed in the K local servers through the K operation violation recognition models to obtain K violation recognition sequences; violation duration calculation is performed based on the K violation recognition sequences to obtain K violation duration information; a violation risk threshold is preset, and the violation risk threshold and the K violation duration information are used to screen and obtain H real-time construction equipment, where H is a positive integer less than or equal to K; based on the H real-time construction equipment and the K violation recognition sequences, H violation operation animations are extracted from the K groups of sequential operation monitoring videos to obtain H violation operation animations; the H real-time construction equipment are used to identify the H violation operation animations as the second risk behavior set.

[0078] Map K groups of time-series operation monitoring videos to corresponding K local servers respectively, so that the video streams correspond one-to-one with the operation violation recognition models deployed in the local servers. On each local server, start the corresponding operation violation recognition model. The model analyzes the operation behavior in the video frame by frame to detect whether there are any illegal operation behaviors, such as irregular equipment use, violation of safety operation regulations, etc. For each operation monitoring video, the recognition model generates a violation recognition sequence based on the recognition results. This sequence contains detailed information on the violations identified at each time point.

[0079] Each violation identification sequence is parsed to extract the time point and duration of the identified violation. Based on the parsing results, the duration of each violation is calculated. For multiple violations in the same violation identification sequence, their total duration is combined to obtain the violation duration information. The violation duration information of each local server is summarized to form K violation duration information.

[0080] According to the safety requirements of the construction site, a violation risk threshold is preset. The threshold can be set based on empirical data, industry standards or safety assessments. For example, if the violation duration of a certain equipment exceeds the set threshold, the equipment is considered high risk.

[0081] The K violation duration information is compared with the preset violation risk threshold one by one. During the comparison process, the equipment exceeding the threshold is identified and screened out to obtain H real-time construction equipment. H is a positive integer less than or equal to K, indicating that the screened-out equipment has a violation duration that is too long. These equipment have a higher violation risk during the construction process.

[0082] From K groups of sequential operation monitoring videos, extract illegal video clips associated with H real-time construction equipment. These clips include clear illegal operation behaviors. Convert these illegal video clips into GIF or other formats of animated images so that specific illegal operations can be quickly reviewed and identified in subsequent analysis. The animated images highlight key illegal actions.

[0083] The generated H illegal operation animations are associated with the corresponding H real-time construction equipment identifications, so that each animation clearly identifies the corresponding equipment, and the identified H illegal operation animations are integrated into the second risk behavior set, which is used for further safety assessment and processing.

[0084] In summary, the three-dimensional visual safety monitoring method for wind power installation sites provided in the embodiments of the present application has the following technical effects:

[0085] By combining real-time monitoring data with three-dimensional modeling, a three-dimensional construction map is constructed, which realizes all-round and blind-spot monitoring of the hoisting site, improves the monitoring coverage, reduces the monitoring blind spots, and improves the comprehensiveness of on-site safety management; locates the violation identification area according to the hoisting construction progress, and cuts and exports the local video sequence in real time for violation identification. By synchronizing the video to the ground violation analysis model, it can instantly identify and mark risk behaviors, thereby reducing potential safety accidents caused by the failure to timely discover violations; by obtaining the operation monitoring video stream of the construction equipment and performing real-time analysis based on the operation violation identification model, it can effectively identify violations in the operation of the construction equipment, further improving the overall safety of the construction site; by integrating risk behaviors into the construction three-dimensional map, a safety visualization map is generated, which not only helps managers intuitively understand the construction progress and risk distribution, but also can clearly show the specific location and scene of the violation, improving the decision-making efficiency and accuracy. In summary, this method significantly improves the safety monitoring capability of the wind power hoisting site through comprehensive and real-time monitoring, timely violation identification, and intuitive three-dimensional visualization, and provides strong technical support for the safety management of the construction site.

[0086] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional visual safety monitoring method for wind power installation site, characterized in that: The method comprises: Build a three-dimensional construction map based on real-time monitoring of the wind turbine installation site; Interactively monitor the progress of hoisting construction, and locate the illegal identification area in the three-dimensional construction map according to the progress of hoisting construction; Preset the video time domain, take the violation identification area and the video time domain as constraints, and cut and derive a local video sequence from the three-dimensional construction map; Pre-constructing a ground violation analysis model, identifying ground violation behaviors by synchronizing the local video sequence to the ground violation analysis model, and obtaining a first risk behavior set; Extracting and obtaining the components to be hoisted based on the hoisting construction progress, and determining a real-time construction equipment group based on the components to be hoisted, wherein the real-time construction equipment group operates in the violation identification area, and the real-time construction equipment group includes K real-time construction equipment; Interactively obtaining K operation monitoring video streams in the operation rooms of the K real-time construction equipment, and identifying operation violation behaviors based on the K operation monitoring video streams to obtain a second risk behavior set; The first risk behavior set and the second risk behavior set are visually marked on the three-dimensional construction map to generate a safety visualization map.

2. The method according to claim 1, characterized in that Constructing a three-dimensional construction map based on real-time monitoring of the wind power installation site, the method further includes: Interactively obtain the construction boundary of the wind power installation site, and collect data based on the construction boundary to obtain the installation site construction information; Taking the construction boundary as a constraint, three-dimensional modeling is performed based on the construction information of the hoisting site to obtain a twin model of the hoisting site; The real-time video stream array collected by the monitoring camera array pre-configured at the wind turbine hoisting site is synchronized to the twin model of the hoisting site to obtain a three-dimensional construction map.

3. The method according to claim 2, characterized in that The real-time video stream array acquired by the monitoring camera array pre-configured at the wind turbine hoisting site is synchronized to the twin model of the hoisting site to obtain a three-dimensional construction map. The method further includes: Interactively obtain the original monitoring layout of the wind turbine installation site, and call multiple historical video streams of multiple initial cameras in the original monitoring layout; Fitting the multiple historical video streams to the twin model of the hoisting site to obtain N monitoring blind areas; Based on the construction information of the hoisting site, obtain the N construction height information of the N monitoring blind areas; Performing monitoring coverage analysis on the N monitoring blind areas according to the N building height information, and locating N newly added monitoring sites; The original monitoring layout is updated based on the N newly added monitoring locations to obtain the monitoring camera array.

4. The method according to claim 1, characterized in that Pre-building a ground violation analysis model, the method further comprises: Interactively obtain multiple sample static behavior image sets of multiple static construction risk behaviors; Taking the video time domain as a constraint, interactively obtaining multiple sample dynamic behavior video sets of multiple dynamic construction risk behaviors; Building multiple static risk identification branches based on the multiple static construction risk behaviors and multiple sample static behavior image sets; Building multiple dynamic risk identification branches based on the multiple dynamic construction risk behaviors and multiple sample dynamic behavior video sets; Connecting the plurality of static risk identification branches in parallel to obtain a static violation analysis channel; Connecting the multiple dynamic risk identification branches in parallel to obtain a dynamic violation analysis channel; The static violation analysis channel and the dynamic violation analysis channel are connected in parallel to complete the construction of the ground violation analysis model.

5. The method according to claim 4, characterized in that The method further comprises: synchronizing the local video sequence to the ground violation analysis model to identify ground violation behaviors to obtain a first risk behavior set. Decomposing the local video sequence into strips to obtain M behavior videos of M ground operation users, wherein the video duration of each behavior video in the M behavior videos satisfies the video time domain, and M is a positive integer; Extract key frames from the M behavior videos to obtain M risk identification images; Extracting and obtaining a first behavior video and a first risk identification image of a first ground operation user based on the M behavior videos and the M risk identification images, wherein the first ground operation user is any one of the M ground operation users; Synchronize the first behavior video and the first risk identification image to the static violation analysis channel and the dynamic violation analysis channel of the ground violation analysis model respectively, perform multivariate identification analysis of violation risks, and obtain a first violation analysis result; Update information of the first risk behavior set according to the first violation analysis result; By analogy, the first risk behavior set is obtained by synchronizing the local video sequence to the ground violation analysis model to perform ground violation behavior identification.

6. The method according to claim 5, characterized in that The method further includes updating information of the first risk behavior set according to the first violation analysis result: If the first violation analysis result is an empty set, formatting the first violation analysis result; If the first violation analysis result is a non-empty set, the first behavior video, the first risk identification image and the first violation analysis result are stored in association, and the first ground operation user is used as a storage identifier as the first ground violation behavior; The first ground violation behavior is updated to the first risk behavior set.

7. The method according to claim 1, characterized in that Interactively obtaining K operation monitoring video streams in the operation rooms of the K real-time construction equipment, and identifying operation violation behaviors based on the K operation monitoring video streams to obtain a second risk behavior set, the method further comprising: Taking the video time domain as a constraint, interactively obtaining K sample operation violation video sets of the K real-time construction equipment; Constructing K operation violation recognition models based on the K sample operation violation video sets, and deploying the K operation violation recognition models on K local servers of the K real-time construction equipment; The K operation monitoring video streams obtained interactively are deframed based on the video time domain as a constraint to obtain K groups of time-series operation monitoring videos; The K groups of time-series operation monitoring video mappings are synchronized to the K local servers, and behavior recognition is performed in the K local servers via the K operation violation recognition models to obtain the second risk behavior set.

8. The method according to claim 7, characterized in that Synchronizing the K groups of sequential operation monitoring video mappings to the K local servers, performing behavior recognition in the K local servers using the K operation violation recognition models to obtain the second risk behavior set, the method further comprising: Synchronize the K groups of sequential operation monitoring video mappings to the K local servers, perform behavior recognition in the K local servers via the K operation violation recognition models, and obtain K violation recognition sequences; Calculate the violation duration based on the K violation identification sequences to obtain K violation duration information; A violation risk threshold is preset, and the violation risk threshold and K violation duration information are used to screen and obtain H real-time construction equipment, where H is a positive integer less than or equal to K; Based on the H real-time construction equipment and the K violation identification sequences, H violation operation dynamic images are extracted from the K groups of sequential operation monitoring videos; The H real-time construction equipment are used to identify the H illegal operation dynamic images as the second risk behavior set.

Images