Mobile tower crane border crossing detection method and device

Abstract

The embodiment of the invention provides a mobile tower crane border crossing detection method and device. Tower crane design drawing data are imported into a virtual simulation engine, a three-dimensional model is constructed, and warning tape marks are set. A physical engine is creatively utilized to simulate tower crane moving tracks under different working conditions, and a border crossing behavior data set is generated based on intersection-to-union ratio calculation. A deep neural network structure is constructed, a trunk feature extraction module and a target detection module are included, and model parameters are optimized through a cross entropy loss function. According to the method, the limitation of traditional mechanical limiting and manual monitoring is broken through, real-time intelligent monitoring of the movable boundary of the tower crane is achieved, and reliable technical support is provided for safety management of a construction site.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method and device for detecting a mobile tower crane crossing a boundary. Background Art

[0002] Mobile tower cranes are widely used in construction, but the control of their activity boundaries has always been a difficult point in engineering safety management. Existing tower crane cross-border protection mainly relies on manual supervision or simple mechanical limit devices, which have monitoring blind spots and are difficult to cope with complex construction environments. Traditional monitoring methods often use fixed sensors or simple image recognition methods, which are easily affected by factors such as occlusion and lighting changes during the actual movement of the tower crane, resulting in insufficient detection accuracy. At the same time, due to the lack of effective data collection and training samples, the existing intelligent monitoring system is difficult to accurately identify cross-border behaviors under various working conditions.

[0003] Especially in large-scale engineering projects, where multiple tower cranes work together, traditional cross-boundary protection solutions cannot effectively warn of potential safety risks. Existing technologies are difficult to meet the actual needs of construction sites in terms of real-time, accuracy and reliability, and there is an urgent need to develop more advanced cross-boundary detection technology solutions. This is not only related to the safety of life and property of construction workers, but also an important part of improving the level of intelligent engineering construction.

[0004] Therefore, how to achieve accurate monitoring and timely warning of the activity boundaries of tower cranes and build an intelligent cross-boundary detection system is a technical problem that urgently needs to be solved in the current field of construction safety management. Summary of the invention

[0005] In response to the problems in the prior art, the present application provides a mobile tower crane out-of-bounds detection method and device, which can realize real-time intelligent monitoring of the tower crane's activity boundary and provide reliable technical support for construction site safety management.

[0006] In order to solve at least one of the above problems, the present application provides the following technical solutions: In a first aspect, the present application provides a mobile tower crane out-of-bounds detection method, comprising: Importing the dimension data in the tower crane design drawing into the virtual simulation engine, the dimension data includes the length, width, height of the tower crane crawler and the height of the control platform, constructing a three-dimensional tower crane model in the virtual simulation engine, planning the tower crane activity boundary area according to the on-site layout requirements, setting a warning belt mark in the boundary area, and configuring the acquisition viewing angle and installation position parameters of multiple virtual monitoring cameras; The tower crane model posture is adjusted to simulate the moving trajectory sequence of the tower crane under different working conditions, the rolling state and the motion state of the tower crane crawler are calculated based on the physical engine, and multiple sets of tower crane moving video sequences are generated. The intersection-and-union ratio of the tower crane area annotation box and the warning belt area annotation box is calculated, and when the intersection-and-union ratio is greater than a preset threshold, the corresponding video frame is marked as an out-of-bounds state, and a tower crane out-of-bounds behavior data set is constructed, and the data set includes the tower crane area annotation box, the warning belt area annotation box and the intersection-and-union ratio; A deep neural network including a backbone feature extraction network and a target detection network is constructed, the data set is input into the deep neural network for training, the parameters of the deep neural network are optimized using a cross entropy loss function, a tower crane out-of-bounds detection model is obtained, the model is deployed to an on-site monitoring system, the collected real-time monitoring images are analyzed based on the model, the real-time intersection-and-union ratio of the detected tower crane area and the warning belt area is calculated, and an out-of-bounds alarm signal is output when the real-time intersection-and-union ratio is greater than the preset threshold.

[0007] Furthermore, the dimension data in the tower crane design drawing is imported into the virtual simulation engine, the dimension data includes the length, width, height of the tower crane crawler and the height of the control platform, and a three-dimensional tower crane model is constructed in the virtual simulation engine, including: Read the two-dimensional structural data of the tower crane design drawing, extract the length parameters, width parameters, height parameters of the tower crane crawler and the height parameters of the control platform, as well as the connection relationship data of the tower crane main structure, and convert the parameters and connection relationship data into a data format supported by the virtual simulation engine; Based on the track parameters and the main body parameters, a geometric skeleton of the tower crane model is constructed in the three-dimensional coordinate system of the virtual simulation engine, material mapping and physical property parameters are added to each component of the tower crane, physical connection constraints and movement freedom restrictions are established between the tower crane components, and the construction of the three-dimensional tower crane model is completed in the virtual simulation engine.

[0008] Furthermore, the tower crane activity boundary area is planned according to the on-site deployment requirements, a warning zone mark is set in the boundary area, and the acquisition viewing angles and installation position parameters of multiple virtual monitoring cameras are configured, including: According to the terrain data and operation scope requirements of the construction site, the coordinates of the contour points of the tower crane activity boundary are calibrated in the three-dimensional scene of the virtual simulation engine, a closed boundary curve is generated through the contour points, a three-dimensional warning belt model with height parameters is created along the boundary curve, and reflective material and color attributes are configured for the warning belt model; Arrange multiple surveillance camera models in a virtual three-dimensional scene, set the focal length parameters, field of view angle parameters and resolution parameters of each surveillance camera, adjust the camera's installation height and orientation angle to ensure that the camera's field of view covers the tower crane activity area and the warning zone area, and calculate the spatial projection matrix of each surveillance camera.

[0009] Furthermore, the tower crane model posture is adjusted to simulate the moving trajectory sequence of the tower crane under different working conditions, the rolling state and the motion state of the tower crane crawler are calculated based on the physical engine, and multiple sets of tower crane moving video sequences are generated, including: Generate a displacement trajectory point sequence of the tower crane model based on the construction condition data, apply driving force and motion constraints to the tower crane model, calculate the contact force and friction force between the tower crane crawler and the ground through the physical engine, update the position coordinates and attitude angle of the tower crane model according to the mechanical equation, and calculate the rolling speed of the tower crane crawler and the deformation state of the crawler segment; Image frames of the tower crane model during its movement are collected from the perspective of the surveillance camera, the image frames are organized into a video sequence in time sequence, the spatial position relationship between the tower crane model and the warning tape model in each image frame is recorded, and the degree of overlap between the projection area of ​​the tower crane model and the projection area of ​​the warning tape model on the image plane is calculated based on the spatial position relationship.

[0010] Furthermore, the intersection-and-union ratio of the tower crane area annotation box and the warning zone area annotation box is calculated, and when the intersection-and-union ratio is greater than a preset threshold, the corresponding video frame is marked as an out-of-bounds state, and a tower crane out-of-bounds behavior data set is constructed, and the data set includes the tower crane area annotation box, the warning zone area annotation box and the intersection-and-union ratio, including: Performing semantic segmentation on each frame image in the video sequence, extracting the contour boundary points of the tower crane model and the contour boundary points of the warning belt model, generating a minimum circumscribed rectangle as a labeling box based on the boundary points, calculating the overlapping area and the union area of ​​the tower crane area labeling box and the warning belt area labeling box, and dividing the overlapping area by the union area to obtain an intersection-union ratio; The video sequence is marked according to a preset intersection-and-union ratio threshold, the coordinates of the tower crane area annotation box, the coordinates of the warning belt area annotation box and the corresponding intersection-and-union ratio of each frame image are stored in the data set, a standardized data label file is generated for each frame image, and the image file and the label file are divided and stored according to the ratio of the training set and the verification set.

[0011] Furthermore, the construction of a deep neural network including a backbone feature extraction network and a target detection network, inputting the data set into the deep neural network for training, and optimizing the parameters of the deep neural network using a cross entropy loss function to obtain a tower crane out-of-bounds detection model includes: A dual-branch neural network structure is built. The first branch is a backbone feature extraction network, which includes multiple convolutional layers, pooling layers, and activation function layers, and is used to extract multi-scale feature maps of the image. The second branch is a target detection network, which includes a region candidate layer and a bounding box regression layer. The feature map is input into the target detection network to generate detection box prediction values ​​for the crane area and the warning zone area, and the prediction values ​​are compared with the true values ​​of the labeled boxes in the data set. The error between the predicted value and the true value of the detection box is calculated based on the cross entropy loss function, the error is back-propagated to each layer of the neural network, the network weight parameters are updated by the gradient descent method, the training set data is iterated for multiple rounds of training, and the model performance is evaluated using the validation set data after each round of training. When the loss value on the validation set converges, the training is stopped, and the network parameters are derived to obtain the tower crane out-of-bounds detection model.

[0012] Further, the model is deployed to the on-site monitoring system, and the collected real-time monitoring images are analyzed based on the model, and the real-time intersection-and-combination ratio of the detected tower crane area and the warning zone area is calculated, and when the real-time intersection-and-combination ratio is greater than the preset threshold, an out-of-bounds alarm signal is output, including: The tower crane crossing detection model is deployed to the edge computing unit of the monitoring system, a data communication link between the monitoring camera and the edge computing unit is established, the real-time image stream collected by the monitoring camera is received, the image stream is preprocessed and normalized, and the preprocessed image is sent to the feature extraction network and target detection network of the model to obtain the position coordinates of the tower crane area and the warning belt area in the real-time image; The real-time intersection-and-union ratio of the tower crane area and the warning zone area is calculated based on the position coordinates, and the real-time intersection-and-union ratio is compared with a preset threshold. When it is detected that the real-time intersection-and-union ratio exceeds the preset threshold for multiple consecutive frames, an out-of-bounds alarm signal is sent to the on-site control center through the communication interface of the monitoring system, and the image frame, timestamp and intersection-and-union ratio data that triggers the alarm are recorded in the system log.

[0013] In a second aspect, the present application provides a mobile tower crane crossing detection device, comprising: A virtual simulation module is used to import the dimension data in the tower crane design drawing into the virtual simulation engine, wherein the dimension data includes the length, width, height of the tower crane crawler and the height of the control platform, construct a three-dimensional tower crane model in the virtual simulation engine, plan the tower crane activity boundary area according to the on-site layout requirements, set a warning belt mark in the boundary area, and configure the acquisition viewing angle and installation position parameters of multiple virtual monitoring cameras; A data monitoring module is used to adjust the posture of the tower crane model, simulate the moving trajectory sequence of the tower crane under different working conditions, calculate the rolling state and movement state of the tower crane crawler based on the physical engine, generate multiple sets of tower crane movement video sequences, calculate the intersection-and-union ratio of the tower crane area annotation box and the warning belt area annotation box, and mark the corresponding video frame as an out-of-bounds state when the intersection-and-union ratio is greater than a preset threshold, and construct a tower crane out-of-bounds behavior data set, which includes the tower crane area annotation box, the warning belt area annotation box and the intersection-and-union ratio; The cross-border alarm module is used to construct a deep neural network including a backbone feature extraction network and a target detection network, input the data set into the deep neural network for training, use the cross entropy loss function to optimize the parameters of the deep neural network, obtain a tower crane cross-border detection model, deploy the model to the on-site monitoring system, analyze the collected real-time monitoring images based on the model, calculate the real-time intersection-and-union ratio of the detected tower crane area and the warning belt area, and output a cross-border alarm signal when the real-time intersection-and-union ratio is greater than the preset threshold.

[0014] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the mobile tower crane out-of-bounds detection method and device when executing the program.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the mobile tower crane out-of-bounds detection method.

[0016] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the mobile tower crane out-of-bounds detection method.

[0017] It can be seen from the above technical solution that the present application provides a method and device for detecting out-of-bounds of a mobile tower crane, by importing the tower crane design drawing data into a virtual simulation engine, constructing a three-dimensional model and setting a warning zone mark. The physical engine is innovatively used to simulate the movement trajectory of the tower crane under different working conditions, and an out-of-bounds behavior data set is generated based on the intersection-and-union ratio calculation. A deep neural network structure is constructed, including backbone feature extraction and target detection modules, and the model parameters are optimized through the cross-entropy loss function. This method breaks through the limitations of traditional mechanical limits and manual supervision, realizes real-time intelligent monitoring of the activity boundary of the tower crane, and provides reliable technical support for construction site safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is one of the flow charts of the mobile tower crane crossing-boundary detection method in the embodiment of the present application; Figure 2 The second flowchart of the mobile tower crane crossing-boundary detection method in the embodiment of the present application; Figure 3 The third flowchart of the mobile tower crane crossing-boundary detection method in the embodiment of the present application; Figure 4 The fourth flowchart of the mobile tower crane crossing-boundary detection method in the embodiment of the present application; Figure 5 FIG5 is a flowchart of a mobile tower crane crossing-boundary detection method in an embodiment of the present application; Figure 6 6 is a flow chart of a mobile tower crane crossing-boundary detection method in an embodiment of the present application; Figure 7 FIG7 is a flow chart of a mobile tower crane crossing-boundary detection method in an embodiment of the present application; Figure 8 It is a structural diagram of a mobile tower crane crossing-boundary detection device in an embodiment of the present application; Fig. 9 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

[0020] Reference numerals: Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0023] Taking into account the problems existing in the prior art, the present application provides a method for detecting mobile tower cranes crossing the boundary, by importing the tower crane design drawing data into a virtual simulation engine, constructing a three-dimensional model and setting warning zone marks. The physical engine is innovatively used to simulate the movement trajectory of the tower crane under different working conditions, and an out-of-bounds behavior data set is generated based on the intersection-and-union calculation. A deep neural network structure is constructed, including backbone feature extraction and target detection modules, and the model parameters are optimized through the cross-entropy loss function. This method breaks through the limitations of traditional mechanical limits and manual supervision, realizes real-time intelligent monitoring of the tower crane's activity boundaries, and provides reliable technical support for construction site safety management.

[0024] In order to realize real-time intelligent monitoring of the activity boundary of the tower crane and provide reliable technical support for the safety management of the construction site, the present application provides an embodiment of a mobile tower crane out-of-bounds detection method, see Figure 1 The mobile tower crane out-of-bounds detection method and device specifically include the following contents: Step S101: Importing dimension data in the tower crane design drawing into a virtual simulation engine, the dimension data including the length, width, height of the tower crane crawler and the height of the control platform, constructing a three-dimensional tower crane model in the virtual simulation engine, planning the tower crane activity boundary area according to the on-site layout requirements, setting a warning zone mark in the boundary area, and configuring the acquisition viewing angle and installation position parameters of multiple virtual monitoring cameras; Optionally, this embodiment first reads the design drawings of the tower crane through the CAD data processing module, focusing on the key dimensions of the track assembly and the main structure. For the track part, the length (usually 500-800mm), width (generally 200-300mm), thickness (about 50-80mm) parameters of each track plate, and the outer dimensions of the track assembly are extracted in detail. At the same time, the diameter and spacing of the support wheel and the drive wheel are measured, and these parameters directly affect the motion characteristics of the track. The control platform is located on the top of the tower crane, and its standard height range is between 15-30 meters, and precise measurement is required to ensure the accuracy of the subsequent monitoring angle.

[0025] To ensure the accuracy of data import, a special data conversion protocol is designed in this embodiment. First, the two-dimensional data in CAD format is converted through STEP or IGES intermediate format to retain the assembly relationship and topological structure between parts. The converted data contains complete parametric features, so that parameter-driven model adjustment can be achieved in the virtual simulation engine. In particular, for the track assembly, the kinematic pair type and degree of freedom information between each link is retained.

[0026] In the virtual simulation engine, this embodiment adopts a hierarchical modeling strategy. First, the skeleton system of the tower crane is constructed, including the chassis frame, the track support structure, the column frame and the control platform frame. A complete coordinate system and reference datum are established for each subsystem. Then, the detailed features of the transmission system are added, including the track tensioning mechanism, the guide wheel group, the supporting wheel group, etc. In order to improve the accuracy of the model, the contact surface of the track chain link is refined to ensure that the rolling and deformation states of the track can be accurately simulated.

[0027] When planning the activity boundary of the tower crane, this embodiment innovatively adopts an adaptive boundary generation algorithm. First, the three-dimensional terrain data of the construction site is imported, including information such as ground elevation and slope changes. Combined with the construction process requirements, multiple key control points are calibrated in the three-dimensional scene. These points include: the vertices of the working area, obstacle boundary points, and other equipment working range boundary points. These control points are fitted by B-spline curves to generate smooth boundary curves.

[0028] The warning tape system adopts a multi-level protection design. A basic warning line is set on the ground layer, with a fluorescent yellow and black stripe texture, a stripe width of 200mm, and an inclination angle of 45 degrees. The height of the vertical extension warning tape is set to 2 meters, and reflective identification tapes are set at different heights. The material adopts the PBR (physical-based rendering) workflow to ensure that the reflective effect can be truly restored in the virtual environment. According to the level of danger, the warning area is divided into three levels: red (no-entry zone), yellow (warning zone) and green (safe zone).

[0029] This embodiment develops an intelligent point placement algorithm when configuring virtual surveillance cameras. First, a uniformly distributed candidate point cloud is generated in the three-dimensional scene, and each candidate point contains an adjustable installation height (8-12 meters) and three rotational degrees of freedom. Through field of view analysis and coverage optimization algorithm, the monitoring effect score of each candidate point is calculated. The scoring factors include: visibility of the warning belt, coverage of the tower crane activity area, overlap of the field of view between cameras, etc.

[0030] The camera parameter configuration adopts the specifications commonly used in actual projects, including: 1 / 1.8-inch sensor, 2880×1620 resolution, 4-8mm zoom lens, horizontal field of view 87°-46°. When determining the installation angle, the change in the angle of sunlight is fully considered to avoid backlight affecting the image quality. At the same time, the camera's pitch angle is controlled between 15°-30° to balance the monitoring range and image distortion.

[0031] Through the above detailed virtual simulation modeling process, this embodiment establishes a high-fidelity test environment for subsequent data collection and model training. This digital twin solution not only improves the planning efficiency of the monitoring system, but also provides a reliable simulation platform for subsequent algorithm development. Practice has shown that the monitoring solution pre-verified based on the virtual environment can significantly reduce the workload of on-site installation and debugging, while ensuring the monitoring blind spots of the cross-border detection system.

[0032] Step S102: adjusting the posture of the tower crane model, simulating the moving trajectory sequence of the tower crane under different working conditions, calculating the rolling state and the motion state of the tower crane crawler based on the physical engine, generating multiple sets of tower crane moving video sequences, calculating the intersection-and-union ratio of the tower crane area annotation box and the warning belt area annotation box, marking the corresponding video frame as an out-of-bounds state when the intersection-and-union ratio is greater than a preset threshold, and constructing a tower crane out-of-bounds behavior data set, the data set including the tower crane area annotation box, the warning belt area annotation box and the intersection-and-union ratio; Optionally, this embodiment adopts an inverse kinematics algorithm for adjusting the posture of the tower crane under different working conditions. First, a multi-body dynamics model of the tower crane is established, and the whole machine is decomposed into key components such as the crawler system, the rotary platform, and the boom system. Each component is set with corresponding mass parameters and moment of inertia, and kinematic constraints are established at the connection. By defining the joint angle and displacement parameters, the relative motion of each component of the tower crane can be accurately controlled.

[0033] When simulating the moving trajectory of the tower crane, this embodiment designs a variety of working conditions. Including basic working conditions such as straight-line walking, turning walking, and slope walking, as well as complex working conditions such as load lifting and boom rotation. For each working condition, the motion parameter curve is planned in detail, such as crawler speed, steering angle, acceleration and deceleration, etc. Especially in the steering process, considering the special kinematic characteristics of the crawler tower crane, steering is achieved through differential control, which is consistent with the operation method in actual engineering.

[0034] This embodiment focuses on the contact dynamics calculation between the track and the ground in the physics engine. The discrete element method is used to simulate the contact process between the track link and the ground, and each link is set with a reasonable friction coefficient and elastic coefficient. The contact area is refined through meshing, and the distribution of contact force and friction force is calculated in real time. During the rolling process of the track, the deformation and relaxation effects of the link are considered to make the simulation results closer to the actual working conditions.

[0035] In order to generate high-quality video sequences, this embodiment synchronously collects images from multiple virtual camera perspectives. Appropriate lighting conditions and rendering parameters are set for each perspective to ensure the clarity and realism of the image. A moderate motion blur effect is added during the rendering process to simulate the imaging characteristics of the camera when tracking a moving object. At the same time, the visual effects under different weather conditions, such as cloudy, sunny, rainy, etc., are considered to increase the diversity of the data set.

[0036] This embodiment innovatively designs an automatic labeling algorithm. In each frame of the image, the tower crane model and the warning zone boundary in the three-dimensional space are mapped to the two-dimensional image plane through projection transformation. Based on the projection results, the minimum circumscribed rectangle is generated as the labeling box. In order to improve the labeling accuracy, the labeling box of the tower crane is dynamically adjusted to ensure that it can completely surround the moving parts of the tower crane.

[0037] When calculating the intersection-and-union ratio, this embodiment adopts an efficient geometric operation method. First, the annotation box is converted into a polygonal representation, and the intersection area and union area of ​​the two polygons are calculated. Considering the safety margin requirements in actual engineering, the intersection-and-union ratio threshold is set as an adjustable parameter, usually between 0.1 and 0.3. When an out-of-bounds state is detected, the relevant scene parameters are automatically recorded, including the position, posture, and moving speed of the tower crane.

[0038] The data set constructed in this embodiment is organized in a hierarchical structure. Each sample contains three levels of data: original image, annotation information, and scene parameters. The annotation information not only contains the coordinates of the bounding box, but also records the quantitative index of the degree of crossing the boundary. Through this structured data organization method, it is convenient for the subsequent training algorithm to accurately identify different degrees of crossing the boundary.

[0039] This data set generation solution based on physical simulation effectively solves the problem of difficulty in obtaining sufficient out-of-bounds samples in actual engineering. Through virtual simulation technology, it is easy to construct out-of-bounds scenarios under various extreme conditions, providing sufficient training data for subsequent detection algorithms. At the same time, the introduction of the physical engine ensures the physical rationality of the generated data and improves the quality and reliability of the training data.

[0040] Step S103: construct a deep neural network including a backbone feature extraction network and a target detection network, input the data set into the deep neural network for training, use the cross entropy loss function to optimize the parameters of the deep neural network, obtain a tower crane out-of-bounds detection model, deploy the model to the on-site monitoring system, analyze the collected real-time monitoring images based on the model, calculate the real-time intersection-and-union ratio of the detected tower crane area and the warning belt area, and output an out-of-bounds alarm signal when the real-time intersection-and-union ratio is greater than the preset threshold.

[0041] Optionally, this embodiment uses a dual-branch feature extraction structure to construct a deep neural network, and the backbone network uses an improved ResNet architecture to enhance the ability to extract tower crane contour features by introducing a spatial attention mechanism. When designing the network, taking into account the particularity of tower crane equipment, a multi-scale receptive field convolution unit is added to the shallow feature extraction module to effectively capture tower crane feature information at different scales.

[0042] For the design of the target detection network, this embodiment innovatively adopts a feature pyramid structure. Anchor frames of different sizes are set on feature maps of different levels, specifically for detecting tower cranes and warning belt areas. Considering the possible scale changes of tower cranes in images, the size range of the anchor frame is set to 32×32 to 512×512 pixels, and the aspect ratios include 1:1, 1:2, and 2:1 to adapt to the morphological characteristics of tower cranes from different perspectives.

[0043] During the training process, this embodiment designs a targeted data enhancement strategy. In addition to conventional transformations such as flipping, rotation, and scaling, random occlusion, brightness adjustment, and contrast change are also introduced. Especially for nighttime working conditions, different degrees of insufficient light are simulated to improve the detection ability of the model in low-light environments. At the same time, by adding Gaussian noise and motion blur, the model's adaptability to fluctuations in monitoring image quality is enhanced.

[0044] The loss function is designed using a multi-task learning framework. In addition to the basic detection box regression loss and category classification loss, an innovative intersection-over-union prediction branch is introduced. This branch directly learns to predict the intersection-over-union ratio between the crane and the warning tape, providing a more accurate assessment of the degree of crossing the boundary. The weights of each part in the loss function are determined through grid search on the validation set to balance the training objectives of different tasks.

[0045] This embodiment adopts a phased training strategy when optimizing network parameters. First, the backbone network is fixed and only the detection head part is trained. After convergence, the overall fine-tuning is performed. The learning rate adopts a cosine annealing strategy, with an initial value set to 0.001, and a periodic adjustment mechanism with hot restart. To prevent overfitting, Dropout and L2 regularization techniques are applied during the training process, and an early stopping strategy is used to monitor the performance of the validation set.

[0046] In the model deployment phase, this embodiment performs network compression and acceleration optimization based on the hardware conditions of the actual monitoring system. Through channel pruning and knowledge distillation technology, the model calculation amount is reduced while maintaining detection accuracy. At the same time, TensorRT is used for reasoning acceleration to achieve the performance requirements of real-time detection.

[0047] In order to improve the reliability of field applications, this embodiment designs a real-time evaluation mechanism. The system tracks the changing trend of the intersection-and-union ratio in continuous multi-frame detection, and reduces false alarms caused by instantaneous detection fluctuations through sliding window averaging and threshold smoothing. When it is detected that the average intersection-and-union ratio of continuous multi-frames exceeds the preset threshold, the system will trigger a graded alarm mechanism.

[0048] The alarm signal is processed using a multi-level response strategy. When the intersection ratio is close to but does not exceed the threshold, the system issues a warning prompt; when it is confirmed that a cross-border has occurred, an audible and visual alarm is immediately triggered, and the cross-border image, timestamp and related parameters are recorded and archived to provide a basis for subsequent accident analysis.

[0049] This embodiment uses deep learning technology to achieve intelligent detection of tower crane out-of-bounds behavior, effectively solving the problem of fatigue and omissions in traditional manual monitoring. This solution has shown good environmental adaptability and detection reliability in actual engineering applications, can monitor the operation status of tower cranes around the clock, and promptly warn of potential safety hazards, providing strong technical support for safety management at construction sites.

[0050] From the above description, it can be seen that the mobile tower crane out-of-bounds detection method provided in the embodiment of the present application can construct a three-dimensional model and set warning belt marks by importing the tower crane design drawing data into the virtual simulation engine. The physical engine is innovatively used to simulate the movement trajectory of the tower crane under different working conditions, and an out-of-bounds behavior data set is generated based on the intersection and union ratio calculation. A deep neural network structure is constructed, including a backbone feature extraction and target detection module, and the model parameters are optimized through a cross-entropy loss function. This method breaks through the limitations of traditional mechanical limits and manual supervision, realizes real-time intelligent monitoring of the tower crane's activity boundaries, and provides reliable technical support for construction site safety management.

[0051] In one embodiment of the mobile tower crane crossing-boundary detection method of the present application, see Figure 2 , and can also include the following: Step S201: reading the two-dimensional structure data of the tower crane design drawing, extracting the length parameters, width parameters, height parameters of the tower crane crawler and the height parameters of the control platform, as well as the connection relationship data of the tower crane main structure, and converting the parameters and connection relationship data into a data format supported by the virtual simulation engine; Step S202: constructing a geometric skeleton of the tower crane model in the three-dimensional coordinate system of the virtual simulation engine based on the track parameters and the main body parameters, adding material mapping and physical property parameters to each component of the tower crane, establishing physical connection constraints and movement freedom restrictions between the tower crane components, and completing the construction of the three-dimensional tower crane model in the virtual simulation engine.

[0052] Optionally, this embodiment first develops a dedicated parsing module for reading the data of the tower crane design drawing. This module can identify the layer information in the CAD drawing and classify and extract data such as the structural wireframe, dimensioning and assembly relationship. For the crawler system, the geometric parameters of each track plate are extracted, including the length of the single plate (usually 500-800mm), width (200-300mm) and height (50-80mm), as well as the type of connection between the crawler links. At the same time, the position coordinates and diameter parameters of the drive wheel and guide wheel are recorded, which directly affect the motion characteristics of the crawler system.

[0053] In the data conversion link, this embodiment adopts a hierarchical data structure design. First, a part-level parametric model is established, and the geometric features of each component are converted into a feature tree form. For complex surfaces, the NURBS surface description method is used to ensure the accuracy of the model. The assembly relationship data is stored in a hierarchical tree structure, which records the parent-child relationship and constraint type between parts. All data is converted into a file format that can be recognized by the virtual simulation engine through an intermediate format.

[0054] This embodiment adopts a bottom-up modeling strategy when constructing the geometric skeleton. First, a global coordinate system is established in the virtual space, and a local coordinate system is established based on the center line of the track system. Through the parametric modeling method, the track plate is quickly generated and assembled according to the design parameters. Each track plate is set with a precise rotation axis to ensure that real rolling motion can be achieved in subsequent dynamic simulations.

[0055] When adding material maps, this embodiment selects appropriate rendering schemes according to the material characteristics of different components. For metal surfaces, a physically based rendering (PBR) material is used, and appropriate roughness and metalness parameters are set to achieve realistic metal texture. The track surface uses a material map with a worn texture to increase the realism of the model. At the same time, warning signs and numbered labels are added at key locations for easy identification.

[0056] The configuration of physical property parameters is an innovation of this embodiment. Physical parameters such as mass, density, and elastic modulus are set for each component according to its actual material properties. In particular, for the track system, appropriate friction coefficients and damping parameters are set in consideration of its dynamic characteristics during motion. The accurate configuration of these parameters is crucial to the authenticity of subsequent motion simulation.

[0057] When establishing physical connection constraints, this embodiment adopts a multi-level constraint system. The track links are connected by hinge constraints, and each constraint is set with appropriate rotation damping and angle limit. A composite constraint is set between the support wheel and the track to ensure rolling contact and limit lateral displacement. The connection of the main structure adopts rigid constraints to ensure the stability of the whole machine.

[0058] This embodiment pays special attention to the reasonable setting of the degrees of freedom of motion. For the slewing mechanism, the degree of freedom of rotation around the vertical axis is set, and its speed and acceleration are controlled by the motor drive parameters. The pitch degree of freedom is set for the boom system, and the lifting action is realized in conjunction with the motion parameters of the hydraulic cylinder. Through this precise degree of freedom control, it is ensured that the virtual model can accurately reflect the motion characteristics of the actual tower crane.

[0059] This parametric virtual modeling method not only improves modeling efficiency, but also ensures the accuracy and modifiability of the model. Through reasonable physical parameter configuration and constraint setting, it lays a solid foundation for subsequent dynamic simulation and can truly reflect the motion state of the tower crane under various working conditions. At the same time, the modular design of the model is also convenient for later maintenance and updating, and has strong practicality and scalability.

[0060] In one embodiment of the mobile tower crane crossing-boundary detection method of the present application, see Figure 3 , and can also include the following: Step S301: according to the terrain data of the construction site and the requirements of the working range, the coordinates of the contour points of the tower crane activity boundary are calibrated in the three-dimensional scene of the virtual simulation engine, a closed boundary curve is generated through the contour points, a three-dimensional warning belt model with height parameters is created along the boundary curve, and reflective material and color attributes are configured for the warning belt model; Step S302: Arrange multiple surveillance camera models in the virtual three-dimensional scene, set the focal length parameters, field of view angle parameters and resolution parameters of each surveillance camera, adjust the installation height and orientation angle of the camera to ensure that the camera's field of view covers the tower crane activity area and the warning zone area, and calculate the spatial projection matrix of each surveillance camera.

[0061] Optionally, this embodiment first establishes a terrain model based on the three-dimensional terrain data of the construction site. The point cloud data obtained by laser measurement or drone mapping uses triangular mesh reconstruction technology to generate an accurate terrain surface. On the terrain model, the safe operating range of the tower crane is calibrated according to the construction specifications and the actual situation on site. The calibration process adopts a three-dimensional space point selection method, and professionals determine the spatial coordinates of key boundary points in the virtual scene according to the construction plan.

[0062] In order to generate a smooth boundary curve, this embodiment uses a B-spline curve interpolation algorithm. By adjusting the weight coefficient of the control point, the continuity and smoothness of the boundary curve are ensured. After the curve is generated, the height parameter of the warning belt is set according to the safety regulations of the construction site, usually in the range of 1.2 meters to 2 meters above the ground. The curve stretching method is used to create a three-dimensional warning belt model with thickness along the boundary curve.

[0063] The material setting of the warning tape is the key to ensure good visibility under various lighting conditions. This embodiment adopts a layered material structure. The bottom layer is a fluorescent material to provide a basic eye-catching effect, and the surface layer is a micro-prismatic reflective material, which can form a significant reflective effect in a weak light environment. The color matching of the material follows the safety warning standard, using red and white or yellow and black stripes, and the stripe width and spacing are set according to standard specifications.

[0064] When arranging surveillance cameras, this embodiment develops a camera position optimization algorithm based on visibility analysis. First, several candidate installation points are set at the site boundary, and for each installation point, its visible coverage of the tower crane activity area is calculated. The position and orientation parameters of the camera are optimized through a genetic algorithm, so that the minimum number of cameras can achieve full coverage of the entire surveillance area.

[0065] The camera parameters are set using a reverse engineering method. First, the minimum pixel size of the target object in the image is determined based on the actual monitoring requirements, and then the required camera resolution and focal length are inferred based on the site size. The camera focal length range selected in this embodiment is adjustable between 8-50mm, the field of view angle is optional between 30-90 degrees, and the resolution reaches 1920×1080 or above, ensuring that the outline features of the tower crane can still be clearly captured at the maximum monitoring distance.

[0066] The calculation of the projection matrix is ​​the key to achieve the mapping between the virtual scene and the actual monitoring image. This embodiment adopts a complete camera calibration model, including an intrinsic parameter matrix and an extrinsic parameter matrix. The intrinsic parameter matrix describes the focal length, principal point offset, radial distortion and other parameters of the camera, while the extrinsic parameter matrix describes the position and posture of the camera in the world coordinate system. Through the combination of these two matrices, the mapping relationship from the three-dimensional space point to the image plane can be accurately established.

[0067] In order to adapt to complex lighting environments, this embodiment adds automatic exposure and wide dynamic range (WDR) functions to the camera model. When there is strong light contrast in the scene, multiple exposure synthesis technology is used to ensure that clear image details can still be obtained in the light and dark transition area. At the same time, for night construction scenes, fill light equipment is arranged near the camera, and its lighting effect is simulated in the virtual scene.

[0068] This embodiment provides a reliable data basis for subsequent cross-border detection through precise scene modeling and camera layout. The optimized camera layout scheme not only ensures the elimination of blind spots in monitoring, but also improves image quality through reasonable parameter configuration, effectively improving the accuracy and reliability of cross-border detection. At the same time, the standardized warning belt setting provides an intuitive safety reference for on-site managers, which helps prevent the occurrence of safety accidents.

[0069] In one embodiment of the mobile tower crane crossing-boundary detection method of the present application, see Figure 4 , and can also include the following: Step S401: generating a displacement trajectory point sequence of a tower crane model based on construction condition data, applying driving force and motion constraints to the tower crane model, calculating the contact force and friction force between the tower crane crawler and the ground through a physical engine, updating the position coordinates and attitude angle of the tower crane model according to mechanical equations, and calculating the rolling speed of the tower crane crawler and the deformation state of the crawler segments; Step S402: Collect image frames of the tower crane model during its movement from the perspective of the surveillance camera, organize the image frames into a video sequence in time sequence, record the spatial position relationship between the tower crane model and the warning tape model in each image frame, and calculate the degree of overlap between the projection area of ​​the tower crane model and the projection area of ​​the warning tape model on the image plane based on the spatial position relationship.

[0070] Optionally, this embodiment first uses a trajectory planning algorithm to generate a path point sequence for the tower crane movement according to the construction working conditions. The trajectory planning takes into account the starting point, the end point, and the key path points in the middle, and generates a smooth motion trajectory through quintic polynomial interpolation. This method not only ensures the continuity of the position, but also ensures the continuity of the speed and acceleration, avoiding sudden changes during the movement process.

[0071] In the physics engine, this embodiment adopts a hierarchical dynamic solution method. First, a multi-rigid body dynamic model of the tower crane is established, including the main components such as the track system, frame, and slewing platform. Each component is given an accurate mass distribution and inertia parameter. For the track system, a discretization method is used to discretize the continuous track into multiple rigid blocks, which are connected by hinge constraints. This model not only ensures the calculation efficiency, but also accurately reflects the motion characteristics of the track.

[0072] This embodiment uses an improved LCP (Linear Complementarity Problem) solver in the contact force calculation. For the contact between the track and the ground, a nonlinear contact model based on the Hertz contact theory is established. The contact force between each track shoe and the ground is obtained by iterative calculation, taking into account the coupling effect of normal pressure and tangential friction. The friction force is calculated using the Coulomb friction model, and the conversion characteristics of static friction and dynamic friction are taken into account.

[0073] In the crawler deformation analysis, this embodiment develops a special deformation calculation module. The relative movement between the crawler segments is simulated by the spring-damper unit, and the displacement and deformation of each segment under the load are calculated. For heavy load conditions, the compensation effect of the crawler tensioner is also considered to ensure that the crawler system always maintains appropriate tension.

[0074] The motion state update adopts the explicit integration method, and the time step is adaptively adjusted according to the motion characteristics. At each time step, the external forces and constraints on each component are first calculated, then the acceleration is solved by the Newton-Euler equation, and finally the velocity and position are obtained by numerical integration. The attitude update is represented by quaternions to avoid the singularity problem that may occur in Euler angles.

[0075] In the image acquisition process, this embodiment implements a multi-view synchronous acquisition mechanism. Each virtual camera captures images at a preset frame rate, and automatically compensates for illumination changes and perspective changes during the acquisition process. The image sequence is stored in an efficient data structure, and each frame of the image is accompanied by a timestamp and camera parameter information.

[0076] The calculation of spatial position relationship is an innovation of this embodiment. The tower crane model and the warning tape model in the three-dimensional space are projected onto the two-dimensional image plane through projection transformation. The projection process takes into account the internal and external parameters of the camera to ensure the accuracy of the projection result. In order to improve the calculation efficiency, the strategy of bounding box hierarchical detection is adopted, and a rough judgment is made first, and then the area where overlap may occur is accurately calculated.

[0077] The calculation of the overlap degree uses an improved IoU (intersection over union) algorithm. First, the projection area is converted into a pixel-level binary mask, and then the area of ​​the overlap area is calculated through fast bit operations. Considering the impact of projection deformation, a distance-based weight factor is introduced when calculating the overlap degree, making the evaluation result more in line with the actual situation.

[0078] This embodiment achieves real-time monitoring of the tower crane's motion state and accurate assessment of the risk of crossing the boundary through precise physical simulation and efficient image processing. The dynamics simulation based on the physical engine ensures the authenticity of the motion process, while the image acquisition and processing mechanism of multi-camera collaboration provides a reliable basis for cross-boundary detection. This method can not only warn of potential safety hazards, but also provide detailed data support for construction management.

[0079] In one embodiment of the mobile tower crane crossing-boundary detection method of the present application, see Figure 5 , and can also include the following: Step S501: semantically segment each frame of the video sequence, extract the contour boundary points of the tower crane model and the contour boundary points of the warning belt model, generate a minimum circumscribed rectangle as a labeling box based on the boundary points, calculate the overlapping area and the union area of ​​the tower crane area labeling box and the warning belt area labeling box, and divide the overlapping area by the union area to obtain an intersection-union ratio; Step S502: Mark the video sequence according to a preset intersection-and-union ratio threshold, store the tower crane area annotation box coordinates, warning zone area annotation box coordinates and the corresponding intersection-and-union ratio of each frame image into a data set, generate a standardized data label file for each frame image, and divide and store the image file and label file according to the ratio of training set and verification set.

[0080] Optionally, this embodiment first performs semantic segmentation on images in a video sequence by deep learning. A segmentation network with a U-Net architecture is used to extract multi-scale features of the image through an encoder-decoder structure. The encoder uses a combination of convolutional layers with residual connections to extract feature maps from low-level textures to high-level semantics layer by layer. The decoder restores feature maps of different scales to segmentation masks of the same resolution as the input image through transposed convolution and feature fusion.

[0081] In the contour extraction stage, this embodiment designs a dual-threshold edge detection algorithm. First, morphological operations are applied to the binary mask obtained by segmentation to remove noise and small fractures. Then, the improved Canny operator is used to extract the contour. The low threshold is used to retain weak but continuous edges, and the high threshold is used to determine the main contour. For targets with complex structures such as tower cranes, special attention is paid to the contour integrity of key parts such as the boom and tower body.

[0082] This embodiment uses an improved rotating rectangle algorithm when generating the minimum circumscribed rectangle. The main direction of the target is obtained through principal component analysis, and then the minimum area rectangle that can completely surround the target is found based on the convex hull algorithm. This method not only takes into account the spatial directionality of the target, but also minimizes redundant areas and improves the accuracy of subsequent intersection-over-union calculations.

[0083] For the coordinate representation of the annotation box, this embodiment adopts a normalized coordinate system. The width and height of the image are normalized to 1, and the position and size of the annotation box are represented by relative coordinates. This representation method is not only convenient for data migration and expansion, but also can adapt to image inputs of different resolutions.

[0084] When calculating the intersection-over-union ratio, this embodiment develops an efficient geometric operation module. First, the vector cross product is used to determine whether the two rectangles intersect. If they do intersect, the area of ​​the overlapping area is accurately calculated using a polygon clipping algorithm. Considering the possible perspective distortion in practical applications, a correction factor based on projection transformation is introduced when calculating the area.

[0085] The dataset is constructed using a stratified sampling strategy. First, samples are divided into different levels according to the intersection-over-union ratio, and then samples are randomly selected in each level according to a fixed ratio to be divided into a training set and a validation set. This method ensures that samples with different degrees of overlap are evenly distributed in the dataset, which is conducive to the robustness training of the model.

[0086] The standardized data labels adopt a unified format specification, including basic information of the image, location parameters of the annotation box, category information and intersection-over-union ratio. The label file adopts a hierarchical structure to facilitate subsequent batch processing and model training. To improve the availability of data, this embodiment also implements an automatic conversion tool for label formats to support the data format requirements of mainstream deep learning frameworks.

[0087] This embodiment provides a high-quality data foundation for subsequent deep learning model training through a sophisticated semantic segmentation and annotation box generation mechanism. The improved intersection-over-union calculation method improves the accuracy of out-of-bounds judgment, while the standardized data set construction process ensures the effectiveness of model training. This method can not only accurately identify the out-of-bounds status of the tower crane, but also provide reliable technical support for the safety management of the construction site.

[0088] The standardized data processing process ensures the consistency and repeatability of the data, providing a good foundation for subsequent model optimization and system upgrades. Through a reasonable data set division strategy, the generalization ability of the model is improved, enabling the system to adapt to the application needs of different construction scenarios.

[0089] In one embodiment of the mobile tower crane crossing-boundary detection method of the present application, see Figure 6 , and can also include the following: Step S601: Building a dual-branch neural network structure, the first branch is a backbone feature extraction network, including multiple convolutional layers, pooling layers, and activation function layers, for extracting multi-scale feature maps of images; the second branch is a target detection network, including a region candidate layer and a bounding box regression layer, the feature map is input into the target detection network to generate detection box prediction values ​​for the tower crane area and the warning zone area, and the prediction values ​​are compared with the true values ​​of the annotation boxes in the data set; Step S602: Calculate the error between the predicted value and the true value of the detection box based on the cross entropy loss function, back-propagate the error to each layer of the neural network, update the network weight parameters by gradient descent method, perform multiple rounds of iterative training on the training set data, and use the validation set data to evaluate the model performance after each round of training. Stop training when the loss value on the validation set converges, and derive the network parameters to obtain the tower crane out-of-bounds detection model.

[0090] Optionally, this embodiment designs an innovative dual-branch neural network structure, which is optimized for the special needs of tower crane cross-border detection. The backbone feature extraction network adopts a deep residual structure, which effectively alleviates the gradient vanishing problem of the deep network through jump connections. In the backbone network, shallow convolution focuses on capturing low-level features such as edges and textures, while deep convolution is responsible for extracting high-level features at the semantic level to form a multi-scale feature map pyramid.

[0091] In view of the characteristics of tower crane detection, this embodiment introduces a spatial attention mechanism in the backbone network. Through adaptive weight calculation, the feature response of key parts of the tower crane (such as the boom and the fuselage) is enhanced, while suppressing background interference. This mechanism is particularly helpful for handling the detection of tower cranes under different viewing angles and lighting conditions.

[0092] The target detection network of the second branch adopts an improved anchor-free structure. The region candidate layer directly regresses the center point and scale of the target through dense prediction, avoiding the limitations of traditional anchor-based methods in dealing with large-scale scale changes. This design is particularly suitable for dealing with the posture changes of tower cranes during construction.

[0093] The bounding box regression layer adopts a multi-scale fusion strategy to combine feature maps of different levels through adaptive weights. In addition to predicting the position and size of the bounding box, the regression branch also introduces the prediction of the rotation angle to better adapt to the directional characteristics of the tower crane and the warning tape.

[0094] In terms of loss function design, this embodiment adopts a multi-task learning framework. The position regression loss uses a variant of the IoU loss, and introduces a direction-aware term to better handle the optimization of the rotated bounding box. The classification branch uses an improved focal loss, and solves the problem of imbalance between positive and negative samples through dynamic weight adjustment.

[0095] The training process adopts a progressive learning strategy. In the initial stage, a large learning rate is used to quickly explore the parameter space; as the training progresses, the learning rate is dynamically adjusted to fine-tune the model near the optimal solution. To prevent overfitting, a variety of regularization techniques are introduced, including dropout, batch normalization, and weight decay.

[0096] This embodiment pays special attention to the design of the verification process. In addition to the conventional detection accuracy index, a continuous evaluation index based on the intersection-over-union ratio is introduced to better measure the model's ability to judge the crane's out-of-bounds status. The verification process uses a sliding window mechanism to comprehensively evaluate the performance of the model on a continuous time series.

[0097] In order to improve the generalization ability of the model, a variety of data enhancement techniques were used during the training process. In addition to conventional transformations such as flipping and rotation, enhancement methods that simulate different weather conditions and lighting changes were also designed specifically. These technologies significantly improved the model's adaptability in complex construction environments.

[0098] The model convergence judgment adopts a multi-indicator comprehensive evaluation mechanism. In addition to monitoring the loss value on the validation set, it also combines the changing trends of indicators such as detection accuracy and recall rate. When multiple indicators tend to be stable at the same time, the model is considered to have reached a convergence state.

[0099] This embodiment achieves accurate detection of tower crane out-of-bounds status through a carefully designed network structure and training strategy. The optimized feature extraction mechanism and multi-task learning framework significantly improve the detection performance of the model, while the rigorous verification mechanism ensures the practicality of the model. This method provides reliable technical support for safety supervision at construction sites and can timely discover and warn potential safety hazards.

[0100] In one embodiment of the mobile tower crane crossing-boundary detection method of the present application, see Figure 7 , and can also include the following: Step S701: deploy the tower crane cross-border detection model to the edge computing unit of the monitoring system, establish a data communication link between the monitoring camera and the edge computing unit, receive the real-time image stream collected by the monitoring camera, pre-process and normalize the image stream, send the pre-processed image to the feature extraction network and target detection network of the model, and obtain the position coordinates of the tower crane area and the warning belt area in the real-time image; Step S702: Calculate the real-time intersection-and-union ratio of the tower crane area and the warning zone area based on the position coordinates, compare the real-time intersection-and-union ratio with a preset threshold, and when it is detected that the real-time intersection-and-union ratio exceeds the preset threshold for multiple consecutive frames, send an out-of-bounds alarm signal to the on-site control center through the communication interface of the monitoring system, and record the image frame, timestamp and intersection-and-union ratio data that triggers the alarm into the system log.

[0101] Optionally, this embodiment implements real-time monitoring of tower crane cross-border detection through edge computing architecture. When the edge computing unit is deployed, model quantization and pruning technology are used to optimize the trained deep learning model into a lightweight version to ensure high detection efficiency under limited computing resources.

[0102] In the process of establishing the data communication link, this embodiment uses a double buffer queue mechanism to process the image stream. The input buffer is responsible for receiving and caching the original image data from the surveillance camera, and the output buffer stores the processed detection results. This design effectively avoids data congestion and frame loss.

[0103] An adaptive normalization strategy is used in the image preprocessing phase. First, the dynamic range of the input image is adjusted, and the image contrast is improved through histogram equalization, especially in the case of insufficient light or overexposure. Then, geometric transformation normalization is performed to adjust the image to the input size required by the model while maintaining the aspect ratio of the original image.

[0104] This embodiment implements dynamic allocation of computing resources during feature extraction. The task scheduler monitors the CPU and GPU load status in real time and automatically adjusts the allocation strategy of computing tasks according to processing requirements. For computationally intensive convolution operations, GPU acceleration is preferred, while lightweight post-processing is handled by the CPU.

[0105] In the object detection phase, this embodiment designs a multi-scale detection strategy. Through the feature pyramid network, detection is performed at different resolution levels at the same time, effectively dealing with the problem of the size change of the tower crane in the field of view. Each detection layer is equipped with an independent detection head, which is responsible for the detection of objects within a specific scale range.

[0106] The real-time intersection-and-union ratio calculation uses an efficient geometric algorithm. The coordinate data of the detection frame is processed in batches through vectorized operations, and the calculation process of the intersection-and-union ratio is accelerated by using parallel computing technology. In order to deal with the rotation problem of the detection frame, a fast overlapping area calculation method based on vector cross product is implemented.

[0107] This embodiment innovatively introduces a time series filtering mechanism. By maintaining a sliding time window, the intersection-and-combination ratio of multiple consecutive frames is recorded. The comprehensive intersection-and-combination ratio within the time window is calculated by weighted average, which effectively suppresses the noise influence of single-frame detection and improves the reliability of cross-border judgment.

[0108] In the alarm triggering mechanism, this embodiment adopts a multi-level threshold strategy. In addition to the main intersection-and-combination ratio threshold, a time duration threshold and a change rate threshold are also set. The alarm is triggered only when these conditions are met at the same time, effectively reducing the false alarm rate.

[0109] The transmission of alarm signals adopts a reliable communication protocol. The reliable transmission of alarm information is ensured through the message queue mechanism, and the priority processing of alarm information is realized. For high-level cross-border alarms, the system will start the fast channel to ensure that the information is delivered to the control center in time.

[0110] The system log records adopt a distributed storage architecture. The image frames triggered by the alarm are stored after lossless compression, and the timestamp and intersection-over-union data are saved in a structured form. The log system supports multi-dimensional query and statistical analysis, which facilitates the subsequent accident backtracking and safety management.

[0111] This embodiment realizes timely detection and early warning of tower crane crossing through the organic combination of edge computing and real-time monitoring. The optimized processing flow and reliable alarm mechanism significantly improve the safety management level of the construction site and provide strong technical support for preventing tower crane operation accidents. The scalability and maintainability of the system also provide a good foundation for subsequent function upgrades and optimizations.

[0112] In order to realize real-time intelligent monitoring of the tower crane activity boundary and provide reliable technical support for construction site safety management, the present application provides an embodiment of a mobile tower crane cross-boundary detection device for realizing all or part of the content of the mobile tower crane cross-boundary detection method, see Figure 8 The mobile tower crane crossing detection device specifically includes the following contents: A virtual simulation module 10 is used to import the dimension data in the tower crane design drawing into the virtual simulation engine, wherein the dimension data includes the length, width, height of the tower crane crawler and the height of the control platform, construct a three-dimensional tower crane model in the virtual simulation engine, plan the tower crane activity boundary area according to the on-site layout requirements, set a warning belt mark in the boundary area, and configure the acquisition viewing angle and installation position parameters of multiple virtual monitoring cameras; The data monitoring module 20 is used to adjust the posture of the tower crane model, simulate the moving trajectory sequence of the tower crane under different working conditions, calculate the rolling state and the motion state of the tower crane crawler based on the physical engine, generate multiple sets of tower crane movement video sequences, calculate the intersection-and-union ratio of the tower crane area annotation box and the warning belt area annotation box, and mark the corresponding video frame as an out-of-bounds state when the intersection-and-union ratio is greater than a preset threshold, and construct a tower crane out-of-bounds behavior data set, which includes the tower crane area annotation box, the warning belt area annotation box and the intersection-and-union ratio; The out-of-bounds alarm module 30 is used to construct a deep neural network including a backbone feature extraction network and a target detection network, input the data set into the deep neural network for training, use the cross entropy loss function to optimize the parameters of the deep neural network, obtain a tower crane out-of-bounds detection model, deploy the model to the on-site monitoring system, analyze the collected real-time monitoring images based on the model, calculate the real-time intersection-and-union ratio of the detected tower crane area and the warning belt area, and output an out-of-bounds alarm signal when the real-time intersection-and-union ratio is greater than the preset threshold.

[0113] From the above description, it can be seen that the mobile tower crane out-of-bounds detection device provided in the embodiment of the present application can construct a three-dimensional model and set warning belt marks by importing the tower crane design drawing data into the virtual simulation engine. The physical engine is innovatively used to simulate the movement trajectory of the tower crane under different working conditions, and an out-of-bounds behavior data set is generated based on the intersection and union ratio calculation. A deep neural network structure is constructed, including backbone feature extraction and target detection modules, and the model parameters are optimized through the cross-entropy loss function. This method breaks through the limitations of traditional mechanical limits and manual supervision, realizes real-time intelligent monitoring of the activity boundary of the tower crane, and provides reliable technical support for construction site safety management.

[0114] From the hardware level, in order to realize real-time intelligent monitoring of the activity boundary of the tower crane and provide reliable technical support for the safety management of the construction site, the present application provides an embodiment of an electronic device for realizing all or part of the contents of the mobile tower crane cross-boundary detection method and device, and the electronic device specifically includes the following contents: Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between the mobile tower crane cross-border detection method and device and related equipment such as core business system, user terminal and related database; the logic controller can be a desktop computer, tablet computer and mobile terminal, etc., but the present embodiment is not limited thereto. In the present embodiment, the logic controller can be implemented with reference to the embodiment of the mobile tower crane cross-border detection method and device, and the embodiment of the mobile tower crane cross-border detection method and device in the embodiment, and the contents thereof are incorporated herein, and the repeated parts are not repeated.

[0115] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0116] In practical applications, part of the mobile tower crane cross-border detection method and device can be executed on the electronic device side as described above, or all operations can be completed in the client device. The specific selection can be based on the processing capacity of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor.

[0117] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0118] Fig. 9 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig. 9As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig. 9 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0119] In one embodiment, the mobile tower crane crossing-border detection method and device functions may be integrated into the central processing unit 9100. The central processing unit 9100 may be configured to perform the following control: Step S101: Importing dimension data in the tower crane design drawing into a virtual simulation engine, the dimension data including the length, width, height of the tower crane crawler and the height of the control platform, constructing a three-dimensional tower crane model in the virtual simulation engine, planning the tower crane activity boundary area according to the on-site layout requirements, setting a warning zone mark in the boundary area, and configuring the acquisition viewing angle and installation position parameters of multiple virtual monitoring cameras; Step S102: adjusting the posture of the tower crane model, simulating the moving trajectory sequence of the tower crane under different working conditions, calculating the rolling state and the motion state of the tower crane crawler based on the physical engine, generating multiple sets of tower crane moving video sequences, calculating the intersection-and-union ratio of the tower crane area annotation box and the warning belt area annotation box, marking the corresponding video frame as an out-of-bounds state when the intersection-and-union ratio is greater than a preset threshold, and constructing a tower crane out-of-bounds behavior data set, the data set including the tower crane area annotation box, the warning belt area annotation box and the intersection-and-union ratio; Step S103: construct a deep neural network including a backbone feature extraction network and a target detection network, input the data set into the deep neural network for training, use the cross entropy loss function to optimize the parameters of the deep neural network, obtain a tower crane out-of-bounds detection model, deploy the model to the on-site monitoring system, analyze the collected real-time monitoring images based on the model, calculate the real-time intersection-and-union ratio of the detected tower crane area and the warning belt area, and output an out-of-bounds alarm signal when the real-time intersection-and-union ratio is greater than the preset threshold.

[0120] From the above description, it can be seen that the electronic device provided in the embodiment of the present application imports the tower crane design drawing data into the virtual simulation engine, constructs a three-dimensional model and sets warning belt marks. The physical engine is innovatively used to simulate the movement trajectory of the tower crane under different working conditions, and an out-of-bounds behavior data set is generated based on the intersection and union ratio calculation. A deep neural network structure is constructed, including a backbone feature extraction and target detection module, and the model parameters are optimized through a cross-entropy loss function. This method breaks through the limitations of traditional mechanical limits and manual supervision, realizes real-time intelligent monitoring of the activity boundary of the tower crane, and provides reliable technical support for construction site safety management.

[0121] In another embodiment, the mobile tower crane cross-border detection method and device can be configured separately from the central processing unit 9100. For example, the mobile tower crane cross-border detection method and device can be configured as a chip connected to the central processing unit 9100, and the functions of the mobile tower crane cross-border detection method and device are realized through the control of the central processing unit.

[0122] like Fig. 9 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig. 9 In addition, the electronic device 9600 may also include Fig. 9 For components not shown, reference may be made to the prior art.

[0123] like Fig. 9 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0124] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0125] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0126] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0127] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0128] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0129] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module. The communication module 9110 (transmitter / receiver) is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0130] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps in the mobile tower crane cross-border detection method and device in the above embodiments, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, all the steps in the mobile tower crane cross-border detection method and device in the above embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented: Step S101: Importing dimension data in the tower crane design drawing into a virtual simulation engine, the dimension data including the length, width, height of the tower crane crawler and the height of the control platform, constructing a three-dimensional tower crane model in the virtual simulation engine, planning the tower crane activity boundary area according to the on-site layout requirements, setting a warning zone mark in the boundary area, and configuring the acquisition viewing angle and installation position parameters of multiple virtual monitoring cameras; Step S102: adjusting the posture of the tower crane model, simulating the moving trajectory sequence of the tower crane under different working conditions, calculating the rolling state and the motion state of the tower crane crawler based on the physical engine, generating multiple sets of tower crane moving video sequences, calculating the intersection-and-union ratio of the tower crane area annotation box and the warning belt area annotation box, marking the corresponding video frame as an out-of-bounds state when the intersection-and-union ratio is greater than a preset threshold, and constructing a tower crane out-of-bounds behavior data set, the data set including the tower crane area annotation box, the warning belt area annotation box and the intersection-and-union ratio; Step S103: construct a deep neural network including a backbone feature extraction network and a target detection network, input the data set into the deep neural network for training, use the cross entropy loss function to optimize the parameters of the deep neural network, obtain a tower crane out-of-bounds detection model, deploy the model to the on-site monitoring system, analyze the collected real-time monitoring images based on the model, calculate the real-time intersection-and-union ratio of the detected tower crane area and the warning belt area, and output an out-of-bounds alarm signal when the real-time intersection-and-union ratio is greater than the preset threshold.

[0131] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application imports the tower crane design drawing data into the virtual simulation engine, constructs a three-dimensional model and sets warning belt marks. The physical engine is innovatively used to simulate the movement trajectory of the tower crane under different working conditions, and an out-of-bounds behavior data set is generated based on the intersection and union ratio calculation. A deep neural network structure is constructed, including a backbone feature extraction and target detection module, and the model parameters are optimized through a cross-entropy loss function. This method breaks through the limitations of traditional mechanical limits and manual supervision, realizes real-time intelligent monitoring of the activity boundary of the tower crane, and provides reliable technical support for construction site safety management.

[0132] The embodiments of the present application also provide a computer program product capable of implementing all the steps in the mobile tower crane cross-border detection method and device in the above embodiments, where the execution subject is a server or a client. When the computer program / instruction is executed by a processor, the steps of the mobile tower crane cross-border detection method and device are implemented. For example, the computer program / instruction implements the following steps: Step S101: Importing dimension data in the tower crane design drawing into a virtual simulation engine, the dimension data including the length, width, height of the tower crane crawler and the height of the control platform, constructing a three-dimensional tower crane model in the virtual simulation engine, planning the tower crane activity boundary area according to the on-site layout requirements, setting a warning zone mark in the boundary area, and configuring the acquisition viewing angle and installation position parameters of multiple virtual monitoring cameras; Step S102: adjusting the posture of the tower crane model, simulating the moving trajectory sequence of the tower crane under different working conditions, calculating the rolling state and the motion state of the tower crane crawler based on the physical engine, generating multiple sets of tower crane moving video sequences, calculating the intersection-and-union ratio of the tower crane area annotation box and the warning belt area annotation box, marking the corresponding video frame as an out-of-bounds state when the intersection-and-union ratio is greater than a preset threshold, and constructing a tower crane out-of-bounds behavior data set, the data set including the tower crane area annotation box, the warning belt area annotation box and the intersection-and-union ratio; Step S103: construct a deep neural network including a backbone feature extraction network and a target detection network, input the data set into the deep neural network for training, use the cross entropy loss function to optimize the parameters of the deep neural network, obtain a tower crane out-of-bounds detection model, deploy the model to the on-site monitoring system, analyze the collected real-time monitoring images based on the model, calculate the real-time intersection-and-union ratio of the detected tower crane area and the warning belt area, and output an out-of-bounds alarm signal when the real-time intersection-and-union ratio is greater than the preset threshold.

[0133] From the above description, it can be seen that the computer program product provided in the embodiment of the present application imports the tower crane design drawing data into the virtual simulation engine, constructs a three-dimensional model and sets warning belt marks. The physical engine is innovatively used to simulate the movement trajectory of the tower crane under different working conditions, and an out-of-bounds behavior data set is generated based on the intersection and union ratio calculation. A deep neural network structure is constructed, including a backbone feature extraction and target detection module, and the model parameters are optimized through a cross-entropy loss function. This method breaks through the limitations of traditional mechanical limits and manual supervision, realizes real-time intelligent monitoring of the tower crane's activity boundaries, and provides reliable technical support for construction site safety management.

[0134] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0138] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A mobile tower crane crossing-boundary detection method, characterized in that: The method comprises: Importing the dimension data in the tower crane design drawing into the virtual simulation engine, the dimension data includes the length, width, height of the tower crane crawler and the height of the control platform, constructing a three-dimensional tower crane model in the virtual simulation engine, planning the tower crane activity boundary area according to the on-site layout requirements, setting a warning belt mark in the boundary area, and configuring the acquisition viewing angle and installation position parameters of multiple virtual monitoring cameras; The tower crane model posture is adjusted to simulate the moving trajectory sequence of the tower crane under different working conditions, the rolling state and the motion state of the tower crane crawler are calculated based on the physical engine, and multiple sets of tower crane moving video sequences are generated. The intersection-and-union ratio of the tower crane area annotation box and the warning belt area annotation box is calculated, and when the intersection-and-union ratio is greater than a preset threshold, the corresponding video frame is marked as an out-of-bounds state, and a tower crane out-of-bounds behavior data set is constructed, and the data set includes the tower crane area annotation box, the warning belt area annotation box and the intersection-and-union ratio; A deep neural network including a backbone feature extraction network and a target detection network is constructed, the data set is input into the deep neural network for training, the parameters of the deep neural network are optimized using a cross entropy loss function, a tower crane out-of-bounds detection model is obtained, the model is deployed to an on-site monitoring system, the collected real-time monitoring images are analyzed based on the model, the real-time intersection-and-union ratio of the detected tower crane area and the warning belt area is calculated, and an out-of-bounds alarm signal is output when the real-time intersection-and-union ratio is greater than the preset threshold.

2. The mobile tower crane crossing-border detection method according to claim 1 is characterized in that: The dimensional data in the tower crane design drawing is imported into the virtual simulation engine, wherein the dimensional data includes the length, width, height of the tower crane crawler and the height of the control platform, and a three-dimensional tower crane model is constructed in the virtual simulation engine, including: Read the two-dimensional structural data of the tower crane design drawing, extract the length parameters, width parameters, height parameters of the tower crane crawler and the height parameters of the control platform, as well as the connection relationship data of the tower crane main structure, and convert the parameters and connection relationship data into a data format supported by the virtual simulation engine; Based on the track parameters and the main body parameters, a geometric skeleton of the tower crane model is constructed in the three-dimensional coordinate system of the virtual simulation engine, material mapping and physical property parameters are added to each component of the tower crane, physical connection constraints and movement freedom restrictions are established between the tower crane components, and the construction of the three-dimensional tower crane model is completed in the virtual simulation engine.

3. The mobile tower crane crossing-border detection method according to claim 1 is characterized in that: The tower crane activity boundary area is planned according to the on-site layout requirements, a warning zone mark is set in the boundary area, and the acquisition viewing angles and installation position parameters of multiple virtual monitoring cameras are configured, including: According to the terrain data and operation scope requirements of the construction site, the coordinates of the contour points of the tower crane activity boundary are calibrated in the three-dimensional scene of the virtual simulation engine, a closed boundary curve is generated through the contour points, a three-dimensional warning belt model with height parameters is created along the boundary curve, and reflective material and color attributes are configured for the warning belt model; Arrange multiple surveillance camera models in a virtual three-dimensional scene, set the focal length parameters, field of view angle parameters and resolution parameters of each surveillance camera, adjust the camera's installation height and orientation angle to ensure that the camera's field of view covers the tower crane activity area and the warning zone area, and calculate the spatial projection matrix of each surveillance camera.

4. The mobile tower crane crossing-border detection method according to claim 1, characterized in that: The adjusting the tower crane model posture, simulating the moving trajectory sequence of the tower crane under different working conditions, calculating the rolling state and the motion state of the tower crane crawler based on the physical engine, and generating multiple sets of tower crane moving video sequences include: Generate a displacement trajectory point sequence of the tower crane model based on the construction condition data, apply driving force and motion constraints to the tower crane model, calculate the contact force and friction force between the tower crane crawler and the ground through the physical engine, update the position coordinates and attitude angle of the tower crane model according to the mechanical equation, and calculate the rolling speed of the tower crane crawler and the deformation state of the crawler segment; Image frames of the tower crane model during its movement are collected from the perspective of the surveillance camera, the image frames are organized into a video sequence in time sequence, the spatial position relationship between the tower crane model and the warning tape model in each image frame is recorded, and the degree of overlap between the projection area of ​​the tower crane model and the projection area of ​​the warning tape model on the image plane is calculated based on the spatial position relationship.

5. The mobile tower crane crossing-border detection method according to claim 1, characterized in that: The intersection-and-union ratio of the tower crane area marking frame and the warning zone area marking frame is calculated, and when the intersection-and-union ratio is greater than a preset threshold, the corresponding video frame is marked as an out-of-bounds state, and a tower crane out-of-bounds behavior data set is constructed, and the data set includes the tower crane area marking frame, the warning zone area marking frame and the intersection-and-union ratio, including: Performing semantic segmentation on each frame image in the video sequence, extracting the contour boundary points of the tower crane model and the contour boundary points of the warning belt model, generating a minimum circumscribed rectangle as a labeling box based on the boundary points, calculating the overlapping area and the union area of ​​the tower crane area labeling box and the warning belt area labeling box, and dividing the overlapping area by the union area to obtain an intersection-union ratio; The video sequence is marked according to a preset intersection-and-union ratio threshold, the coordinates of the tower crane area annotation box, the coordinates of the warning belt area annotation box and the corresponding intersection-and-union ratio of each frame image are stored in the data set, a standardized data label file is generated for each frame image, and the image file and the label file are divided and stored according to the ratio of the training set and the verification set.

6. The mobile tower crane crossing-border detection method according to claim 1, characterized in that: The method comprises constructing a deep neural network including a backbone feature extraction network and a target detection network, inputting the data set into the deep neural network for training, optimizing the parameters of the deep neural network using a cross entropy loss function, and obtaining a tower crane cross-boundary detection model, including: A dual-branch neural network structure is built. The first branch is a backbone feature extraction network, which includes multiple convolutional layers, pooling layers, and activation function layers, and is used to extract multi-scale feature maps of the image. The second branch is a target detection network, which includes a region candidate layer and a bounding box regression layer. The feature map is input into the target detection network to generate detection box prediction values ​​for the crane area and the warning zone area, and the prediction values ​​are compared with the true values ​​of the labeled boxes in the data set. The error between the predicted value and the true value of the detection box is calculated based on the cross entropy loss function, the error is back-propagated to each layer of the neural network, the network weight parameters are updated by the gradient descent method, the training set data is iterated for multiple rounds of training, and the model performance is evaluated using the validation set data after each round of training. When the loss value on the validation set converges, the training is stopped, and the network parameters are derived to obtain the tower crane out-of-bounds detection model.

7. The mobile tower crane crossing-border detection method according to claim 1, characterized in that: The model is deployed to the on-site monitoring system, and the collected real-time monitoring images are analyzed based on the model, and the real-time intersection-and-combination ratio of the detected tower crane area and the warning belt area is calculated, and when the real-time intersection-and-combination ratio is greater than the preset threshold, an out-of-bounds alarm signal is output, including: The tower crane crossing detection model is deployed to the edge computing unit of the monitoring system, a data communication link between the monitoring camera and the edge computing unit is established, the real-time image stream collected by the monitoring camera is received, the image stream is preprocessed and normalized, and the preprocessed image is sent to the feature extraction network and target detection network of the model to obtain the position coordinates of the tower crane area and the warning belt area in the real-time image; The real-time intersection-and-union ratio of the tower crane area and the warning zone area is calculated based on the position coordinates, and the real-time intersection-and-union ratio is compared with a preset threshold. When it is detected that the real-time intersection-and-union ratio exceeds the preset threshold for multiple consecutive frames, an out-of-bounds alarm signal is sent to the on-site control center through the communication interface of the monitoring system, and the image frame, timestamp and intersection-and-union ratio data that triggers the alarm are recorded in the system log.

8. A mobile tower crane crossing detection device, characterized in that: The device comprises: A virtual simulation module is used to import the dimension data in the tower crane design drawing into the virtual simulation engine, wherein the dimension data includes the length, width, height of the tower crane crawler and the height of the control platform, construct a three-dimensional tower crane model in the virtual simulation engine, plan the tower crane activity boundary area according to the on-site layout requirements, set a warning belt mark in the boundary area, and configure the acquisition viewing angle and installation position parameters of multiple virtual monitoring cameras; A data monitoring module is used to adjust the posture of the tower crane model, simulate the moving trajectory sequence of the tower crane under different working conditions, calculate the rolling state and movement state of the tower crane crawler based on the physical engine, generate multiple sets of tower crane movement video sequences, calculate the intersection-and-union ratio of the tower crane area annotation box and the warning belt area annotation box, and mark the corresponding video frame as an out-of-bounds state when the intersection-and-union ratio is greater than a preset threshold, and construct a tower crane out-of-bounds behavior data set, which includes the tower crane area annotation box, the warning belt area annotation box and the intersection-and-union ratio; The cross-border alarm module is used to construct a deep neural network including a backbone feature extraction network and a target detection network, input the data set into the deep neural network for training, use the cross entropy loss function to optimize the parameters of the deep neural network, obtain a tower crane cross-border detection model, deploy the model to the on-site monitoring system, analyze the collected real-time monitoring images based on the model, calculate the real-time intersection-and-union ratio of the detected tower crane area and the warning belt area, and output a cross-border alarm signal when the real-time intersection-and-union ratio is greater than the preset threshold.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the mobile tower crane cross-border detection method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the mobile tower crane cross-border detection method described in any one of claims 1 to 7 are implemented.

Images