A tower crane cab blind area and rear platform perception compensation method and system

Abstract

The application relates to the technical field of engineering machinery safety monitoring, and provides a tower crane cab blind area and rear platform perception compensation method and system. The method is characterized in that a rotating radar module integrated and installed below a tower crane trolley is used to scan the elevation angles of the areas on both sides of the cab and the area behind the tower body, and the environmental data of the areas on both sides of the cab and the area behind the tower body are acquired; based on the acquired environmental data, a construction environment model of a tower crane operation scene is constructed; an operation behavior recognition algorithm is used to process the construction environment model, and corresponding potential shielding objects and danger sources are identified; according to the identification result, a visual auxiliary image containing blind area and rear platform area information is generated; operation data of the tower crane is collected, the movement track of a hoisted object and the movement track of surrounding people and vehicles are predicted, and if it is judged that a dynamic target enters the blind area, early warning information is generated; and the visual auxiliary image and the early warning information are fed back to a tower crane driving interface.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of engineering machinery safety monitoring, in particular to a tower crane cab blind area and rear platform perception compensation method and system. BACKGROUND

[0002] As a core vertical transportation equipment in construction, the tower crane cab blind area (especially the two sides and the rear of the tower body) is prone to cause the driver to be unable to perceive the surrounding obstacles and dynamic targets (such as construction personnel and vehicles) in real time due to the tower structure blocking, the change of the hoisting object position and the complex construction site environment, which may cause safety hazards such as collision and overturning. The traditional safety monitoring method mainly relies on the driver's visual observation, rearview mirror or fixed camera, which has the following defects:

[0003] 1. Limited monitoring range: fixed angle sensors (such as cameras) can only cover a local area and cannot dynamically adapt to the tower crane working amplitude and height changes, resulting in dead angles in monitoring the blind area on both sides of the cab and the rear of the tower body;

[0004] 2. Insufficient environmental modeling capability: the existing technology lacks real-time modeling and dynamic updating of the construction environment (such as three-dimensional terrain, temporary facilities and vegetation), making it difficult to accurately identify potential obstructions and hazards;

[0005] 3. Lack of risk prediction: only real-time images can be displayed passively, without combining tower crane operation data to predict the motion trajectory of the hoisting object and surrounding targets, so as to provide early warning of the risk of dynamic targets entering the blind area;

[0006] 4. Inefficient human-computer interaction: the visual information lacks risk level division and targeted display, and the driver is easily distracted by redundant information, resulting in delayed warning response.

[0007] Therefore, there is an urgent need for a method to solve at least one of the above problems. SUMMARY

[0008] The present application provides a tower crane cab blind area and rear platform perception compensation method and system, aiming to solve the problem in the prior art that the monitoring scheme for the tower crane blind area mainly uses a single sensor (such as a camera or a fixed radar) to monitor a local area, without forming a complete closed loop of "environmental data acquisition-three-dimensional modeling-intelligent identification-dynamic prediction-classified warning". In particular, there is a lack of a comprehensive scheme for performing layered scanning on multiple areas by a rotating radar module, constructing a dynamic construction environment model by combining operation behavior identification algorithms, and trajectory prediction and risk classification warning based on operation data.

[0009] In a first aspect, the embodiments of the present application provide a tower crane cab blind area and rear platform perception compensation method; the method comprises:

[0010] The rotating radar module integrated under the trolley of the tower crane scans the elevation angle of the areas on both sides of the cab and the area behind the tower body, and obtains environmental data of the areas on both sides of the cab and the area behind the tower body.

[0011] Based on the obtained environmental data, a construction environment model of the tower crane operation scene is constructed, the construction environment model is processed by using an operation behavior recognition algorithm to identify corresponding potential obstacles and hazards, and a visual auxiliary image containing blind area and rear platform area information is generated in real time according to the identification result, including: based on the construction environment model and the identification result, a two-dimensional bird's eye view and a three-dimensional local magnification view are generated by using computer graphics technology; in the two-dimensional bird's eye view, the blind areas on both sides of the cab and the area behind the tower body are marked by semi-transparent blocks, different colors are used to distinguish the risk levels, and the real-time positions and motion direction arrows of dynamic targets are superimposed and displayed; in the three-dimensional local magnification view, the obstacle profiles within 10 meters around the rear platform and the details of dynamic targets are highlighted; the refresh frequency of the visual auxiliary image is synchronized with the radar scanning frequency, and the display scale is adjusted according to the cab view angle.

[0012] Operation data of the tower crane is collected, the motion trajectories of the hoisted objects and the motion trajectories of the surrounding people and vehicles are predicted based on the operation data, and it is determined whether a dynamic target enters the blind area;

[0013] If it is determined that a dynamic target enters the blind area, a warning information is generated; the visual auxiliary image and the warning information are fed back to the tower crane driving interface.

[0014] In some embodiments, the rotating radar module integrated under the trolley of the tower crane scans the elevation angle of the areas on both sides of the cab and the area behind the tower body, and obtains environmental data of the areas on both sides of the cab and the area behind the tower body, including: controlling the rotating radar module to rotate horizontally at a preset angular velocity, and performing layered scanning at preset elevation angle intervals during the rotation, and for each preset fan-shaped area on both sides of the cab and the rectangular area behind the tower body, real-time collecting distance, azimuth and reflection intensity data of each scanning point to form a point cloud data sequence; wherein the preset elevation angle interval and the scanning area range are dynamically adjusted according to the current working amplitude and height parameters of the tower crane.

[0015] In some embodiments, the construction environment model of the tower crane working scene is constructed based on the acquired environment data, including: time stamp alignment and spatial coordinate conversion of the continuously collected point cloud data sequence, converting the radar coordinate system data into global coordinate system data with the tower crane rotation center as the origin; denoising the global coordinate system data by a voxel filtering algorithm to remove noise points and invalid data at a long distance; generating a static environment model containing three-dimensional terrain, fixed buildings, temporary facilities and vegetation based on multi-frame point cloud data stitching technology, and marking the real-time position contour of the tower crane structure components to form a dynamically updated construction environment three-dimensional grid model.

[0016] In some embodiments, the construction environment model is processed using a working behavior recognition algorithm to identify corresponding potential obstructions and hazards, including: inputting the construction environment model into a preset deep learning neural network model, identifying the type, position and contour size of static obstructions and dynamic targets through a target detection module; wherein the static obstructions at least include buildings, scaffolds and piled materials, and the dynamic targets at least include construction vehicles, personnel and mobile machinery; dividing the risk level of the blind area and the rear platform area through a semantic segmentation module, marking the fixed obstacles causing line-of-sight obstruction and the dynamic targets entering the dangerous area, and outputting the identification results containing target categories, risk levels and spatial positions.

[0017] In some embodiments, the operation data of the tower crane is collected, including: acquiring the gear signal of the operation handle, the rotation speed and current data of each mechanism motor, the encoder feedback of the hook height, the trolley position and the jib rotation angle through the tower crane controller in real time, collecting the real-time data of the tower crane inclination sensor, wind speed sensor and weight sensor, and forming a multi-dimensional operation data sequence containing operation instructions, mechanism states and environmental parameters.

[0018] In some embodiments, the motion trajectory of the hoisted object and the motion trajectory of the surrounding people and vehicles are predicted based on the operation data to determine whether a dynamic target enters the blind area, including: for the hoisted object, the spatial trajectory within 1-3 seconds in the future is predicted using a kinematic model according to the current hook position, trolley speed and jib rotation angular velocity; for the surrounding dynamic targets, their future motion direction and speed are predicted through a Kalman filtering algorithm based on historical position data; a blind area spatial coordinate system is established to calculate the intersection of the predicted trajectory and the blind area space in real time, and when the predicted position of the dynamic target enters the 0.5-meter range within the blind area boundary, it is determined to enter the blind area.

[0019] In some embodiments, if it is determined that a dynamic target enters the blind area, the pre-warning information is generated, including: generating the pre-warning information according to the type, speed and risk level of entering the blind area of the dynamic target; the pre-warning information includes a first pre-warning signal and a second pre-warning signal; the first pre-warning signal and the second pre-warning signal both include the target type, the entering position and the remaining time of the predicted arrival of the dangerous distance; wherein, when a person is detected to enter a low-risk blind area at a speed lower than 1.5 m / s, the first pre-warning signal is generated; when a vehicle is detected to enter a medium-risk blind area at a speed higher than 5 km / h or a person is detected to enter a high-risk blind area, the second pre-warning signal is generated.

[0020] In some embodiments, the visualization auxiliary image and the pre-warning information are fed back to the tower crane driving interface, including: dividing an independent display area on a display screen in the tower crane cab, displaying the visualization auxiliary image on the left side area in full screen, and displaying a list of currently identified dangerous sources in the right side area in real time; when the pre-warning information is generated, a floating warning window is popped up at the top of the display screen, covering 20% of the currently displayed content, and the pre-warning information automatically disappears after the dynamic target leaves the blind area.

[0021] In a second aspect, the application provides a blind area and rear platform perception compensation system for a tower crane cab, including:

[0022] A data acquisition unit is configured to scan the elevation angles of the areas on both sides of the cab and the area behind the tower body by a rotating radar module integratedly installed below the tower crane trolley, and acquire the environmental data of the areas on both sides of the cab and the area behind the tower body.

[0023] A model construction unit is configured to construct a construction environment model of a tower crane working scene based on the acquired environmental data, process the construction environment model by a working behavior recognition algorithm, identify corresponding potential occlusions and dangerous sources, and generate a visualization auxiliary image containing blind area and rear platform area information in real time according to the identification result.

[0024] A data acquisition unit is configured to acquire operation data of the tower crane, predict the motion trajectories of the hoisted objects and the surrounding people and vehicles based on the operation data, and determine whether a dynamic target enters the blind area.

[0025] A pre-warning generation unit is configured to generate pre-warning information if it is determined that a dynamic target enters the blind area, and feed back the visualization auxiliary image and the pre-warning information to the tower crane driving interface.

[0026] The embodiment of the application solves the problem of visual angle blind area of the traditional monitoring device by rotating the radar module for dynamic scanning to obtain environmental data of the areas on both sides of the cab and behind the tower body in real time, and the monitoring range covers the whole working condition of the tower crane operation; based on the three-dimensional construction environment model and the deep learning algorithm, static obstacles (such as buildings and scaffolds) and dynamic targets (such as personnel and vehicles) are accurately identified and risk level classification is performed to improve the intelligent level of the hazard identification; in combination with the tower crane operation data and the Kalman filtering algorithm, the motion trajectory of the hoisted object and the surrounding target is predicted in advance to realize early warning of the “dynamic target entering the blind area” and change the safety prevention and control from passive response to active prevention; the visual auxiliary image and the graded warning information are fed back to the driving interface in real time, the risk level is distinguished by color, the dynamic target motion arrow is marked, and the floating warning window is provided to reduce the information processing load of the driver and improve the decision-making efficiency; the scanning parameters and the display scale are dynamically adjusted according to the working amplitude and height of the tower crane and the visual angle of the cab to ensure the stability of the monitoring precision and display effect under complex working conditions.

[0027] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory and are not restrictive of the application. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0029] Figure 1 is a step schematic flow chart of a tower crane cab blind area and rear platform perception compensation method provided by an embodiment of the application;

[0030] Figure 2 is a principle schematic diagram of a tower crane cab blind area and rear platform perception compensation method provided by an embodiment of the application;

[0031] Figure 3 is a structure schematic block diagram of a tower crane cab blind area and rear platform perception compensation system provided by an embodiment of the application;

[0032] Figure 4 is a structure schematic block diagram of a controller provided by an embodiment of the application.

[0033] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory and are not restrictive of the application. DETAILED DESCRIPTION

[0034] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts are within the scope of the present application.

[0035] The flowcharts shown in the drawings are only exemplary and do not necessarily include all contents and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can be further decomposed, combined or partially merged, so that the actual execution order can be changed according to actual conditions.

[0036] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the terms "first", "second", etc. are used to distinguish the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order, and the terms "first", "second", etc. also do not necessarily mean that they are different.

[0037] It should be understood that the terms used in the present application are only for the purpose of describing specific embodiments and do not intend to limit the present application. As used in the present application specification and the appended claims, unless otherwise clear from the context, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0038] It should also be understood that the term "and / or" used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0039] As a core vertical transportation equipment in construction, the cab visual blind area of tower crane (especially the area on both sides and behind the tower body) is prone to cause the driver to be unable to perceive the surrounding obstacles and dynamic targets (such as construction personnel and vehicles) in real time due to the obstruction of the tower crane structure, the change of the position of the hoisted object and the complex environment of the construction site, which may cause safety hazards such as collision and overturning. The traditional safety monitoring means mainly relies on the driver's visual observation, rearview mirror or fixed camera, which has the following defects:

[0040] 1. Limited monitoring range: fixed angle sensors (such as cameras) can only cover a local area and cannot dynamically adapt to the working amplitude and height changes of the tower crane, resulting in dead angles in monitoring the blind area on both sides of the cab and behind the tower body;

[0041] 2. Insufficient environmental modeling capability: the existing technology lacks real-time modeling and dynamic updating of the construction environment (such as three-dimensional terrain, temporary facilities, and vegetation), making it difficult to accurately identify potential obstructions and hazards;

[0042] 3. Lack of risk prediction: only real-time images can be displayed passively, without combining tower crane operation data to predict the motion trajectory of the hoisted object and surrounding targets, and unable to provide early warning of the risk of dynamic targets entering the blind area;

[0043] 4. Inefficient human-computer interaction: the visual information lacks risk level classification and targeted display, and the driver is easily distracted by redundant information, leading to delayed warning response.

[0044] Therefore, there is an urgent need for a method to solve at least one of the above problems.

[0045] Please refer to Figures 1 to 2 , the embodiments of the present application provide a tower crane cab blind area and rear platform perception compensation method, applied to a controller. At the same time, it should be pointed out that the method provided by the present application involves each information extracted under the authorization of the relevant user and in accordance with the relevant provisions, which does not infringe on the user's privacy.

[0046] As Figure 1 shown, the provided tower crane cab blind area and rear platform perception compensation method includes steps S101 to S104. The details are as follows:

[0047] Step S101. Through the rotating radar module integrated below the tower crane trolley, the areas on both sides of the cab and the area behind the tower body are scanned in pitch angle, and the environmental data of the areas on both sides of the cab and the area behind the tower body are obtained.

[0048] Specifically, through the rotating radar module integrated below the tower crane trolley, the areas on both sides of the cab (left and right sides of the hoisting arm) and the area behind the tower body are dynamically scanned, and environmental data including three-dimensional coordinates, distance, and angle are obtained, solving the problem of limited monitoring range of traditional fixed cameras.

[0049] The radar selection adopts a 120°-270° field of view angle mechanical rotating laser radar (such as a 16-line / 32-line LiDAR), supports ±45° pitch angle adjustment, and is driven by a servo motor to realize horizontal 360° rotation and dynamic adjustment of the pitch angle, thereby covering the full height and luffing range during tower crane operation. The radar module is fixed to the center of the tower crane trolley bottom and moves along the trolley in the transverse direction (luffing direction), so as to ensure that the scanning range is dynamically adjusted according to the trolley position and covers the blind area on both sides of the cab (20-50 meters in the transverse direction) and the rear of the tower body (30-80 meters behind the hoisting arm). The radar attitude data (pitch angle and roll angle) are obtained in real time by integrating an IMU (inertial measurement unit), and the spatial coordinate calibration of the scanning data is realized in combination with the tower height sensor and the rotation angle encoder.

[0050] The layered scanning dynamically adjusts the radar pitch angle according to the tower crane operation height (such as every 5 meters as a layer), and covers the multi-layer areas of low altitude (ground personnel), medium altitude (temporary facilities), and high altitude (surrounding buildings).

[0051] The key area enhancement automatically increases the scanning frequency of the rear counterweight area and the blind area on both sides when the tower crane luffs to a specific amplitude (such as close to the maximum working radius), thereby improving the data density. The real-time point cloud data (X, Y, Z coordinates + reflection intensity) are generated, which contains the position information of obstacles (such as fences and trees) and dynamic targets (personnel and vehicles), and the data refresh rate is ≥10 Hz, which meets the real-time requirement.

[0052] Step S102. Based on the obtained environment data, a construction environment model of the tower crane operation scene is constructed; an operation behavior recognition algorithm is used to process the construction environment model to identify corresponding potential obstacles and hazards; and according to the identification result, a visual auxiliary image containing blind area and rear platform area information is generated in real time, including: based on the construction environment model and the identification result, a two-dimensional bird's eye view and a three-dimensional local enlarged view are generated by using computer graphics technology; in the two-dimensional bird's eye view, the blind area on both sides of the cab and the area behind the tower body are marked with semi-transparent blocks, different colors are used to distinguish the risk levels, and the real-time position and motion direction arrow of the dynamic target are superimposed and displayed; in the three-dimensional local enlarged view, the obstacle profile and dynamic target details within a 10-meter range around the rear platform are highlighted; the refresh frequency of the visual auxiliary image is synchronized with the radar scanning frequency, and the display scale is adjusted according to the cab viewing angle.

[0053] Specifically, a three-dimensional construction environment model is constructed based on radar point cloud data, potential risks are identified by an operation behavior recognition algorithm, a visual interface containing blind area annotations is generated, and the problems of insufficient environment modeling and information redundancy are solved.

[0054] The three-dimensional environment modeling includes: point cloud processing: denoising the original point cloud by using voxel filtering (Voxel Grid) and outlier removal (RANSAC), constructing a dynamically updated three-dimensional grid model through an octree (Octree), and mapping static obstacles such as terrain, temporary facilities (temporary housing, material piles), and vegetation in real time. SLAM positioning: combining tower crane rotary encoder data, updating the tower crane's own position in real time through the LOAM (Laser Radar SLAM) algorithm, constructing an environment map under the global coordinate system, supporting multi-frame point cloud splicing, and solving the problem of changing view angle caused by tower crane rotation.

[0055] Risk target identification includes: static obstacle identification: extracting fixed obstacles (such as fences and tower crane foundations) based on point cloud clustering algorithm (DBSCAN), and labeling them as "permanent obstruction area"; identifying ground temporary facilities (scaffolding, power distribution box) through height threshold (such as ≤2 meters). Dynamic target detection: using a deep learning target detection model (such as PointPillars, YOLO-LIDAR) to identify personnel (carrying safety helmet features) and vehicles (matching contour size), and tracking their motion direction combined with optical flow method, and marking them as "dynamic hazard source".

[0056] Visual aid image generation includes: blind area labeling: presetting the driving cab on both sides (lateral ±15 meters, height 2-5 meters) and the tower body rear (radius 30 meters, height 0-10 meters) as key blind areas in the three-dimensional model, and labeling them with semi-transparent red areas.

[0057] Risk level division includes: first-level risk (distance <5 meters): obstacles / personnel highlighted in red, with flashing edges; second-level risk (5-15 meters): yellow labeling, showing the outline; safe area: gray transparent display of the background environment. Multi-view fusion: projecting the three-dimensional model into 2D plan view (top view, side view), superimposed on the cab screen, focusing on displaying the relative relationship between the tower crane's current position and the surrounding targets, supporting gesture zoom to view details.

[0058] Among them, through the generation of multi-view visual interface, the two-dimensional bird's eye view, three-dimensional detail map and risk level labeling are fused, and the radar scanning frequency is synchronized in real time.

[0059] The two-dimensional bird's eye view takes the tower crane as the center, renders a top-down plan view within a radius of 80 meters, labels the blind areas on both sides with semi-transparent orange sectors, and labels the rear blind area with a red rectangle; dynamic targets are represented by icons (personnel → human icon, vehicle → truck icon), and the icon color corresponds to the risk level (red-L2, yellow-L1), with a motion direction arrow (arrow length represents speed).

[0060] Three-dimensional local zoom-in view: default display of 10-meter range around the rear platform, support for gesture zoom (touch screen operation), highlighting of obstacle edges (such as scaffold steel pipes) and dynamic target contours (point cloud rendering as mesh models).

[0061] Through synchronization with radar refresh frequency (10 Hz), no delay is ensured; adjustment of cab viewing angle: when the driver operates the slewing handle, the northward bird's eye view is automatically adjusted to the current crane boom direction, improving spatial awareness efficiency.

[0062] In some embodiments, computer graphics technology is used to generate two-dimensional bird's eye view and three-dimensional local zoom-in view, including: first, based on the slewing center of the tower crane, a three-dimensional world coordinate system is established, the point cloud data collected by the rotating radar module is converted into relative coordinates with the tower crane as the origin, and invalid environmental data is filtered through a pre-set distance threshold; for the generation of two-dimensional bird's eye view, the point cloud data of the blind area on both sides of the cab and the rear area of the tower body in the three-dimensional world coordinate system is orthographically projected onto the horizontal XY plane, the fan angle range of the blind area on both sides and the rectangular boundary range of the rear area are determined based on the tower crane structure parameters (including the cab width, the tower body width and the radar installation pitch angle range), and a semi-transparent polygon filling algorithm is used to mark the above-mentioned areas with color blocks, wherein the color block color mapping is pre-set with a risk level rule - according to the distance and speed of potential obstacles or hazards from the tower crane, the risk level is divided into three levels and corresponds to red, yellow and blue filling colors respectively; for dynamic targets, their real-time coordinates are obtained through a target tracking algorithm, geometric symbols of dynamic targets are marked on the projection plane, and arrows indicating the direction of motion are drawn according to the calculation results of the motion trajectory; for the generation of three-dimensional local zoom-in view, a spherical area with a radius of 10 meters is determined with the center point of the rear platform of the tower crane as the reference, point cloud data of all obstacles and dynamic targets in the area is extracted, point cloud clustering algorithm is used to separate each independent target, and mesh modeling algorithm is used to generate the contour line frame of the obstacles and the simplified three-dimensional model of the dynamic targets; when rendering the three-dimensional view, the rear platform structure is treated as semi-transparent to highlight the surrounding targets, and the display accuracy is dynamically adjusted according to the distance between the target and the rear platform - the closer the target, the higher the accuracy of the mesh model rendering the contour details; in the view display control link, the slewing angle, trolley position and height and other attitude data of the tower crane are obtained in real time, the display scale of the two-dimensional bird's eye view and the sectioning viewing angle of the three-dimensional local zoom-in view are dynamically adjusted according to the field of view range corresponding to the current viewing angle of the cab, ensuring that the display content of the visual aid image is mapped in real time with the actual working position of the tower crane, and the view refresh thread and the radar data acquisition thread are kept consistent in frequency through a synchronization lock mechanism.

[0063] The application of computer graphics technology in this embodiment includes the following core technical links:

[0064] One, three-dimensional space modeling and coordinate transformation technology, including: coordinate system construction: based on the tower machine rotation center to establish a three-dimensional world coordinate system, convert radar point cloud data into relative coordinates with the tower machine as the origin, solve the spatial unified reference problem of multi-sensor data (corresponding to the coordinate space transformation technology in computer graphics, including translation, rotation matrix operation). Data filtering: filter invalid environmental data (such as long-distance noise points) through a pre-set distance threshold, which is equivalent to the space clipping (Clipping) technology in graphics, ensuring that subsequent processing only focuses on the effective work area.

[0065] Two, two-dimensional view generation technology, including: orthographic projection transformation: orthographic projection of three-dimensional point cloud data to horizontal XY plane, generating the basic framework of two-dimensional bird's eye view, directly applying orthogonal projection (Orthographic Projection) technology, preserving the true size ratio of the target. Region division and visualization: based on tower machine structure parameters (driver's cabin width, radar pitch angle, etc.) to delimit blind area sector and rear rectangular area, which is essentially a geometric region definition in graphics (such as sector described by polar coordinate equation, rectangle defined by axis-aligned bounding box AABB). Use semi-transparent polygon filling algorithm (such as scan line filling algorithm) to mark the blind area with color blocks, use Alpha blending technology to achieve semi-transparent effect, ensure the visual fusion of marked area and background environment. Dynamic target visualization: mark dynamic targets with geometric symbols (such as circles, triangles), draw motion direction arrows based on target tracking results (essentially line drawing and arrow symbol generation algorithm), which belongs to basic primitive rendering technology in computer graphics.

[0066] Three, three-dimensional view generation technology, including: local area modeling: delimit a 10-meter radius spherical area centered on the tower machine rear platform, extract point cloud data within the area, which corresponds to spatial query technology in graphics (such as sphere bounding box detection). Point cloud processing and mesh generation: separate independent targets through point cloud clustering algorithm (such as DBSCAN), providing structured data for subsequent modeling, which belongs to geometric processing technology in the preprocessing stage. Use mesh modeling algorithm (such as triangular meshing) to generate obstacle contour frame and dynamic target simplified model, directly apply surface reconstruction technology in three-dimensional modeling to convert point cloud data into renderable mesh structure (Mesh). Rendering optimization: semi-transparent processing of the rear platform structure (set transparency through material properties), and dynamically adjust mesh rendering precision according to target distance (high subdivision near, low subdivision far), which is a combination of level of detail (LOD, Level of Detail) technology and transparent rendering technology in graphics.

[0067] Four, visual interactive control technology, including: dynamic adjustment of view angle: real-time acquisition of tower crane attitude data (rotation angle, trolley position, etc.), dynamic adjustment of two-dimensional view display scale and three-dimensional view section angle, the essence is the view transformation (View Transformation) technology in computer graphics, to ensure that the visual content matches the actual field of view of the cab. Synchronous rendering control: through thread synchronization lock mechanism to ensure that the view refresh frequency is consistent with the radar scanning frequency, which belongs to the synchronization control technology of rendering pipeline and data acquisition in real-time computer graphics, to avoid picture tearing or data delay.

[0068] Five, color and risk mapping technology, including: using preset rules to map risk levels to red, yellow and blue fill colors, which belongs to the color mapping (Color Mapping) and visualization coding technology in computer graphics, to improve the driver's recognition efficiency of risks through visual coding and solve the problem of information overload.

[0069] The combination of the above computer graphics technologies breaks through the limitations of traditional tower crane blind area monitoring, including: two-dimensional bird's eye view through orthogonal projection and region annotation, which compresses three-dimensional space information into easy-to-understand plane view, solving the problem of visualization abstraction of multi-dimensional data; three-dimensional local magnification view through grid modeling and LOD technology, which reduces the computational load while preserving details, achieving a balance between high precision and real-time; dynamic synchronization and view angle adaptation through coordinate transformation and view control, which makes the virtual model and the actual working state of the tower crane real-time mapping, forming a complete closed loop of "environment perception-data processing-graphics rendering-human-computer interaction", and forming an engineering practical visualization compensation scheme.

[0070] Step S103. Collecting operation data of the tower crane, predicting the motion trajectory of the hoisted object and the motion trajectory of the surrounding people and vehicles based on the operation data, and determining whether a dynamic target enters the blind area.

[0071] Specifically, by collecting tower crane operation data (such as rotation angle, amplitude speed), combining with the position of dynamic targets in the environment model, the motion trajectory of the hoisted object and the surrounding people and vehicles is predicted to determine whether it enters the blind area, solving the problem of risk prediction deficiency.

[0072] Operation data collection is achieved by accessing tower crane sensor data: rotation encoder (precision ±0.1°), amplitude motor speed sensor, lifting height sensor, hook load sensor, to obtain real-time tower crane state (such as current amplitude R=20 meters, rotation angle θ=45°, hook height H=30 meters). The hoisted object motion model assumes that the hoisted object moves at a constant speed during amplitude variation, and the position coordinates (R(t+Δt), θ(t+Δt), H(t+Δt)) in the future 1~3 seconds are predicted by combining wind resistance coefficient correction.

[0073] Dynamic target trajectory prediction predicts the future moving direction and speed of dynamic targets such as personnel and vehicles by using Kalman Filter or Interactive Multiple Model (IMM) algorithm according to historical trajectory (past 5 frames of position), for example: walking speed of personnel: 1~1.5m / s, random turning probability modeling; driving speed of vehicle: 5~10m / s, path planning prediction along the construction road.

[0074] Blind area entry judgment defines the blind area space range as a three-dimensional cylindrical / pie area (for example, two side blind areas: with the tower crane as the center, ±20 meters horizontally, 0~50 meters longitudinally, and 0~15 meters in height; rear blind area: a radius of 30 meters behind the tower crane, and 0~10 meters in height). The intersection of the predicted trajectory and the blind area space is calculated in real time, and if a dynamic target enters the blind area boundary (buffer distance 1 meter) within 2 seconds in the future, a warning is triggered; if it enters the core blind area (distance <5 meters), an emergency warning is triggered.

[0075] Step S104. If it is judged that a dynamic target enters the blind area, generate a warning information; feed back the visual auxiliary image and the warning information to the tower crane driving interface.

[0076] Specifically, by integrating the visual auxiliary image and the hierarchical warning information into the driving interface, the driver's response efficiency is improved through multi-modal interaction, and the problem of low efficiency of human-computer interaction is solved.

[0077] The warning information generation includes: sound warning: first level risk (entering the core blind area): continuous bee sound + voice "personnel approaching from behind!"; second level risk (entering the buffer area): intermittent bee sound + voice "vehicle approaching the blind area from the left!". Visual warning: in the visual interface, a red frame (first level) / yellow frame (second level) is displayed around the dynamic target, and a moving direction arrow and an estimated entering time (such as "1.5 seconds later into the blind area") are marked.

[0078] The driving interface design includes: main interface layout: the left side is a three-dimensional environment overhead view (showing the relative position of the tower crane and the surrounding targets), the right side is a side view rendered by real-time point cloud (emphasizing the blind area on both sides of the cab), and the lower part displays the warning log scrolling. AR augmented display: through the HUD (head-up display) of the front windshield of the cab, the risk area outline (such as a red semi-transparent pie) is superimposed on the real field of view to assist the driver in quickly locating the blind area position.

[0079] The interactive response mechanism supports the driver to manually switch the view angle (such as clicking the screen to view the rear close-up), but when the first level warning occurs, a full screen warning is forced to pop up, and non-emergency operations are shielded; the warning information is automatically associated with the tower crane operation permission: when a dynamic target enters the core blind area and the moving direction of the hoisted object may cause a collision, the system automatically limits the rotation / amplitude change speed of the tower crane until the risk is eliminated.

[0080] In some embodiments, the environment data of the two side areas of the cab and the rear area of the tower body are obtained by scanning the elevation angle of the two side areas of the cab and the rear area of the tower body through the rotating radar module installed below the tower trolley, including: controlling the rotating radar module to rotate horizontally at a preset angular velocity of 360 degrees, and performing layered scanning at preset elevation angle intervals during rotation, and collecting distance, azimuth and reflection intensity data of each scanning point in real time for each preset fan-shaped area on both sides of the cab and the rectangular area behind the tower body to form a point cloud data sequence; wherein the preset elevation angle interval and the scanning area range are dynamically adjusted according to the current working amplitude and height parameters of the tower crane.

[0081] By controlling the horizontal rotation and elevation layered scanning of the rotating radar module, dynamic coverage of the two side fan-shaped blind areas and the rear rectangular blind area of the tower crane is realized, and the scanning parameters are adaptively adjusted according to the real-time working condition of the tower crane.

[0082] The scanning control mechanism includes: horizontal rotation: the radar module is driven by a servo motor to rotate at a uniform speed of 10° / s~30° / s (such as 20° / s, single circle scanning time 18 seconds) at a preset angular velocity of 360°, ensuring full-range environment data collection. Elevation layering: during rotation, layered scanning is performed at an elevation angle interval of 5°~15° (such as every 10° as a layer), for example, covering a range of -30° (looking down at the ground) to +30° (looking up at high altitude), and the scanning duration of each layer matches the horizontal rotation speed.

[0083] Region-specific scanning includes: two side fan-shaped areas: define a single blind area as a fan-shaped area with the tower crane as the apex, a central angle of 60°, and a radius of 50 meters (one on the left and one on the right), and when scanning, preferentially increase the point cloud density of this area (such as shorten the scanning interval to 2°). Rectangular area behind: define the area behind the tower body as a rectangular area with a length of 80 meters, a width of 30 meters, and a height of 20 meters, and when the tower crane turning angle is ≥180° (the jib is facing the rear), automatically increase the scanning frequency of this area.

[0084] Dynamic parameter adjustment dynamically reduces the blind area scanning radius to R+20 meters according to the current working amplitude of the tower crane (such as trolley position R=30 meters), to avoid invalid long-distance scanning; combined with height sensor data (such as hook height H=40 meters), adjust the elevation scanning range to H±10 meters to focus on the risk area around the working height.

[0085] In some embodiments, the construction environment model of the tower crane working scene is constructed based on the acquired environment data, including: time stamp alignment and spatial coordinate conversion of the continuously collected point cloud data sequence, converting the radar coordinate system data into global coordinate system data with the tower crane rotation center as the origin; denoising the global coordinate system data by a voxel filtering algorithm to remove noise points and invalid data at a long distance; generating a static environment model containing three-dimensional terrain, fixed buildings, temporary facilities and vegetation based on multi-frame point cloud data stitching technology, and marking the real-time position contour of the tower crane structure components to form a dynamically updated construction environment three-dimensional grid model.

[0086] A dynamic three-dimensional environment model centered on the tower crane is constructed through space-time calibration, denoising and multi-frame stitching of point cloud data, containing static facilities and the contour of the tower crane structure itself.

[0087] The space-time data processing includes: time stamp alignment: attaching accurate time stamps (accuracy ±1 ms) to the point cloud data collected by the radar, combined with the tower crane encoder synchronization signal (such as triggering once per 1° change of the rotation angle), to ensure the time consistency of multiple frames of data. Coordinate conversion: converting the radar coordinate system (with the radar installation center as the origin) into the global coordinate system (with the tower crane rotation center as the origin, the X-axis pointing to the initial direction of the boom, and the Z-axis vertically upward), and the conversion formula includes a translation matrix (offset of the radar relative to the tower crane center) and a rotation matrix (IMU attitude angle compensation).

[0088] Data denoising and modeling includes: voxel filtering: setting a 3D voxel grid (such as 0.2m x 0.2m x 0.2m), taking the mean of the point cloud in each voxel, and removing sparse noise points and invalid data at a distance >100 meters; multi-frame stitching: aligning consecutive scanning frames using the ICP (Iterative Closest Point) algorithm, combining with the tower crane rotation angle encoder data, generating a globally consistent static environment model, and real-time labeling of the contour point cloud of the tower crane structure (such as the balance arm, the tower body).

[0089] The model updating mechanism triggers local model updating when the tower crane changes the amplitude (trolley moves) or rotates, and only re-stitches the affected area (such as a radius of ±5 meters), improving processing efficiency.

[0090] In some embodiments, the construction environment model is processed by using the job behavior recognition algorithm to identify corresponding potential obstructions and hazards, comprising: inputting the construction environment model into a preset deep learning neural network model, identifying the types, positions and contour sizes of static obstructions and dynamic targets through a target detection module; wherein the static obstructions at least include buildings, scaffolds and piled materials, and the dynamic targets at least include construction vehicles, personnel and mobile machinery; dividing the risk levels of blind areas and rear platform areas through a semantic segmentation module, marking fixed obstacles causing line-of-sight obstruction and dynamic targets entering dangerous areas, and outputting the recognition results containing target categories, risk levels and spatial positions.

[0091] The static obstructions and dynamic targets are identified based on the deep learning model, the risk levels are divided through semantic segmentation, and the structured recognition results are output.

[0092] The deep learning model architecture comprises: a target detection module: using PointNet++ or PointRCNN network, inputting point cloud data (containing XYZ coordinates + reflectivity), and outputting the categories of static obstructions (buildings, scaffolds, material piles) and dynamic targets (vehicles, personnel, forklifts), and bounding box coordinates (XYZ, length, width, height, heading angle). A semantic segmentation module: using DeepLabv3+ or U-Net variants, classifying points by points for blind area space, labeling “permanent obstruction area” (such as a fence), “temporary obstruction area” (such as a mobile scaffold), and “dynamic risk area” (personnel activity area), and the risk levels are divided according to the distance from the tower crane center (<5 meters for high risk, 5-15 meters for medium risk, >15 meters for low risk).

[0093] The recognition result output comprises: structured data containing target ID, category (such as “construction personnel” and “concrete truck”), risk level (L1 / L2 / L3), real-time coordinates (X, Y, Z), contour size (length x width x height), and motion speed (dynamic target).

[0094] In some embodiments, the operation data of the tower crane is collected, comprising: acquiring the gear signals of the operation handle, the rotation speeds and current data of each mechanism motor, the hook height, trolley position and jib rotation angle feedback by the encoder through the tower crane controller, collecting real-time data of the tower crane inclination sensor, wind speed sensor and weight sensor, and forming a multi-dimensional operation data sequence containing operation instructions, mechanism states and environmental parameters.

[0095] The operation instructions, mechanism states and environmental parameters of the tower crane are collected to form a multi-dimensional data containing time sequence, providing input for trajectory prediction.

[0096] The data acquisition range includes: operation instructions: main / auxiliary hook handle gear position (0-10 gears), amplitude handle direction (left / right) and gear position, rotation handle direction (left / right) and gear position, resolution ±0.5 gears. Mechanism state: each motor speed (encoder pulse count, accuracy ±1 rpm), current (Hall sensor, accuracy ±5%), hook height (absolute value encoder, accuracy ±1 cm), trolley position (amplitude encoder, accuracy ±2 cm), rotation angle (absolute value encoder, accuracy ±0.1°). Environmental parameters: inclination sensor (X / Y axis inclination angle, accuracy ±0.5°), wind speed sensor (wind speed / direction, accuracy ±0.5 m / s), weight sensor (weight of the hoisted object, accuracy ±1% FS).

[0097] The data sequence is constructed by packing data at 20 ms intervals to form a multi-dimensional vector containing timestamps, operation instructions, mechanism states, and environmental parameters, such as: [t, main hook gear position, amplitude speed, rotation angle, hook height, wind speed, inclination angle].

[0098] In some embodiments, the motion trajectory of the hoisted object and the motion trajectory of the surrounding people and vehicles are predicted based on the operation data, and it is determined whether a dynamic target enters the blind area, including: for the hoisted object, the spatial trajectory in the future 1-3 seconds is predicted using a kinematic model according to the current hook position, trolley speed, and hoist arm rotation angular velocity; for the surrounding dynamic target, its future motion direction and speed are predicted based on historical position data through a Kalman filter algorithm; a blind area spatial coordinate system is established, and the intersection of the predicted trajectory and the blind area space is calculated in real time, and when the predicted position of the dynamic target enters the 0.5-meter range within the blind area boundary, it is determined that the dynamic target enters the blind area.

[0099] The trajectory of the hoisted object is predicted through a kinematic model, the trajectory of the dynamic target is predicted through a Kalman filter, and it is determined whether the dynamic target enters the blind area based on the spatial intersection.

[0100] The hoisted object trajectory prediction includes: kinematic model: assuming that the trolley amplitude speed v_r (m / s), rotation angular velocity ω (° / s), and hoisting speed v_h (m / s) are constant, the hoisted object position after t seconds is calculated as: radial distance: R(t) = R0 + v_r*t; rotation angle: θ(t) = θ0 + ω·t*π / 180; height: H(t) = H0 + v_h*t; (when wind load is considered, a lateral offset compensation Δx = 0.1*v_w*t is added, and v_w is the wind speed). Dynamic target prediction includes Kalman filtering: the state vector is defined as [x, y, z, vx, vy, vz], the transition matrix considers the uniform motion model, the observation matrix matches the radar point cloud coordinates, the update frequency is consistent with the radar (10 Hz), and the future 1-3 second position is predicted (step size 0.1 second).

[0101] The blind area space is defined as: two-side blind area: |θ-90°|≤30° (30° sector on the left / right), R≤50 meters, Z≤20 meters; rear blind area: θ=180°±45°, R≤80 meters, Z≤15 meters; when the minimum distance between the predicted position and the blind area space is less than 0.5 meters (buffer boundary), it is determined that the dynamic target has entered the blind area.

[0102] In some embodiments, when it is determined that the dynamic target has entered the blind area, the pre-warning information is generated, including: generating the pre-warning information according to the type, speed and risk level of the dynamic target entering the blind area; the pre-warning information includes a first pre-warning signal and a second pre-warning signal; the first pre-warning signal and the second pre-warning signal both include the target type, the entering position and the remaining time to the dangerous distance; when a person is detected to enter a low-risk blind area at a speed lower than 1.5 m / s, the first pre-warning signal is generated; when a vehicle is detected to enter a medium-risk blind area at a speed higher than 5 km / h or a person enters a high-risk blind area, the second pre-warning signal is generated.

[0103] According to the target type, speed and risk area, two levels of pre-warning signals are generated to clearly indicate the risk urgency.

[0104] The pre-warning grading rules include: first pre-warning (low urgency): trigger condition: a person enters a low-risk blind area (distance>15 meters) at a speed of ≤1.5 m / s (normal walking); signal characteristics: yellow warning box + intermittent bee sound (frequency 1 Hz), voice prompt “pay attention to the person approaching on the left”. Second pre-warning (high urgency): trigger condition: a vehicle enters a medium-risk blind area (5-15 meters) at a speed of >5 km / h (1.39 m / s); a person enters a high-risk blind area (distance<5 meters); signal characteristics: red warning box + continuous bee sound (frequency 2 Hz), voice prompt “vehicle approaching from behind at high speed! Stop immediately and turn around!”, and mark the remaining time (such as “2 seconds remaining to reach the dangerous distance”).

[0105] The remaining time is calculated according to the current speed of the target and the dangerous distance (such as the core blind area boundary of 5 meters): remaining time=distance / speed, accuracy ±0.1 seconds.

[0106] In some embodiments, the visualization auxiliary image and the pre-warning information are fed back to the tower crane driving interface, including: dividing independent display areas on the tower crane cab display screen, displaying the visualization auxiliary image on the left side area full screen, and displaying the current identified hazard source list on the right side area in real time; when the pre-warning information is generated, a floating warning window is popped up at the top of the display screen, covering 20% of the current display content, and the pre-warning information automatically disappears after the dynamic target leaves the blind area.

[0107] By dividing a special display area on the driving interface, integrating the visualization image and the pre-warning information, and prompting the risk through the floating window.

[0108] Interface layout design includes left main area (60% screen): full-screen display of two-dimensional bird's-eye view + three-dimensional local map (switchable), real-time rendering of blind area boundary, obstacles and dynamic targets. Right list area (40% screen): scroll display of the current identified hazard source list, each record containing: target type, distance, risk level, position direction (such as "3 o'clock direction behind, distance 8 meters, L2 level").

[0109] Early warning interaction mechanism includes: floating warning window: when the early warning is triggered, a red semi-transparent window (covering 20% area) is popped up at the top of the screen, displaying core information (target type + distance + icon), and clicking expands the detailed trajectory; automatic disappearance rule: when the dynamic target is more than 1 meter away from the blind area boundary and lasts for 5 seconds, the early warning window automatically closes to avoid driver manual confirmation interference. The display screen uses a 12-inch industrial touch screen with a brightness of ≥500 nit and supports glove operation; the HUD synchronously displays the early warning icon (such as a red arrow pointing to the tail when rear warning).

[0110] In some embodiments, to solve the problem of small target missing detection caused by sparse radar point cloud in complex construction environment, visual sensor and laser radar fusion are introduced to improve the identification accuracy of dynamic targets (such as personnel without safety helmet and small mobile machinery) through cross-modal data enhancement.

[0111] A 120° wide-angle RGB camera (resolution 1920x1080, frame rate 30fps) is installed beside the radar module, with the lens facing the blind area on both sides of the cab and the rear, synchronously collecting visual images; the radar and camera external parameters (translation vector T, rotation matrix R) are calibrated to establish the mapping relationship between pixel coordinates and three-dimensional point cloud (such as through Zhang's calibration method + ICP point cloud registration).

[0112] Cross-modal processing at the algorithm layer includes: data alignment through timestamp synchronization (error < 50ms) of radar point cloud (XYZ + reflectivity) and image pixels (RGB), and through ROI projection to map point cloud clusters to the corresponding image area; feature fusion model through constructing Point-Image Fusion network, inputting point cloud BEV (bird's-eye view) features and image CNN features (extracted by ResNet), enhancing the feature expression of small targets (such as personnel head) through attention mechanism (Cross-Attention), and outputting three-dimensional target detection results containing visual semantics (such as distinguishing between "personnel wearing safety helmet" and "personnel not wearing safety helmet").

[0113] The detection accuracy for people within 5 meters is improved, and the minimum target size is reduced from 30 cm to 15 cm (such as identifying ground tool kits and other easy-to-collision objects); combined with visual color features (such as red safety hats and yellow warning clothes), the risk level classification of dynamic targets is optimized (persons without safety hats are automatically marked as secondary risk).

[0114] In some embodiments, to solve the problem of lag in sudden risk response for traditional fixed scanning strategy, deep reinforcement learning (DRL) is introduced to dynamically adjust the radar scanning frequency and pitch angle according to the real-time risk level, realizing "risk-driven" intelligent scanning.

[0115] The state-action space definition includes: state S: contains 12-dimensional features such as the number of current blind area targets, risk level distribution, tower operation parameters (luffing speed, rotation angle), and radar remaining power; action A: adjustable parameters include scanning angular velocity (10° / s, 20° / s, 30° / s), pitch layer interval (5°, 10°, 15°), and focus on key areas (enhanced scanning to the left / right / back); reward R: risk target missed detection deducts 10 points, scanning efficiency improvement (increase of effective data volume per unit time) adds 5 points, and power consumption reduction adds 3 points, to build a long-term maximum benefit objective function.

[0116] Training and online optimization include: offline training: using construction scene simulation data (such as Gazebo to build tower operation environment), adopting PPO (proximal policy optimization) algorithm to train strategy network, and outputting optimal scanning parameter combination; online fine-tuning: edge controller collects actual working condition data in real time, updates strategy model every 5 minutes, and adapts to environmental differences of different construction sites (such as dense building groups vs. open sites).

[0117] Dynamic strategy execution automatically increases the scanning frequency to 20 Hz and reduces the pitch interval to 5° when a rear vehicle is detected to approach at high speed (secondary risk), to preferentially ensure the point cloud density of the risk area; when the risk is low (such as night shutdown), the scanning speed is reduced to 10° / s, and the device endurance time is extended.

[0118] In some embodiments, to solve the problem of low spatial cognitive efficiency of traditional 2D visualization interface, a three-dimensional immersive navigation system based on AR glasses is developed to fuse virtual risk labeling with real scene, realizing "no visual line transfer" risk perception.

[0119] AR hardware and registration technology equip industrial-grade AR glasses (such as HoloLens 2, field of view 52°, resolution 2Kx2K) to obtain the driver's head pose in real time through the UWB positioning module (accuracy ±10 cm); establish the mapping relationship between the tower crane coordinate system and the AR space, and realize virtual-real registration by using inertial tracking + environment feature point matching (ORB-SLAM) to ensure that the risk labeling is stable and fits the real scene (such as superimposing the virtual contour of the rear obstacle on the actual field of view of the driver).

[0120] Immersive interaction design includes: three-dimensional risk labeling: in the AR field of view, render the blind area boundary with a red translucent grid, and display dynamic targets with a glowing contour (person → human contour, vehicle → vehicle contour), and trigger contour flashing when the distance is <5 meters; voice interaction linkage: the driver can trigger the AR view to automatically turn left by voice command "view the left blind area", and simultaneously enlarge and display the point cloud details in that area; tactile feedback enhancement: combined with a wearable vibrating bracelet, when a risk target enters the core blind area, the vibration module in the corresponding direction (left / right / back) vibrates at high frequency, forming a multi-modal warning.

[0121] When it is detected that the driver's line of sight deviates from the operation area for a long time (judged by eye tracking), the AR labeling brightness is automatically enhanced and a voice prompt "please pay attention to the current operation area" is given; offline mode is supported (when the network is interrupted, AR assistance is continuously provided based on the local environment model). To solve the problem of insufficient model generalization ability caused by differences in different construction site environments, an edge computing and cloud AI collaborative architecture is constructed to realize adaptive updating of the detection model and global data sharing.

[0122] Hierarchical architecture design includes: edge layer: deploy a lightweight target detection model (such as a quantized version of PointPillars, model size <100MB) to process local point cloud and visual data in real time, and output structured risk data (target type, coordinates, risk level); cloud layer: collect desensitized data uploaded by each construction site edge node (remove location information, only keep target categories and environmental features), train a global model through federated learning (Federated Learning), and push updated parameters every two weeks.

[0123] Dynamic updating mechanism through incremental learning: after the edge node detects cloud model updates, fine-tune on local cached data first (such as adding a new "new construction machinery" category), compare the difference between the new and old model detection results (D-Score>0.3 triggers retraining), and avoid wasting computing power; abnormal data feedback: when a construction site continuously detects unannotated target types (such as drones entering the operation area) for 3 times, it is automatically labeled as "unknown target" and encrypted to upload to the cloud, triggering special scene model iteration.

[0124] Data security is guaranteed by the edge node communicating with the cloud using the SM4 encryption algorithm, and the transmission data is processed by differential privacy (epsilon=0.5) to ensure that the site location and operation details are not leaked. The cloud model update package is accompanied by a digital signature, and the edge node will only replace it after verification to prevent malicious tampering.

[0125] Please refer to Figure 3 as shown, Figure 3 is a structural diagram of a tower crane cab blind area and rear platform perception compensation system 200 provided by the embodiments of the present application. The tower crane cab blind area and rear platform perception compensation system 200 is used to execute the steps of the tower crane cab blind area and rear platform perception compensation method shown in each of the above embodiments. The tower crane cab blind area and rear platform perception compensation system 200 can be a single server or a server cluster, or the tower crane cab blind area and rear platform perception compensation system 200 can be a terminal, which can be a handheld terminal, a notebook computer, a wearable device, or a robot, etc.

[0126] As Figure 3 shown, the tower crane cab blind area and rear platform perception compensation system 200 includes:

[0127] A data acquisition unit 201 is configured to scan the elevation angle of the area on both sides of the cab and the area behind the tower body by a rotating radar module integratedly installed below the tower crane trolley, and acquire the environmental data of the area on both sides of the cab and the area behind the tower body.

[0128] A model construction unit 202 is configured to construct a construction environment model of a tower crane operation scene based on the acquired environmental data, process the construction environment model by using an operation behavior recognition algorithm to identify corresponding potential occlusions and hazards, and generate a visual auxiliary image containing blind area and rear platform area information in real time according to the identification result, including: generating a two-dimensional bird's eye view and a three-dimensional local magnification view by using computer graphics technology based on the construction environment model and the identification result; in the two-dimensional bird's eye view, marking the blind area on both sides of the cab and the area behind the tower body with semi-transparent blocks, distinguishing the risk levels by different colors, and superimposing and displaying the real-time positions and motion direction arrows of dynamic targets; in the three-dimensional local magnification view, highlighting the obstacle profiles within a 10-meter range around the rear platform and the details of dynamic targets; the refresh frequency of the visual auxiliary image is synchronized with the radar scanning frequency, and the display scale is adjusted according to the cab viewing angle.

[0129] A data acquisition unit 203 is configured to acquire operation data of the tower crane, predict the motion trajectories of the hoisted object and the surrounding people and vehicles based on the operation data, and determine whether a dynamic target enters the blind area.

[0130] The early warning generation unit 204 is configured to generate early warning information if it is determined that a dynamic target enters the blind area, and feed back the visual auxiliary image and the early warning information to a tower crane driving interface.

[0131] It should be noted that, for the convenience and brevity of description, the specific working processes of the tower crane cab blind area and rear platform perception compensation system and the modules described above can be clearly understood by those skilled in the art, and the corresponding contents in the above embodiments of the tower crane cab blind area and rear platform perception compensation method will not be repeated here.

[0132] The tower crane cab blind area and rear platform perception compensation method described above can be implemented in the form of a computer program, which can run on the device as shown. Figure 3

[0133] Please refer to Figure 4 , Figure 4 is a structural schematic block diagram of the controller provided by the embodiment of the present application. The controller includes a processor, a memory and a network interface connected through a device bus, wherein the memory can include a storage medium and an internal memory.

[0134] The storage medium can store an operating device and a computer program. The computer program includes program instructions, which, when executed, can cause the processor to execute any kind of tower crane cab blind area and rear platform perception compensation method.

[0135] The processor is configured to provide computing and control capabilities to support the operation of the entire controller.

[0136] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium, which, when executed by the processor, can cause the processor to execute any kind of tower crane cab blind area and rear platform perception compensation method.

[0137] The network interface is configured to perform network communication, such as sending assigned tasks. Those skilled in the art can understand that Figure 4 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the terminal to which the scheme of the present application is applied. The specific controller can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0138] ​It should be appreciated that the processor can be a central processing unit (CPU), the processor can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0139] In one embodiment, the processor is configured to execute a computer program stored in the memory to implement the following steps:

[0140] The environment data of the two sides of the cab and the area behind the tower body are obtained by scanning the tilt angle of the two sides of the cab and the area behind the tower body through the rotating radar module integrated below the trolley of the tower crane.

[0141] Based on the obtained environment data, a construction environment model of the tower crane working scene is constructed, and an operation behavior recognition algorithm is used to process the construction environment model to identify corresponding potential obstacles and hazards. According to the identification result, a visual auxiliary image containing blind area and rear platform area information is generated in real time, including: based on the construction environment model and the identification result, a two-dimensional bird's eye view and a three-dimensional local enlarged view are generated by using computer graphics technology; in the two-dimensional bird's eye view, the two sides of the cab and the area behind the tower body are marked with semi-transparent blocks, and the risk levels are distinguished by different colors, and the real-time positions and motion direction arrows of dynamic targets are superimposed and displayed; in the three-dimensional local enlarged view, the obstacle profiles within 10 meters around the rear platform and the details of dynamic targets are highlighted; the refresh frequency of the visual auxiliary image is synchronized with the radar scanning frequency, and the display scale is adjusted according to the cab viewing angle;

[0142] The operation data of the tower crane is collected, the motion trajectories of the hoisted objects and the surrounding people and vehicles are predicted based on the operation data, and it is judged whether a dynamic target enters the blind area;

[0143] If it is judged that a dynamic target enters the blind area, a warning information is generated; the visual auxiliary image and the warning information are fed back to the tower crane driving interface.

[0144] In some embodiments, the rotating radar module installed below the trolley of the tower crane is used to scan the elevation angle of the areas on both sides of the cab and the area behind the tower body, and to obtain the environmental data of the areas on both sides of the cab and the area behind the tower body, including: controlling the rotating radar module to rotate horizontally at a preset angular velocity for 360 degrees, and performing layered scanning at preset elevation angle intervals during rotation, and collecting distance, azimuth and reflectivity data of each scanning point in real time for each preset fan-shaped area on both sides of the cab and the rectangular area behind the tower body, to form a point cloud data sequence; wherein the preset elevation angle interval and the scanning area range are dynamically adjusted according to the current working amplitude and height parameters of the tower crane.

[0145] In some embodiments, based on the obtained environmental data, a construction environment model of the tower crane working scene is constructed, including: time stamp alignment and spatial coordinate conversion are performed on the continuously collected point cloud data sequence, the radar coordinate system data is converted into global coordinate system data with the center of the tower crane as the origin; the global coordinate system data is denoised by a voxel filtering algorithm to remove noise points and invalid data at a long distance; based on a multi-frame point cloud data stitching technology, a static environment model containing three-dimensional terrain, fixed buildings, temporary facilities and vegetation is generated, and the real-time position contour of the tower crane structure component is marked to form a dynamically updated construction environment three-dimensional grid model.

[0146] In some embodiments, the construction environment model is processed using a working behavior recognition algorithm to identify corresponding potential obstructions and hazards, including: inputting the construction environment model into a preset deep learning neural network model, and identifying the type, position and contour size of static obstructions and dynamic targets through a target detection module; wherein the static obstructions at least include buildings, scaffolds and piled materials, and the dynamic targets at least include construction vehicles, personnel and mobile machinery; the risk level of the blind area and the rear platform area is divided by a semantic segmentation module, and the fixed obstacles causing line-of-sight obstruction and the dynamic targets entering the dangerous area are marked, and the identification results containing target categories, risk levels and spatial positions are output.

[0147] In some embodiments, the operation data of the tower crane is collected, including: acquiring the gear signal of the operating handle, the rotation speed and current data of each mechanism motor, the encoder feedback of the hook height, the trolley position and the jib rotation angle through the tower crane controller in real time, collecting the real-time data of the tower crane inclination sensor, the wind speed sensor and the weight sensor, and forming a multi-dimensional operation data sequence containing operation instructions, mechanism states and environmental parameters.

[0148] In some embodiments, the operation data is used to predict the movement trajectory of the hoist and the movement trajectory of the surrounding people and vehicles, and to determine whether a dynamic target enters the blind area, including: for the hoist, the spatial trajectory in the next 1-3 seconds is predicted by using a kinematic model according to the current hook position, trolley speed and slewing angle speed; for the surrounding dynamic target, the future movement direction and speed are predicted by using a Kalman filtering algorithm based on historical position data; a blind area spatial coordinate system is established, and the intersection of the predicted trajectory and the blind area space is calculated in real time, and when the predicted position of the dynamic target enters the 0.5-meter range of the blind area boundary, it is determined that the dynamic target enters the blind area.

[0149] In some embodiments, if it is determined that a dynamic target enters the blind area, a warning information is generated, including: according to the type, speed and risk level of entering the blind area of the dynamic target, a warning information is generated: the warning information includes a first warning signal and a second warning signal; the first warning signal and the second warning signal both contain the target type, the entering position and the remaining time to the dangerous distance; wherein, when a person is detected to enter a low-risk blind area at a speed lower than 1.5m / s, a first warning signal is generated; when a vehicle is detected to enter a medium-risk blind area at a speed higher than 5km / h or a person enters a high-risk blind area, a second warning signal is generated.

[0150] In some embodiments, the visualization auxiliary image and the warning information are fed back to the tower crane driving interface, including: an independent display area is divided on the display screen of the tower crane cab, the left area displays the visualization auxiliary image in full screen, and the right area displays a list of currently identified dangerous sources in real time; when a warning information is generated, a floating warning window is popped up at the top of the display screen, covering 20% of the current display content, and the warning information automatically disappears after the dynamic target leaves the blind area.

[0151] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed in the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A tower crane cab blind area and back platform perception compensation method, characterized in that, The application relates to a tower crane blind area monitoring method and device. The application comprises the following steps: a rotating radar module is installed below a trolley of a tower crane to scan the elevation angles of regions on both sides of a cab and a region behind a tower body, and environment data of the regions on both sides of the cab and the region behind the tower body are acquired; a construction environment model of a tower crane working scene is constructed based on the acquired environment data, including the following steps: time stamp alignment and space coordinate conversion are performed on a continuous point cloud data sequence to convert radar coordinate system data into global coordinate system data with the center of the tower crane as an origin; a voxel filtering algorithm is used to remove noise and invalid data at a long distance; a static environment model containing a three-dimensional terrain, fixed buildings, temporary facilities and vegetation is generated based on a multi-frame point cloud data splicing technology, and a real-time position contour of a tower structure component is marked to form a dynamically updated construction environment three-dimensional grid model; wherein, the time and space data processing comprises the following steps: a precise timestamp of precision of plus or minus 1 ms is added to the point cloud data collected by the radar, and the timestamp is triggered once per 1 DEG when the rotation angle changes; the radar coordinate system with the center of the radar installation as the origin is converted into the global coordinate system with the center of the tower crane as the origin, the X axis points to the initial direction of the crane boom, the Z axis is perpendicular upward, and the conversion formula comprises a translation matrix corresponding to the offset of the radar relative to the center of the tower crane and a rotation matrix corresponding to the IMU attitude angle compensation; the construction environment model is processed by using a working behavior recognition algorithm to identify corresponding potential shielding objects and danger sources; a visual auxiliary image containing blind area and rear platform region information is generated in real time according to the identification result, including the following steps: a two-dimensional bird's eye view and a three-dimensional local enlarged view are generated by using computer graphics technology based on the construction environment model and the identification result; in the two-dimensional bird's eye view, the blind area on both sides of the cab and the region behind the tower body are marked by using a semi-transparent block, different colors are used to distinguish risk levels, and the real-time position and motion direction arrow of a dynamic target are superimposed and displayed; in the three-dimensional local enlarged view, the obstacle contour within a range of 10 meters around the rear platform and the dynamic target details are highlighted; the refresh frequency of the visual auxiliary image is synchronous with the radar scanning frequency, and the display scale is adjusted according to the cab visual angle; Operation data of the tower crane are collected, the motion trajectories of a hoisted object and surrounding people and vehicles are predicted based on the operation data, and it is judged whether a dynamic target enters the blind area; If it is judged that a dynamic target enters the blind area, a warning information is generated; the visual auxiliary image and the warning information are fed back to a tower crane driving interface. The scanning of the elevation angles of the regions on both sides of the cab and the region behind the tower body by the rotating radar module installed below the trolley of the tower crane to acquire the environment data of the regions on both sides of the cab and the region behind the tower body comprises the following steps:

2. The method of claim 1, wherein, ​ The rotating radar module is controlled to rotate horizontally at a preset angular velocity for 360 degrees, and layered scanning is performed at preset pitch angles during rotation, real-time distance, azimuth and reflection intensity data of each scanning point are collected for each preset fan-shaped region on both sides of the cab and the rectangular region behind the tower body, and a point cloud data sequence is formed; wherein the preset pitch angle interval and the scanning region range are dynamically adjusted according to the current working amplitude and height parameters of the tower crane.

3. The method of claim 1, wherein, The construction environment model is processed by using a work behavior recognition algorithm to identify corresponding potential obstacles and dangerous sources, including: The construction environment model is input into a preset deep learning neural network model, and the types, positions and contour sizes of static obstacles and dynamic targets are identified by a target detection module; wherein the static obstacles at least include buildings, scaffolds and piled materials, and the dynamic targets at least include construction vehicles, personnel and mobile machinery; A risk level division is performed on the blind area and the rear platform region by a semantic segmentation module, fixed obstacles causing line-of-sight obstruction and dynamic targets entering dangerous areas are marked, and an identification result containing target categories, risk levels and spatial positions is output.

4. The method of claim 1, wherein, The operation data of the tower crane are collected, including: Gear signals of an operation handle, rotation speeds and current data of motors of each mechanism, hook height, trolley position and jib rotation angle feedback by an encoder are obtained in real time by a tower crane controller, real-time data of a tower inclination sensor, a wind speed sensor and a weight sensor are collected, and a multi-dimensional operation data sequence containing operation instructions, mechanism states and environmental parameters is formed.

5. The method of claim 1, wherein, Based on the operation data, the motion trajectory of the hoisted object and the motion trajectory of the surrounding people and vehicles are predicted, and whether a dynamic target enters the blind area is judged, including: For the hoisted object, the spatial trajectory in the future 1-3 seconds is predicted by using a kinematic model according to the current hook position, trolley speed and jib rotation angular velocity; For the surrounding dynamic targets, their future motion direction and speed are predicted by using a Kalman filtering algorithm based on historical position data; a blind area spatial coordinate system is established, the intersection of the predicted trajectory and the blind area space is calculated in real time, and when the predicted position of a dynamic target enters the blind area boundary within a range of 0.5 meters, it is determined that the dynamic target enters the blind area.

6. The method of claim 1, wherein, If it is judged that a dynamic target enters the blind area, a warning information is generated, including: According to the type, speed and risk level of entering the blind area of the dynamic target, a warning information is generated: the warning information includes a first warning signal and a second warning signal; the first warning signal and the second warning signal both contain target type, entering position and remaining time to dangerous distance; wherein when a person is detected to enter a low-risk blind area at a speed lower than 1.5 m / s, a first warning signal is generated; when a vehicle is detected to enter a medium-risk blind area at a speed higher than 5 km / h or a person enters a high-risk blind area, a second warning signal is generated.

7. The method of claim 1, wherein, The visual auxiliary image and the warning information are fed back to the tower crane driving interface, including: An independent display area is divided on the display screen of the tower crane cab, the visual auxiliary image is displayed full screen on the left side area, and the current identified dangerous source list is displayed in real time on the right side area. When generating the early warning information, a floating warning window pops up at the top of the display screen, covering 20% of the current display content, and the early warning information automatically disappears after the dynamic target leaves the blind area.

8. A blind area and back platform perception compensation system for a tower crane cab, characterized in that, The method comprises the following steps: A data acquisition unit is configured to scan the elevation angles of the areas on both sides of the cab and the area behind the tower body by a rotating radar module integratedly installed below the trolley of the tower crane, and acquire the environmental data of the areas on both sides of the cab and the area behind the tower body. A model construction unit is configured to construct a construction environment model of the tower crane operation scene based on the acquired environmental data, including: time stamp alignment and spatial coordinate conversion of the continuous point cloud data sequence to convert the radar coordinate system data into global coordinate system data with the center of the tower crane rotation as the origin; denoising processing of the global coordinate system data by a voxel filtering algorithm to remove noise points and long-distance invalid data; generation of a static environment model containing three-dimensional terrain, fixed buildings, temporary facilities and vegetation based on multi-frame point cloud data splicing technology, and marking of the real-time position contour of the tower structure components to form a dynamically updated construction environment three-dimensional grid model; wherein the space-time data processing includes: time stamp alignment, adding an accurate time stamp with an accuracy of ±1 ms to the point cloud data collected by the radar, and triggering synchronization once every 1° change in the rotation angle; coordinate conversion, converting the radar coordinate system with the radar installation center as the origin into the global coordinate system, and the global coordinate system with the center of the tower crane rotation as the origin, the X-axis pointing to the initial direction of the jib, the Z-axis being perpendicular upward, and the conversion formula including a translation matrix corresponding to the offset of the radar relative to the center of the tower crane and a rotation matrix corresponding to the IMU attitude angle compensation; processing the construction environment model by using an operation behavior recognition algorithm to identify corresponding potential occlusions and hazards; generating a visual aid image containing blind area and rear platform area information in real time according to the identification result, including: generating a two-dimensional bird's eye view and a three-dimensional local magnification view by using computer graphics technology based on the construction environment model and the identification result; in the two-dimensional bird's eye view, marking the blind areas on both sides of the cab and the area behind the tower body with semi-transparent blocks, distinguishing the risk levels by different colors, and superimposing the real-time position and motion direction arrow of the dynamic target; in the three-dimensional local magnification view, highlighting the obstacle contour within a 10-meter range around the rear platform and the details of the dynamic target; the refresh frequency of the visual aid image is synchronized with the radar scanning frequency, and the display scale is adjusted according to the cab viewing angle; A data acquisition unit is configured to acquire operation data of the tower crane, predict the motion trajectories of the hoisted object and the surrounding people and vehicles based on the operation data, and determine whether a dynamic target enters the blind area; An early warning generation unit is configured to generate early warning information if it is determined that a dynamic target enters the blind area; and feed back the visual aid image and the early warning information to the tower crane driving interface.

Images