A monitoring method and controller applied to an aerial work platform

Abstract

This application relates to the field of engineering machinery safety technology, specifically to a monitoring method for aerial work platforms. The monitoring method includes: acquiring dynamic data and multiple two-dimensional images of the aerial work platform, whereby the dynamic data includes the boom angle, overall vehicle posture, and working height; performing a stitching and correction operation on the multiple two-dimensional images based on the boom angle and overall vehicle posture to obtain a target image; and generating a virtual boundary line based on the target image and working height. This virtual boundary line is used to identify the safe operating range of the aerial work platform. This application embodiment installs multiple imaging devices on the boom and vehicle body to acquire multiple two-dimensional images. Based on the boom angle and overall vehicle posture, a stitching and correction operation is performed on the multiple two-dimensional images to obtain the target image, eliminating dynamic blind spots. A virtual boundary line is generated using the target image and working height, and real-time monitoring is performed to determine whether obstacles enter the virtual boundary line, meeting the monitoring needs in complex operating environments.

Description

Technical Field

[0001] This application relates to the field of engineering machinery safety technology, and in particular to a monitoring method and controller for aerial work platforms. Background Technology

[0002] Aerial work platforms, as core equipment for high-altitude operations, are widely used in construction, power maintenance, and municipal operations. Their operational safety directly impacts the personal safety of operators and the property safety of the surrounding environment. However, existing environmental monitoring systems for aerial work platforms have numerous technical deficiencies, leading to significant safety hazards during operations, specifically as follows: First, blind spots are a significant problem. The boom structure of aerial work platforms is complex, and the operator's cab is positioned high up. Due to the boom's obstruction and limited operating angle, large blind spots are created around and below the vehicle. Operators cannot monitor the distribution of personnel and obstacles in these areas in real time, greatly increasing the risk of collisions, rollovers, and other safety accidents, seriously threatening operational safety. Second, the effectiveness of single sensors is limited. Currently, some aerial work platforms are only equipped with radar or a single camera as monitoring equipment. Radar cannot provide intuitive panoramic visual images, and single cameras have limited monitoring range. Both have weak ability to identify low obstacles, making it difficult to meet the monitoring needs in complex operating environments. Summary of the Invention

[0003] One objective of this application is to provide a monitoring method and controller for aerial work platforms, in order to improve the technical problem that related technologies cannot meet the monitoring needs in complex working environments.

[0004] The first aspect of this invention provides a monitoring method for aerial work platforms. The aerial work platform includes a boom, a vehicle body, and multiple imaging devices. The boom is connected to the vehicle body, and the multiple imaging devices are respectively installed on the boom and the vehicle body for acquiring multiple two-dimensional images of the aerial work platform and a preset area. The monitoring method includes: acquiring dynamic data of the aerial work platform and multiple two-dimensional images, wherein the dynamic data includes the boom angle, overall vehicle posture, and working height; performing a stitching and correction operation on the multiple two-dimensional images based on the boom angle and the overall vehicle posture to obtain a target image; and generating a virtual boundary line based on the target image and the working height, wherein the virtual boundary line is used to identify the safe working range of the aerial work platform.

[0005] Optionally, in a first implementation of the first aspect of the present invention, the step of performing a stitching correction operation on multiple two-dimensional images based on the boom angle and the vehicle posture to obtain a target image includes: inputting the multiple two-dimensional images into a preset deep learning model to extract target features; encoding the target features to obtain feature encoding information, the feature encoding information being used to provide a basis for feature matching between the multiple two-dimensional images; capturing global context information in the multiple two-dimensional images through the convolutional and pooling layers of the deep learning model to obtain global features; and combining the boom angle and the vehicle posture to perform a stitching correction operation on the feature encoding information and the global features to obtain the target image.

[0006] Optionally, in a second implementation of the first aspect of the present invention, the step of performing a stitching correction operation on the feature encoding information and the global features to obtain a target image includes: fusing the feature encoding information and the global features to obtain fused features; performing feature matching on multiple paths of the two-dimensional images based on the fused features to obtain matching features; performing alignment and projection transformation operations between multiple paths of the two-dimensional images based on the matching features to obtain a corrected image; and performing a stitching correction operation on the corrected image to obtain the target image.

[0007] Optionally, in a third implementation of the first aspect of the present invention, the deep learning model includes a bottleneck layer, and the step of fusing the feature encoding information and the global features to obtain fused features includes: using the bottleneck layer to perform global information integration processing on the global features to obtain global abstract features; and fusing the feature encoding information and the global abstract features to obtain fused features.

[0008] Optionally, in a fourth implementation of the first aspect of the present invention, the deep learning model includes a decoder and an encoding layer. The step of performing alignment and projection transformation operations between multiple paths of the two-dimensional images based on the matching features to obtain a corrected image includes: performing an upsampling operation on the matching features through the decoder to obtain sampled features; performing a feature reconstruction operation on the sampled features to obtain a high-resolution image structure, wherein the high-resolution image structure is a spatial structure containing the edges, contours, and textures of the two-dimensional images; extracting low-level image features based on the encoding layer; fusing the high-resolution image structure and the low-level image features through skip connections in the encoding layer to obtain detail features; and performing alignment and projection transformation operations between multiple paths of the two-dimensional images based on the detail features to obtain a corrected image.

[0009] Optionally, in a fifth implementation of the first aspect of the present invention, performing a stitching correction operation on the corrected image to obtain a target image includes: establishing a feature mapping relationship between multiple two-dimensional images and the corrected image; and performing a stitching correction operation on the corrected image based on the feature mapping relationship to obtain the target image.

[0010] Optionally, in a sixth implementation of the first aspect of the present invention, the aerial work platform includes a radar device for collecting target data of moving obstacles. After generating a virtual boundary line based on the target image and the working height, the system further includes: generating alarm information based on a preset trajectory prediction model in response to a moving obstacle intruding into a preset warning zone within a preset time, wherein the warning zone is a safety warning area defined by the aerial work platform as the center and according to a preset safety distance; and generating predicted trajectory information based on the target data in response to a moving obstacle not intruding into the preset warning zone within a preset time.

[0011] Optionally, in the seventh implementation of the first aspect of the present invention, the warning interval includes a primary warning zone and a secondary warning zone. The step of generating alarm information based on a preset trajectory prediction model in response to a moving obstacle intruding into a preset warning interval within a preset time includes: marking the position of the moving obstacle in response to the moving obstacle intruding into the secondary warning zone; and generating alarm information in response to the moving obstacle intruding into the primary warning zone.

[0012] Optionally, in an eighth implementation of the first aspect of the present invention, the step of predicting the trajectory of the moving obstacle by combining the target data collected by the radar device to generate predicted trajectory information includes: preprocessing the target data to obtain effective data; and inputting the effective data into the trajectory prediction model to generate predicted trajectory information.

[0013] A second aspect of the present invention provides a controller including a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, wherein, when the processor executes the one or more computer programs, the controller enables the monitoring method described above.

[0014] The embodiments of this application can achieve the following technical effects: Multiple shooting devices are installed on the boom and the vehicle body. The shooting devices collect the working status of the aerial work platform vehicle. According to the boom angle and the overall vehicle posture, the multi-channel two-dimensional images are stitched and corrected to obtain the target image, so as to eliminate the dynamic blind spots caused by boom movement, meet the monitoring needs in complex working environments, and generate virtual boundary lines through the target image and working height, and monitor in real time whether obstacles enter the virtual boundary lines, thereby improving the safety of the aerial work platform vehicle during operation. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a structural schematic diagram of an aerial work platform provided in an embodiment of this application; Figure 2 A schematic diagram of a module for an aerial work platform provided in an embodiment of this application; Figure 3 This is a communication diagram of the imaging device provided in an embodiment of this application; Figure 4 A flowchart illustrating a monitoring method for aerial work platforms provided in this application embodiment; Figure 5 A flowchart illustrating a monitoring method for aerial work platforms, provided as another embodiment of this application; Figure 6 A schematic diagram of the warning interval provided in the embodiments of this application; Figure 7 A schematic diagram of the monitoring device provided in the embodiments of this application; Figure 8 This is a schematic diagram of the controller provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0018] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. Moreover, the terms "first," "second," and "third" used in this application do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0019] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0020] In related technologies, aerial work platforms have complex boom structures, high-positioned operator cabins, and large blind spots around and below the vehicle, making them prone to collisions and rollovers. Traditional aerial work platforms rely on rearview mirrors and manual ground control, which is inefficient, highly susceptible to lighting and weather conditions, and prone to communication errors. Furthermore, some models use radar or a single camera, failing to provide panoramic visual images and exhibiting weak recognition of low-lying obstacles such as pipelines and potholes. When the vehicle moves or the boom extends or retracts, traditional systems struggle to update environmental information in real time, resulting in delayed or distorted stitched images.

[0021] Therefore, the embodiments of this application use software to stitch and correct the original two-dimensional image, without relying on a specific hardware architecture, to eliminate large-area blind spots and improve the safety of aerial work vehicle operations.

[0022] Please see Figure 1 The aerial work platform 100 includes a boom 11, a body 12, multiple shooting devices 13 and a working platform 14. The boom 11 is connected to the body 12. The multiple shooting devices 13 are respectively installed on the boom 11 and the body 12, and are used to collect multiple two-dimensional images of the aerial work platform 100 and a preset area.

[0023] The boom 11 is used to lift, extend, contract, and rotate the work platform 14, delivering personnel or equipment on the work platform 14 to designated high-altitude work positions. The vehicle body 12 is used to facilitate the relocation, site movement, and parking positioning of the aerial work vehicle 100. Multiple camera devices 13 provide 360° panoramic view, eliminating blind spots under the vehicle body 12, boom 11, and the work area, achieving a blind spot coverage rate of >98% and reducing the collision accident rate by 70%+. The camera devices 13 can be wide-angle fisheye cameras, with a horizontal viewing angle of ≥190° for each individual wide-angle fisheye camera. The camera devices 13 assist the driver in the cab of the aerial work vehicle 100 in safe operation, and are used to monitor the boom 11 posture, the work platform 14 status, surrounding obstacles, and the environment in real time, providing early warnings of collision risks. In this embodiment, the aerial work vehicle 100 can be a 16-meter articulated boom aerial work vehicle 100.

[0024] Please see Figure 2 The aerial work platform 100 also includes an inertial measurement unit 15, a boom attitude sensor 16, a controller 17, a turntable display screen 18, a radar device 19, and a buzzer 110.

[0025] An inertial measurement unit 15 is mounted on the vehicle body 12 to collect the vehicle body 12's attitude, acceleration, and angular velocity information, providing data support for overall vehicle attitude judgment and anti-rollover warning. A boom attitude sensor 16 is used to detect the boom 11's telescopic length, luffing angle, slewing angle, and other attitude parameters in real time.

[0026] Please see Figure 3 The controller 17 and turntable display screen 18 are connected to the shooting device 13 via CAN communication, with a display latency of <80ms, meeting the ISO 13406-2 standard. The AR interface of the turntable display screen 18 intuitively displays the distance and orientation of obstacles, reducing the cognitive load on the operator. The turntable display screen 18 is located in the cab and is a 10-inch anti-glare touchscreen that supports split-screen mode. In split-screen mode, the operator can view the panoramic view on the left side and the single-channel high-definition view on the right side of the aerial work platform 100. The radar device 19 and buzzer 110 are connected to the controller 17. The buzzer 110 is used to emit an alarm sound to remind the operator to restrict the operation of the aerial work platform 100. The radar device 19 is used to collect target data of moving obstacles. The controller 17 is used to receive multi-channel two-dimensional images and status information collected by each shooting device 13 via CAN. The communication system configures the parameters of the imaging device 13 and processes the data of multiple two-dimensional images before sending them to the turntable display screen 18 for real-time display. At the same time, the controller 17 can identify blind spots and obstacles in operation based on the processed image information, and combine the operating status of the boom 11 and the vehicle body 12 to achieve safety warning and auxiliary control.

[0027] In some embodiments, shooting devices 13 are provided on the top, rear and side of the vehicle body 12. The boom 11 is a multi-section foldable boom 11. One end of the boom 11 is connected to the vehicle body 12, and the other end of the boom 11 is connected to the working platform 14. The working height of the working platform 14 is achieved by adjusting the folding angle of the boom 11.

[0028] In some embodiments, the boom 11 includes a first boom section, a second boom section, a third boom section, a fourth boom section, and a connecting part connected in sequence. One end of the first boom section is connected to the vehicle body 12, and one end of the connecting part is connected to the work platform 14. A camera device 13 is provided on the first boom section, the second boom section, the third boom section, and the fourth boom section. The camera device 13 can cover a 5-meter radius around the vehicle. Camera devices 13 are provided at the bottom and rear of the work platform 14, and waterproof covers are installed on the camera devices 13 to achieve IP67 protection. Multiple camera devices 13 move with the first boom section, the second boom section, the third boom section, and the fourth boom section to eliminate the dynamic blind spots generated by the movement of the boom 11 in real time. A camera device 13 is provided on the front, rear, left, and right sides of the vehicle body 12. The number of camera devices 13 provided on the aerial work vehicle 100 is not limited to the above-mentioned number and can be set according to the actual situation.

[0029] In addition, the shooting device 13 can also be installed at the connection points of the first arm, the second arm, the third arm, the fourth arm and the connecting part, and the shooting device 13 has a vibration-resistant design.

[0030] This application provides a monitoring method for an aerial work platform 100. The monitoring method provided can be implemented by various types of electronic devices with computing power, such as the controller 17 in the above embodiment. This controller 17 can be the controller 17 in the aerial work platform 100, the controller 17 in the camera device 13, or the controller 17 in the work platform 14, etc. Please refer to... Figure 4 The monitoring methods include, but are not limited to, the following steps: S10. Acquire dynamic data and multiple two-dimensional images of the aerial work platform 100. The dynamic data includes the boom angle, overall vehicle posture, and working height of the aerial work platform 100.

[0031] The boom angle refers to the angle between the boom 11 and the horizontal plane, directly detected by the boom attitude sensor 16. It reflects the tilt degree of the boom 11 when raised and lowered, and is used to determine the working posture and safety range of the boom 11. The overall vehicle posture refers to the tilt state of the vehicle body 12, including the pitch angle and roll angle of the vehicle body 12. It is detected by the inertial measurement unit 15 and is used to determine whether the aerial work platform 100 is level and whether there is a risk of rollover. The working height refers to the vertical height of the working platform 14 relative to the ground. It is calculated by the controller 17 based on parameters such as the boom 11 length and boom angle, and is used for height limiting, over-limit prevention, and safety warning.

[0032] Multiple two-dimensional images are obtained by capturing video images through multiple shooting devices 13, which can fully cover the area around the body 12 of the aerial work platform 100, the area under the boom 11, and the blind spot, eliminating blind spots; providing a complete image data source for subsequent panoramic stitching, obstacle recognition and safety warning, and improving the visibility and operational safety of the aerial work process.

[0033] S20. Based on the boom angle and vehicle posture, perform stitching and correction operations on multiple two-dimensional images to obtain the target image.

[0034] The adaptive stitching algorithm is used to stitch together multiple two-dimensional images. The adaptive stitching algorithm can dynamically adjust the image stitching parameters, distortion correction coefficient and fusion weight according to the real-time changes of boom angle and vehicle posture. It can automatically adapt to the shooting angle shift under different working postures, eliminate image misalignment and stitching gaps caused by boom 11 movement and vehicle body 12 tilt, and generate stable, continuous and distortion-free panoramic surround view images to ensure the accuracy and real-time performance of blind spot display. A deep learning model is used to eliminate lens distortion of shooting device 13 to obtain distortion-free target images.

[0035] S30. Generate a virtual boundary line based on the target image and the working height. The virtual boundary line is used to identify the safe working range of the aerial work platform 100.

[0036] The virtual boundary line is a visual warning line calculated by the controller 17 based on environmental information and obstacle positions in the target image, combined with the current working height. It is used to dynamically mark the boundary of the area where the aerial work vehicle 100 is allowed to work safely in the target image, and to distinguish between the safe working area and the dangerous restricted area.

[0037] Virtual boundary lines visually represent the safe working area in the image, allowing operators to quickly determine the working boundaries and avoid working beyond the safe posture. The virtual boundary lines are dynamically generated based on the real-time working height and can adaptively adjust with changes in the boom's posture, improving the accuracy of safety warnings at different working heights.

[0038] In this embodiment, multiple shooting devices 13 are installed on the boom 11 and the vehicle body 12. The shooting devices 13 collect the working status of the aerial work platform 100. Based on the boom angle and the overall vehicle posture, the multi-channel two-dimensional images are stitched and corrected to obtain the target image, thereby eliminating the dynamic blind spots caused by the movement of the boom 11 and meeting the monitoring needs in complex working environments. A virtual boundary line is generated by the target image and the working height, and the obstacle is monitored in real time to see whether it enters the virtual boundary line, thereby improving the safety of the aerial work platform 100 during operation.

[0039] In some embodiments, performing a stitching correction operation on multiple two-dimensional images based on the boom angle and vehicle posture to obtain a target image includes: inputting the multiple two-dimensional images into a preset deep learning model to extract target features; encoding the target features to obtain feature encoding information, which is used to provide a basis for feature matching between the multiple two-dimensional images; capturing global contextual information in the multiple two-dimensional images through the convolutional and pooling layers of the deep learning model to obtain global features; and combining the boom angle and vehicle posture to perform a stitching correction operation on the feature encoding information and global features to obtain the target image.

[0040] The deep learning model, an intelligent algorithm integrated into the controller 17, is trained based on deep learning architectures such as convolutional neural networks. It intelligently processes two-dimensional images captured by multiple imaging devices 13, primarily adapting to the dynamic operation scenarios of the aerial work platform 100. It can combine real-time data such as boom angle, vehicle posture, and working height to complete core operations such as feature extraction, distortion correction, viewpoint optimization, and seam smoothing of two-dimensional images. This provides precise image processing support for the adaptive stitching algorithm, facilitating the high-quality generation of panoramic target images. The entire process of obtaining the target image is a data-driven image transformation process, rather than traditional geometric model correction. Feature encoding information is a high-dimensional feature vector obtained by the controller 17 after feature extraction and encoding of various two-dimensional images through the deep learning model. It is used to represent key visual features in the image, such as edges, corners, textures, contours, and overlapping region features. The convolutional layer is the core feature extraction layer of the deep learning model. Through convolution operations with multiple convolutional kernels and the two-dimensional image, it extracts local features such as edges, corners, textures, and contours from the two-dimensional image and generates corresponding feature maps, providing basic feature information for subsequent image matching and stitching. The pooling layer is located after the convolutional layer and is used to downsample the feature map output by the convolutional layer. While retaining key feature information, it compresses the amount of feature data, reduces the computational complexity of the model, improves the image processing speed, enhances the robustness of features, and reduces feature interference caused by changes in lighting and boom 11 shaking.

[0041] Global features refer to the overall semantic features extracted and fused from the entire two-dimensional image by a deep learning model after multiple convolution and pooling operations. They are used to characterize the overall layout, scene structure, regional correlation, and overall texture distribution of the image, which is different from local features that only reflect local details.

[0042] This application improves the accuracy and adaptability of image stitching, effectively solving the problems of multi-channel two-dimensional image viewpoint offset and misalignment caused by boom 11 swing and vehicle body 12 tilt during aerial work platform 100 operation. By combining real-time data of boom angle and vehicle posture, targeted stitching correction is performed on feature encoding information and global features to ensure seamless and distortion-free target image stitching, restoring the real spatial relationship of the work scene. Deep learning models are used to extract target features and generate feature encoding information, providing accurate basis for feature matching of multi-channel images. Simultaneously, convolutional and pooling layers capture global contextual information, taking into account both local image details and the overall scene correlation, avoiding stitching mismatches caused by relying solely on local features, and improving the stability and robustness of stitching in complex working environments. Complex working environments include lighting changes, obstacle occlusion, and boom 11 movement interference.

[0043] The target image obtained after stitching and correction can completely and clearly present the panoramic scene around the aerial work vehicle 100 and the work area, providing a high-quality image foundation for the subsequent generation of virtual boundary lines, ensuring the accuracy of the safe work area marking, helping operators to intuitively grasp the work environment, and reducing blind spots.

[0044] The entire splicing and correction process is combined with the real-time boom angle and overall vehicle posture dynamic adjustment of the aerial work platform 100 to achieve adaptive linkage between image processing and operation status. No manual intervention is required, which reduces the operator's operating burden. At the same time, it improves the real-time performance of image processing, adapts to the dynamic operation needs of the aerial work platform 100, and indirectly improves operation efficiency and safety.

[0045] In some embodiments, performing a stitching correction operation on feature encoding information and global features to obtain a target image includes: fusing feature encoding information and global features to obtain fused features; performing feature matching on multiple two-dimensional images based on the fused features to obtain matching features; performing alignment and projection transformation operations between multiple two-dimensional images based on the matching features to obtain a corrected image; and performing a stitching correction operation on the corrected image to obtain the target image.

[0046] Fusion features refer to the comprehensive feature data obtained by fusing feature encoding information with global features. This integrates the local detail matching characteristics of feature encoding information with the overall scene correlation of global features. It retains the local feature matching basis of multiple 2D images while also considering the overall image layout and scene context information, providing more comprehensive and accurate feature support for subsequent feature matching. Matching features refer to the set of matching feature points and feature regions between multiple 2D images obtained after feature comparison and correspondence recognition based on fusion features. This is used to identify the corresponding overlapping areas and common features in 2D images acquired by different shooting devices 13, and is the core basis for image alignment and projection transformation. Alignment operation is based on matching features, adjusting the position and angle of multiple 2D images to ensure precise correspondence and no misalignment of overlapping areas. Projection transformation operation combines the perspective shift caused by boom angle and vehicle posture to perform perspective transformation and perspective correction on the aligned image, converting 2D images from different perspectives to a unified coordinate system and eliminating image distortion and misalignment caused by differences in shooting perspective. The stitching correction operation refers to the process of blending seams, equalizing colors, and smoothing edges of the corrected image to eliminate gaps, color differences, and edge misalignments at the stitching points of multiple corrected images, ultimately generating a complete, continuous, distortion-free, and color-consistent target image.

[0047] The real-time image compensation process during the movement of boom 11 in the projection transformation operation is as follows: When the aerial work platform 100 is powered on / operating, the image compensation process continuously loops and runs in real time. It reads the boom attitude sensor 16 in real time to obtain the current boom angle, including the extension, rotation, and luffing angles of the boom 11. It also reads the inertial measurement unit 15 to obtain the vehicle's attitude, including the tilt, vibration, and acceleration data of the vehicle body 12. Based on the boom angle and vehicle body attitude, it automatically updates the perspective transformation matrix for image stitching in real time. Based on the updated matrix, it performs adaptive stitching correction on images acquired by multiple cameras, ultimately outputting a stable, distortion-free, and misaligned target image. This embodiment of the application can perform real-time dynamic adaptive stitching, automatically correcting for boom 11 movement and vehicle body 12 tilt, ensuring a consistently stable and clear target image, eliminating perspective shifts and image misalignment, improving the accuracy of blind spot display, and operating fully automatically without manual intervention, thus enhancing operational safety.

[0048] By fusing feature encoding information with global features, matching biases caused by relying on a single feature are avoided, effectively solving the problem of feature mismatch in complex operating scenarios. Alignment operations are performed based on matching features to ensure accurate correspondence of overlapping areas in multiple images. Projection transformation is performed in conjunction with boom angle and vehicle posture to specifically correct shooting perspective deviations caused by boom 11 movement and vehicle body 12 tilt, unifying images from different perspectives to the same coordinate system, significantly reducing the probability of image misalignment and distortion, and obtaining high-quality corrected images. By performing splicing correction operations such as seam fusion and color equalization on the corrected images, gaps, color differences, and abrupt edges at splicing points are eliminated. The generated target image is complete, continuous, distortion-free, and color-uniform, realistically restoring the panoramic scene around the aerial work platform 100 and the operating area, providing high-quality image support for subsequent virtual boundary line generation and safety warnings.

[0049] In some embodiments, the deep learning model includes a bottleneck layer, which performs fusion processing on feature encoding information and global features to obtain fused features. This includes: using the bottleneck layer to perform global information integration processing on global features to obtain global abstract features, and fusing feature encoding information and global abstract features to obtain fused features.

[0050] The bottleneck layer is the core transition layer in a deep learning model, located between feature extraction and feature fusion. Its core function is to reduce the dimensionality, refine, and integrate global features. While retaining the core semantic information of global features, it compresses the dimension of feature data, removes redundant information, and strengthens the representational ability of global features. This provides an efficient and accurate global feature foundation for subsequent feature fusion, adapting to the real-time requirements of aerial work platform 100 image processing. The core semantic information of global features includes, for example, the overall layout of the work scene and the overall outline of obstacles.

[0051] By integrating and refining global information through a bottleneck layer, redundant interference information in the original global features is eliminated, resulting in highly condensed global abstract features. This preserves core information such as the overall scene layout and regional correlation of the image while reducing the amount of feature data and avoiding interference from redundant interference information in subsequent fusion and matching operations. Redundant interference information includes invalid feature information caused by changes in lighting and boom 11 shaking. The bottleneck layer enables the deep learning model to adapt to the dynamic operating scenarios of the aerial work platform 100. It can flexibly adjust the global information integration strategy according to changes in boom angle and overall vehicle posture, generating global abstract features adapted to the current operating posture. These features are then fused with feature encoding information, ensuring that the fused features always adapt to dynamically changing shooting angles and operating environments, thus improving the robustness and practicality of the deep learning model.

[0052] In some embodiments, the deep learning model includes a decoder and an encoding layer. Performing alignment and projection transformation operations between multiple two-dimensional images based on matching features to obtain a corrected image includes: performing an upsampling operation on the matching features through the decoder to obtain sampled features; performing a feature reconstruction operation on the sampled features to obtain a high-resolution image structure, where the high-resolution image structure is a spatial structure containing the edges, contours, and textures of the two-dimensional image; extracting low-level image features based on the encoding layer; fusing the high-resolution image structure and the low-level image features through skip connections in the encoding layer to obtain detail features; and performing alignment and projection transformation operations between multiple two-dimensional images based on the detail features to obtain the corrected image.

[0053] The decoder is the core network structure in a deep learning model responsible for feature restoration and spatial reconstruction, and it is usually constructed symmetrically with the encoder layer. The main function of the decoder is to upsample matching features that have lost spatial resolution after downsampling, gradually restoring the detailed texture and spatial size of the features, and restoring the abstract feature vectors into high-resolution feature maps with image spatial structure, providing basic data support for subsequent image alignment and projection transformation.

[0054] Upsampling is a process that enlarges the size of low-dimensional matching features received by the decoder through interpolation, deconvolution, or other methods, thereby increasing the resolution of the feature map. Upsampling can fill in spatial details lost during feature extraction, restore edge and contour information of the two-dimensional image, and enable the feature map to more accurately correspond to the pixel position relationships of the two-dimensional image.

[0055] Feature reconstruction refers to the process of learning and fitting the original spatial structure of an image based on sampled features through multi-layer convolution or deconvolution operations. The purpose of feature reconstruction is to restore the high-resolution image structure containing high-frequency information such as image edges, contours, and textures from the sampled features, so that the feature map can accurately represent the spatial geometric relationships of the two-dimensional image. Alignment and projection transformation based on the restored high-resolution image structure can ensure the accurate spatial correspondence of two-dimensional images captured by different cameras, effectively eliminating image misalignment.

[0056] The encoding layer is the front-end feature extraction network of the deep learning model. By stacking convolutional and pooling layers, it performs multi-layer feature extraction on the input two-dimensional image, gradually compressing the image dimension and capturing low-level image features. Low-level image features mainly refer to low-level, high-resolution details such as edges, corners, and local textures in the image. This enables the alignment and projection transformation process to withstand complex conditions such as boom movement, lighting changes, and noise interference, significantly improving the robustness and stability of image correction.

[0057] Skip connections are a network structure connection method that directly copies or transforms high-resolution features from a low-level layer in the coding layer and fuses them with features from the corresponding layer in the decoder. The core function of skip connections is to compensate for spatial details lost during the encoding process, fusing high-resolution image structures with low-level image features, preserving minute details of the image, and ensuring the accuracy of alignment and projection transformation.

[0058] When the shooting angle shifts drastically due to changes in the working posture, the decoder can quickly reconstruct the image structure after the angle change, while the coding layer and skip connections ensure that detailed features are not lost, thereby achieving robust adaptation to dynamic working environments and ensuring the continuous and reliable operation of the 100° panoramic surround view system for aerial work platforms.

[0059] In some embodiments, performing a stitching correction operation on the corrected image to obtain the target image includes: establishing a feature mapping relationship between the multi-channel two-dimensional image and the corrected image, and performing a stitching correction operation on the corrected image based on the feature mapping relationship to obtain the target image.

[0060] Feature mapping relationship refers to the one-to-one correspondence between multiple two-dimensional images and corresponding correction images established based on matching features and detail features. Essentially, it represents the spatial location and feature attribute correspondence between feature points and feature regions in two-dimensional images and corresponding feature points and feature regions in correction images. It clarifies the specific mapping position of each feature in the correction image after alignment and projection transformation of the two-dimensional image, providing accurate feature correspondence basis for subsequent stitching correction.

[0061] The feature mapping relationship clarifies the feature correspondence between the two-dimensional image and the correction image, enabling the stitching correction operation to accurately locate the overlapping area and corresponding feature points of multiple correction images. This avoids problems such as stitching misalignment and obvious gaps caused by the disconnect between the features of the correction image and the original image, ensuring that the target image is stitched seamlessly and the connection is natural, thus restoring the real spatial relationship of the work scene.

[0062] Please see Figure 5 After generating the virtual boundary line based on the target image and the job height, the following steps are also included: S40. Based on the preset trajectory prediction model, in response to a moving obstacle intruding into the preset warning zone 41 within a preset time, an alarm message is generated. The warning zone 41 is a safety warning area centered on the aerial work platform 100 and defined according to a preset safety distance.

[0063] Radar device 19 is an environmental perception sensor deployed on the aerial work platform 100. It emits electromagnetic waves and receives echoes to accurately collect target data of moving obstacles around the vehicle body 12. The target data includes position data, speed data, and distance data. Moving obstacles refer to external entities within the working range of the aerial work platform 100 whose position, attitude, or speed changes over time, including but not limited to pedestrians, other vehicles, and mobile machinery. The trajectory prediction model is an algorithm model running in the controller 17. Based on the obstacle position and speed data collected in real time by radar device 19, combined with historical motion trajectories, the trajectory prediction model includes a kinematic model and a machine learning model. Through the kinematic model and machine learning model, the movement position, direction, and speed changes of the moving obstacle over a future period of time are mathematically modeled and trend predicted. The kinematic model is a uniform velocity / uniform acceleration model, and the machine learning model can be a Kalman filter model. The uniform velocity model assumes that the moving obstacle maintains a constant speed within a preset time, and the speed magnitude and direction of movement do not change. The uniform velocity model can quickly predict the short-term motion trajectory of the obstacle and is suitable for scenarios where the obstacle's motion state is stable. The uniform acceleration model is a model that assumes that a moving obstacle maintains a constant acceleration for a preset time and that the speed changes uniformly over time. It can accurately capture the dynamic motion state of the obstacle, such as acceleration and deceleration, and is suitable for scenarios where the motion state of the obstacle changes (such as pedestrians accelerating towards the obstacle or vehicles decelerating).

[0064] The Kalman filter model is a recursive filtering algorithm model based on linear system state estimation. It is integrated into the controller 17. Its core function is to perform noise suppression, state estimation and prediction on the effective data collected and preprocessed by the millimeter-wave radar device 19. Through the iterative process of prediction-update, it corrects the measurement data deviation and outputs smooth, continuous and accurate obstacle motion state data, providing stable support for obstacle tracking and trajectory prediction.

[0065] Warning zone 41 refers to a three-dimensional spatial safety warning area defined with the aerial work platform 100 as the geometric center, based on the vehicle size, working height, boom 11 turning radius, and preset safety distance. In this embodiment, warning zone 41 can be a safety warning area with the aerial work platform 100 as the geometric center and a preset safety distance of 3 meters.

[0066] The alarm information is a warning signal actively generated by the controller 17 when a moving obstacle enters the preset warning zone 41 within a preset time. The forms of alarm information include, but are not limited to: visual alarms on the turntable display screen 18, audible alarms on the in-vehicle buzzer 110, braking commands to control the movement of the boom 11, etc., which are used to immediately remind the operator and intervene in the operation of the vehicle.

[0067] S50. In response to the fact that the moving obstacle does not enter the preset warning interval 41 within a preset time, the trajectory of the moving obstacle is predicted based on the target data, and the predicted trajectory information is generated.

[0068] Predicted trajectory information is data generated by the trajectory prediction model based on the target data, describing the movement path of a moving obstacle within a preset time period in the future.

[0069] Compared to related technologies that rely solely on visual images or static sensors, this application embodiment uses a trajectory prediction model to predict the future trajectory of moving obstacles based on location data collected by radar. This allows the aerial work platform 100 to predict the future path of a moving obstacle before it actually contacts the work area. If a moving obstacle is about to enter the warning zone 41, the controller 17 can generate an alarm message or intervention action in advance, fundamentally reducing the probability of collision accidents and achieving true active safety. Based on the predicted trajectory information, the controller 17 can not only issue warnings but also adjust the posture or travel path of the boom 11 of the aerial work platform 100 in reverse. For example, if the controller 17 predicts that a pedestrian will pass from the left, it can automatically control the boom 11 to pause and turn left or avoid the pedestrian in advance. This upgrades the aerial work platform 100 from a passive monitored object to an active intelligent planning subject, significantly improving the vehicle's automated control capabilities and operational safety.

[0070] In some embodiments, please refer to Figure 6 The warning interval 41 includes a primary warning zone 411 and a secondary warning zone 412. Based on a preset trajectory prediction model, in response to a moving obstacle intruding into the preset warning interval 41 within a preset time, alarm information is generated, including: in response to a moving obstacle intruding into the secondary warning zone 412, the location of the moving obstacle is marked; in response to a moving obstacle intruding into the primary warning zone 411, alarm information is generated.

[0071] The Level 1 Warning Zone 411 is a core safety warning area centered on the aerial work platform 100 and defined at a closer, preset safety distance. Located inside the Level 2 Warning Zone 412, it is a high-risk, dangerous area. In this embodiment, the Level 1 Warning Zone 411 is defined as a 1-meter radius centered on the aerial work platform 100. If a moving obstacle continues to approach within 1 meter, a Level 3 audible and visual alarm is triggered, and the operating speed of the aerial work platform 100 is limited. A red flashing frame is superimposed on the screen of the turntable display 18, and the buzzer 110 is simultaneously triggered. When a moving obstacle enters the Level 1 Warning Zone 411, it indicates a direct and imminent threat to vehicle operation, boom 11 operation, and overall vehicle safety. The secondary warning zone 412 is an outer warning area defined by a relatively far preset safety distance centered on the aerial work platform 100, surrounding the primary warning zone 411. It is a zone of concern. In this embodiment, the secondary warning zone 412 is defined as a 2-meter radius centered on the aerial work platform 100. The ranges of the primary and secondary warning zones 411 are not limited to the above data and can be modified according to the application scenario. When a moving obstacle enters the secondary warning zone 412, it only indicates that the obstacle is approaching the safe working area and poses a potential risk, but does not yet constitute an imminent danger. Marking the position of the moving obstacle means that when the controller 17 detects a moving obstacle intruding into the secondary warning zone 412, it records, marks, and synchronizes the real-time coordinates, direction of movement, distance, and other positional information of the moving obstacle to the turntable display screen 18 for continuous tracking and status alerts of potential dangerous targets. Alarm information refers to the audible and visual warnings, screen highlighting prompts, or safety linkage control signals generated by the controller 17 when an obstacle intrudes into the first-level warning zone 411, which are used to remind operators to take safety measures such as avoidance or shutdown immediately.

[0072] A tiered early warning system, employing a primary warning zone (411) and a secondary warning zone (412), is implemented to achieve graded safety control of moving obstacles, avoiding frequent false alarms or delayed alarms caused by a single warning zone. When an obstacle intrudes into the secondary warning zone (412), only its location is marked and continuously tracked without triggering an alarm. This allows for early detection of potential risks without interfering with normal operations, improving the system's user experience. When an obstacle intrudes into the primary warning zone (411), an alarm is immediately generated, providing an emergency alarm for high-risk situations at close range. This ensures a rapid response to imminent dangers and effectively prevents accidents such as collisions with the boom 11 or personnel approaching the obstacle.

[0073] In some embodiments, predicting the trajectory of a moving obstacle by combining the target data collected by the radar device 19 and generating predicted trajectory information includes: preprocessing the target data to obtain valid data, inputting the valid data into the trajectory prediction model, and generating predicted trajectory information.

[0074] Preprocessing refers to the screening, purification, calibration, and standardization of target data. Core operations include removing outlier data, filtering noise interference, correcting data deviations, and standardizing data formats. The aim is to remove invalid, redundant, and interfering data, improve data quality, and provide reliable data support for the trajectory prediction model. Valid data refers to the true, accurate, interference-free target data that meets the requirements of trajectory prediction after data preprocessing. It accurately reflects the actual movement state and location information of moving obstacles, eliminating abnormal data caused by environmental interference (such as rain, fog, and electromagnetic interference) and equipment errors during radar device 19 acquisition. This data is the direct input data for the trajectory prediction model.

[0075] Preprocessing removes outliers, noise, and redundant information from the target data, preventing prediction bias caused by invalid data input into the trajectory prediction model. This ensures that the effective data input into the model accurately reflects the motion state of moving obstacles, significantly improving the accuracy of predicted trajectory information and providing a reliable basis for safety warnings. High-altitude work scenarios are prone to interference factors such as rain, fog, electromagnetic interference, and ground reflection, which can cause deviations in radar-acquired target data. Preprocessing can specifically filter out such interference, enabling the trajectory prediction model to operate stably under complex conditions, avoiding prediction failures or misjudgments due to data interference, and improving robustness.

[0076] Based on the real-time perception of the millimeter-wave radar device 19 and in response to the fact that a moving obstacle has not entered a preset warning zone 41 within a preset time, effective data is clustered and tracked. Point cloud clustering is performed on the effective data, aggregating multiple radar points belonging to the same moving obstacle into a single target, distinguishing different obstacles, and achieving multi-target separation. Moving obstacle tracking, based on the clustering results, performs correlation matching on data of the same moving obstacle in consecutive frames to achieve continuous tracking of its position and motion state.

[0077] Furthermore, image enhancement algorithms can be used to support rain and fog modes, and HDR fusion can be used to suppress strong light. The image enhancement algorithm refers to the image processing algorithm integrated in the controller 17, which is used to optimize the two-dimensional images captured by the shooting device 13. Its core function is to improve the image's clarity, contrast, and detail, and to correct problems such as blurring, distortion, and loss of detail in two-dimensional images caused by environmental interference, thus providing a high-quality image foundation for subsequent image stitching, feature extraction, and obstacle recognition.

[0078] HDR (High Dynamic Range) fusion refers to the process of fusing multiple frames of images with different exposure intensities captured by the shooting device 13 through image enhancement algorithms. It integrates the highlight and shadow details of the multiple frames to generate a clear image with high dynamic range. The core function is to achieve strong light suppression and solve the problems of overexposure of highlights and loss of shadow details in images under strong light conditions.

[0079] It should be noted that in the above embodiments, there is no necessarily a certain order between the steps. Those skilled in the art can understand from the description of the embodiments of this application that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in turn, etc.

[0080] As another aspect of this application, this application provides a monitoring device applied to an aerial work platform 100. The monitoring device can be a software module, which includes several instructions stored in a memory. A processor can access the memory, invoke the instructions, and execute them to complete the monitoring methods described in the various embodiments above.

[0081] In some implementations, the monitoring device can also be constructed from hardware components. For example, the monitoring device can be constructed from one or more chips, which can work in coordination to complete the monitoring methods described in the various implementations above. As another example, the monitoring device can also be constructed from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0082] Please see Figure 7 The monitoring device 600 includes a data acquisition unit 61, a panoramic stitching module 62, and a boundary line generation unit 63.

[0083] The acquisition unit 61 is used to acquire dynamic data and multiple two-dimensional images of the aerial work platform 100; the panoramic stitching module 62 is used to perform stitching correction operations on multiple two-dimensional images based on the boom angle and the overall vehicle posture to obtain the target image; the boundary line generation unit 63 is used to generate virtual boundary lines based on the target image and the working height. The monitoring device 600 is modularly designed and supports OTA remote upgrade of algorithm models. The controller 17 of the aerial work platform 100 remotely downloads, updates, replaces, and optimizes the deep learning model, trajectory prediction model, Kalman filter model, image stitching algorithm, etc. deployed on the aerial work platform 100 through a wireless communication network (4G / 5G / Wi-Fi) without disassembling the device or performing on-site operations.

[0084] It should be noted that the monitoring device 600 described above can execute the monitoring method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in the embodiments of the monitoring device 600 can be found in the monitoring method provided in the embodiments of this application.

[0085] Please see Figure 8 The controller 17 includes one or more processors 171 and a memory 172. The memory 172 is connected to one or more processors 171, for example, via a bus.

[0086] Processor 171 is configured to support controller 17 in performing the corresponding functions in the methods described in the above method embodiments. Processor 171 may be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The aforementioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0087] Memory 172 is used to store program code, etc. Memory 172 may include volatile memory (VM), such as random access memory (RAM); memory 172 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory 172 may also include combinations of the above types of memory 172.

[0088] The memory 172 can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the monitoring method in the embodiments of this application. The processor 171 executes various functional applications and data processing of the monitoring method and monitoring device 600 by running the non-volatile software programs, instructions, and modules stored in the memory 172, that is, it realizes the functions of the monitoring method and the various modules or units of the monitoring device 600 provided in the above method embodiments.

[0089] The memory 172 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on the use of the monitoring device 600, etc. In some embodiments, the memory 172 may optionally include memory 172 remotely located relative to the processor 171, and these remote memories may be connected to the monitoring device 600 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0090] One or more modules are stored in memory 172. When executed by one or more processors 171, they perform the monitoring method in any of the above method embodiments. For example, they perform the method steps described in the above method embodiments to realize the function of the modules described in the above device embodiments.

[0091] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by the controller 17, cause the controller 17 to perform the method as described in the foregoing embodiments.

[0092] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0093] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A monitoring method for aerial work platforms, characterized in that, The aerial work platform includes a boom, a body, and multiple camera devices. The boom is connected to the body, and the multiple camera devices are respectively mounted on the boom and the body for acquiring multiple two-dimensional images of the aerial work platform and a preset area. The monitoring method includes: The system acquires dynamic data of the aerial work platform and multiple two-dimensional images, including the boom angle, overall vehicle posture, and working height of the aerial work platform. Based on the boom angle and the vehicle posture, a stitching and correction operation is performed on the multiple two-dimensional images to obtain the target image; A virtual boundary line is generated based on the target image and the working height. The virtual boundary line is used to identify the safe working range of the aerial work vehicle.

2. The monitoring method according to claim 1, characterized in that, The step of performing a stitching and correction operation on multiple two-dimensional images based on the boom angle and the vehicle posture to obtain the target image includes: The multiple two-dimensional images are input into a preset deep learning model to extract target features; The target features are encoded to obtain feature encoding information, which is used to provide a basis for feature matching between multiple two-dimensional images. The global context information in the two-dimensional images is captured by the convolutional and pooling layers of the deep learning model to obtain global features; By combining the boom angle and the vehicle posture, a stitching correction operation is performed on the feature encoding information and the global features to obtain the target image.

3. The monitoring method according to claim 2, characterized in that, The step of performing a concatenation and correction operation on the feature encoding information and the global features to obtain the target image includes: The feature encoding information and the global features are fused to obtain fused features; Based on the fusion features, feature matching is performed on the multiple two-dimensional images to obtain matching features; Based on the matching features, perform alignment and projection transformation operations between multiple two-dimensional images to obtain a corrected image; Perform a stitching correction operation on the corrected image to obtain the target image.

4. The monitoring method according to claim 3, characterized in that, The deep learning model includes a bottleneck layer, and the process of fusing the feature encoding information and the global features to obtain fused features includes: The bottleneck layer is used to perform global information integration processing on the global features to obtain global abstract features; The feature encoding information and the global abstract feature are fused to obtain the fused feature.

5. The monitoring method according to claim 3, characterized in that, The deep learning model includes a decoder and an encoder layer. The step of performing alignment and projection transformation operations between multiple paths of the two-dimensional images based on the matching features to obtain a corrected image includes: The decoder performs an upsampling operation on the matching features to obtain sampled features; A feature reconstruction operation is performed on the sampled features to obtain a high-resolution image structure, wherein the high-resolution image structure is a spatial structure containing the edges, contours, and textures of the two-dimensional image. Extracting low-level image features based on the coding layer; By using skip connections in the coding layer, the high-resolution image structure is fused with the low-level image features to obtain detailed features; Based on the detailed features, alignment and projection transformation operations are performed between multiple 2D images to obtain a corrected image.

6. The monitoring method according to claim 3, characterized in that, The step of performing a stitching correction operation on the corrected image to obtain the target image includes: Establish feature mapping relationships between the multiple two-dimensional images and the corrected image; Based on the feature mapping relationship, a stitching correction operation is performed on the corrected image to obtain the target image.

7. The monitoring method according to claim 1, characterized in that, The aerial work platform includes a radar device for acquiring target data of moving obstacles. After generating a virtual boundary line based on the target image and the working height, the system further includes: Based on a preset trajectory prediction model, an alarm message is generated in response to a moving obstacle intruding into a preset warning zone within a preset time. The warning zone is a safety warning area centered on the aerial work vehicle and defined according to a preset safety distance. If a moving obstacle does not intrude into a preset warning zone within a preset time, the trajectory of the moving obstacle is predicted based on the target data, and predicted trajectory information is generated.

8. The monitoring method according to claim 7, characterized in that, The warning zone includes a primary warning zone and a secondary warning zone. Based on a preset trajectory prediction model, in response to a moving obstacle intruding into the preset warning zone within a preset time, an alarm message is generated, including: In response to the moving obstacle intruding into the secondary warning zone, the location of the moving obstacle is marked; An alarm message is generated in response to the moving obstacle intruding into the primary warning zone.

9. The monitoring method according to claim 7, characterized in that, The step of predicting the trajectory of the moving obstacle by combining the target data collected by the radar device and generating predicted trajectory information includes: The target data is preprocessed to obtain valid data; The valid data is input into the trajectory prediction model to generate predicted trajectory information.

10. A controller, characterized in that, The system includes a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, and the processor, when executing the one or more computer programs, causing the controller to implement the monitoring method as described in any one of claims 1-9.

Images