Dynamic monitoring and safety verification method for attitude of boom of overhead working vehicle

Abstract

The invention discloses a dynamic monitoring and safety verification method for the attitude of a boom of a high-altitude operation vehicle, and the method comprises the steps: installing a plurality of laser radars, and enabling the scanning range of the laser radars to cover all the moving space of the boom; cantilever crane point cloud data are collected through laser radars, the point cloud data obtained through scanning of all the laser radars are uniformly converted into a vehicle body coordinate system according to the pose calibration relation between the laser radars, and the point cloud data are merged and spliced to form a complete three-dimensional point cloud of the surrounding environment of a vehicle; semantic segmentation is carried out on the complete three-dimensional point cloud by utilizing a deep neural network, all arm joints and joint points of the arm support are identified, the posture of the arm support is identified, and then the posture of the arm support is dynamically monitored and safely checked on the basis of a real-time space model of the arm support. According to the invention, real-time, rapid and accurate recognition of the posture of the boom is realized in the process of aerial operation of the vehicle, and on the basis, dynamic monitoring of the posture of the high-altitude boom and action safety early warning are realized.

Description

Technical Field

[0001] The invention relates to a method for dynamic monitoring and safety verification of an aerial work vehicle boom posture, and belongs to the field of engineering machinery. Background Art

[0002] The boom of an engineering vehicle is an important mechanical structural component used in construction operations and is widely used in lifting, pouring concrete, piling, rescue and other high-altitude operations. Existing boom posture detection methods mainly rely on mechanical sensors, inertial sensors and other technologies, mainly including:

[0003] (1) An angle sensor is installed at each articulated joint of the boom to record the rotation angle of the joint. According to the geometric structure of the boom and the known angle, the end position and posture of the boom are calculated through a kinematic model. For example, the patent, a method and device for determining the posture of the boom, a processor and engineering machinery (CN116815838A), obtains the boom angle of each boom section relative to the horizontal plane in each posture, and determines the sampling point position corresponding to the preset sampling point through the acquired boom angle, the pre-stored boom length of each boom section and the corresponding position of the preset sampling point on each boom section. Then, the obtained sampling point position is used to determine the current boom section moment value corresponding to each boom section. Then, the current boom section moment value corresponding to the boom is determined according to each current boom section moment value, based on the correspondence between the pre-stored boom section moment value and the boom posture type, and the current posture type of the boom is determined by the current boom section moment value.

[0004] (2) A gravity sensor is installed on each boom section to calculate the angle between each boom section and the horizontal line by detecting the weight of the boom section, and then calculate the angle between the boom sections.

[0005] (3) Use the base station and tag device to communicate and obtain the three-dimensional coordinates of the boom node, thereby obtaining the overall posture of the boom. For example, the patent, a large-scale mechanical boom posture perception method and device (CN117528415A), obtains the distance information between the base station and the tag at the boom node through the ranging module, combines the length information of each section of the large-scale mechanical boom, and uses the boom structure characteristics and motion characteristics to design a section-by-section positioning algorithm to solve the three-dimensional coordinates of the boom node. However, this method is limited by the positioning accuracy of the wireless positioning method, and it is difficult to accurately calculate the boom posture; and it is easily interfered by wireless communication, which reduces the detection accuracy of the boom posture.

[0006] In summary, the existing solutions have the following problems: ① Mechanical sensor data is easily disturbed by factors such as vibration and mechanical wear, and it is difficult to maintain accuracy in dynamic and high-load scenarios. ② Cameras are sensitive to lighting conditions and are easily restricted by weather or environment (such as rainy and foggy weather or dusty construction environments), and often require supplementary sensors such as lidar; and the position of the camera in the site is fixed, and it needs to be installed every time construction is carried out, which has a high installation cost. The camera on the boom has a limited field of view and may cause blind spots. Summary of the invention

[0007] In view of the problems existing in the above-mentioned prior art, the present invention provides a method for dynamic monitoring and safety verification of the boom posture of an aerial work vehicle, which can realize real-time, rapid and accurate identification of the boom posture during the vehicle's aerial operation, and on this basis, realize dynamic monitoring of the high-altitude boom posture and action safety warning.

[0008] In order to achieve the above-mentioned object, the present invention adopts a method for dynamic monitoring and safety verification of the boom posture of an aerial work vehicle, comprising the following steps:

[0009] (1) Install multiple laser radars so that the scanning range of the laser radar covers all spaces where the boom moves;

[0010] (2) The arm point cloud data is collected through the laser radar. According to the position and posture calibration relationship between the laser radars, the point cloud data obtained by all laser radar scans are uniformly converted to the vehicle body coordinate system, and the point cloud data are merged and spliced ​​to form a complete three-dimensional point cloud of the vehicle's surrounding environment;

[0011] (3) A deep neural network is used to perform semantic segmentation on the complete three-dimensional point cloud, identify all the boom sections and joints, and recognize the boom posture. Subsequently, the boom posture is dynamically monitored and safety verified based on the real-time spatial model of the boom.

[0012] As an improvement, in step (1), four laser radars are installed at four diagonal positions on the top of the vehicle cab, and one laser radar is installed on each of the left and right sides of the rear of the vehicle; the laser radar is a Livox Avia solid-state laser radar.

[0013] As an improvement, in step (2), the laser radar collects boom point cloud data, downsamples the collected point cloud data using the PCL point cloud library, aligns the point cloud data using the ICP algorithm, and finally converts the laser radar coordinate system to the vehicle body coordinate system so that the point cloud data are in the same coordinate system, and splices the point cloud data to obtain a three-dimensional model of the boom.

[0014] As an improvement, when the laser radar coordinate system is converted to the vehicle body coordinate system, the relationship between the points in the laser radar coordinate system and the points in the vehicle body coordinate system is:

[0015] The coordinates of the point in the vehicle coordinate system are (x c ,y c ,z c ), the coordinates of the point in the laser radar coordinate system are (x l ,y l ,z l ).

[0016] As an improvement, after the point cloud data in step (2) is spliced, it is labeled using the Point Labeler tool.

[0017] As an improvement, the annotated point cloud data in step (3) is combined with a semantic segmentation network based on a 3D U-Net framework, and a cylindrical partitioning module is used to achieve accurate segmentation of the arm area and eliminate interfering objects.

[0018] As an improvement, the safety check in step (3) includes: fitting the central axis of each boom in the model according to the real-time spatial model of the boom, determining the relationship between the length of each boom and the angle between the booms, comparing the value with the detection value of the angle sensor and the length sensor installed on each boom, so as to achieve double verification of the boom posture.

[0019] As an improvement, during the operation of the aerial work vehicle, the laser radar continuously obtains the real-time point cloud data of the boom, and calculates the horizontal projection distance D of the farthest node of the boom according to the three-dimensional coordinates of the point cloud at the end of the boom. x and vertical projection distance H x ,Right now:

[0020]

[0021] H x =z i -z0

[0022] Among them, (x0, y0, z0) are the coordinates of the vehicle base, (x i ,y i ,z i ) is the horizontal coordinate of the farthest point of the boom;

[0023] If the horizontal projection distance D of the farthest node of the boom is detected x Exceeding the maximum limit or vertical projection distance H xIf the maximum height limit is exceeded, or the boom length or angle exceeds the limit, it is determined that the posture does not meet the safety verification conditions, an alarm signal is issued, and further extension or retraction operations of the boom are restricted.

[0024] As an improvement, according to the real-time spatial model of the boom, when there is an object around the boom and it approaches a certain distance from the boom, an anti-collision safety warning is issued through the ultrasonic sensor installed on the side of the vehicle, and the operator is reminded of the warning position;

[0025] When the object approaches to the limit distance, an anti-collision safety alarm is issued and the arm is restricted from moving toward that side.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] ① Using solid-state laser radar, which has the characteristics of long distance, high precision and high scanning frequency, is suitable for use in the working environment of the boom of aerial work vehicles. Multiple laser radars (for example, six radars) are used to ensure that complete point cloud data can always be obtained during the boom deployment and rotation process.

[0028] ② Obtain the point cloud data of the boom in different postures, use the PCL point cloud library to downsample, align, and splice the point cloud to obtain a complete 3D model of the boom, and complete the preprocessing of the point cloud data. Use the ICP algorithm to align the point cloud to ensure that the point cloud data collected by different lidars can be correctly merged and spliced.

[0029] ③ Use the Point Labeler tool to label the 3D point cloud training samples and clearly distinguish the various parts of the boom. Use a semantic segmentation network based on the 3D U-Net framework, combined with a cylindrical partitioning module, to handle density changes and sparse point clouds well. The model achieves accurate segmentation of the boom area and eliminates interfering objects through point-by-point feature extraction, voxel classification, and point-by-point refinement. Use the PCL tool to display the segmented point cloud and highlight the various parts of the boom.

[0030] ④ Compare the actual boom posture with the theoretical posture calculated by the sensor in real time, perform double safety verification of the posture, and compare it with the safe operating range of the boom of the aerial work vehicle to improve the safety of boom operation.

[0031] ⑤ By monitoring the distance between the boom and surrounding obstacles through a real-time model, it can provide all-round protection to avoid collision accidents when the boom moves. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0033] Figure 1 A top view of the laser radar installation structure of the present invention;

[0034] Figure 2 A side view of the laser radar installation structure of the present invention;

[0035] Figure 3 It is a schematic diagram of the connection between the laser radar and the vehicle controller of the present invention;

[0036] Figure 4 It is a schematic diagram of the vehicle body coordinate system of the present invention;

[0037] Figure 5 It is a schematic diagram of the rotation of the laser radar coordinate system of the present invention around the z-axis;

[0038] Figure 6 It is a schematic diagram of XOY plane coordinate system transformation of the present invention;

[0039] Figure 7 It is a schematic diagram of YOZ plane coordinate system transformation of the present invention;

[0040] Figure 8 This is a flow chart of point cloud data collection and preprocessing of the present invention;

[0041] Fig. 9 This is a point cloud stitching flow chart of the present invention;

[0042] Fig.10 It is a network model framework diagram of the present invention;

[0043] Fig.11 It is a flow chart of arm posture verification of the present invention;

[0044] Fig.12 It is a boom safety warning flow chart of the present invention;

[0045] Fig.13 This is a flow chart of the anti-collision safety warning of the present invention. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present invention more clear, the technical solutions of the present application are described in detail below with the help of accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solutions of the present application, rather than limitations on the technical solutions of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0047] A method for dynamic monitoring and safety verification of the boom posture of an aerial work vehicle comprises the following steps:

[0048] S1. LiDAR installation

[0049] The laser radar can obtain the scanning data of the object at continuous time points, and because the boom moves at a relatively slow speed, and the slow movement usually does not cause serious blurring of the point cloud, the radar can clearly capture the shape and position changes of the object. In the present invention, the Livox Avia solid-state laser radar is used as an example for illustration. It has the characteristics of long range (up to 450m), high precision, light weight (only 498g) and stability. The laser radar can generate high-precision three-dimensional point cloud data and provide more accurate spatial position information than traditional sensors (such as angle sensors or IMU); for complex boom structures, the laser radar can capture the slight changes of the boom in three-dimensional space and accurately determine the position, angle and spatial posture of the end of the boom. Compared with other methods, the laser radar has lower error accumulation and higher measurement accuracy, and is particularly suitable for real-time posture detection in dynamic environments. The laser radar has strong environmental adaptability and can work in various complex and harsh environments, including low light, strong light, rain, fog, dust and other conditions. Unlike visual sensors that rely on light, LiDAR measures distance directly through laser beams, is not affected by changes in ambient light, and can provide stable performance at night, in haze or other adverse weather conditions. LiDAR performs non-contact measurements, which avoids physical wear and extends the service life of the sensor. Therefore, it is very suitable for situations where the work platform is far away from the LiDAR after the boom is fully extended. The field of view of Livox Avia is a cone area of ​​77.2° vertically and 70.4° horizontally.

[0050] The boom moves in a three-dimensional space, and the installation plan of the laser radar needs to ensure that the range scanned by the laser radar covers all the spaces where the boom moves. The installation plans described below are limited to the case vehicle, and the installation plans for different models need to be adjusted according to the specific situation.

[0051] Taking the XCMG rescue fire truck DG100 as an example, the vehicle is about 12m long, the front of the vehicle is about 3m long, and the width is 2.55m. The boom has three boom sections. The first telescopic arm has a telescopic range of 14.07-52m, the second telescopic arm has a telescopic range of 15-35.5m, and the third boom section is not retractable. The boom of the fire truck is in a folded state when not performing tasks, and is unfolded in a working state, and can rotate 360° at the fulcrum of the rear of the vehicle. In order to capture the overall posture of the boom, the present invention adopts Figure 1 , Figure 2 The installation structure shown.

[0052] A laser radar is installed at the front end of the vehicle facing the boom, and a yaw angle is generated in the direction of the boom extension on the basis of facing the boom. In this way, since the actual horizontal field of view of the Livox Avia laser radar is conical, all point cloud data in the horizontal direction of the boom can be obtained. Because the boom can rotate 360° around the rotation center of the vehicle body, another laser radar is installed on the other side in the same way;

[0053] However, the maximum vertical field of view of the Livox Avia LiDAR is only 77.2°, and when the first and second boom sections are fully deployed, the radar field of view cannot cover the entire range of the boom. Therefore, two LiDARs of the same model are needed to complement each other's vertical field of view. One is located at the very end of the front of the vehicle, and the other is located at the very front of the vehicle. The direction they face forms an angle with the direction of the above-mentioned LiDAR, so that the radar field of view can cover the entire area of ​​the three-dimensional space.

[0054] In addition, since the laser radar at the front of the vehicle cannot observe the state of the arm rotating to the rear of the vehicle, a laser radar is installed on both sides of the rear of the vehicle. In order to fully scan the state when the arm is farthest from the vehicle in the horizontal direction, one side of the radar is installed at a certain angle toward the rear of the vehicle, and the other side is installed at a certain angle toward the front of the vehicle, so that they can complement each other's horizontal visual blind spots. In the vertical direction, both radars rotate in the positive direction around the x-axis in their own coordinate system, forming a certain angle between the two radars to cover the entire area of ​​the three-dimensional space.

[0055] LivoxAvia LiDAR is usually connected via Ethernet. The DG100 aerial fire truck uses a vehicle-mounted controller designed for fire truck boom operation and monitoring. Therefore, first use a standard Cat5e / Cat6 network cable to connect the Ethernet interface of LivoxAvia to the Ethernet interface (RJ45) of the vehicle-mounted controller. Then use Livox Viewer to configure the static IP address of the LiDAR or directly modify the IP address of the vehicle-mounted controller to ensure that the LiDAR and the vehicle-mounted controller are in the same network segment. Figure 3 shown.

[0056] S2. Boom space point cloud data collection and processing

[0057] It mainly includes collecting point cloud data of various postures of the boom, and splicing and labeling the collected point cloud data.

[0058] When collecting arm point cloud data through lidar, first install the Livox SDK and Livox ROS Driver, run the corresponding launch configuration file in Livox ROS Driver, receive and display the point cloud data scanned by the lidar in real time, and run the topic recording node on the ROS terminal to store the point cloud data in rosbag; then parse the point cloud data in rosbag into the PCD format supported by the PCL library to prepare for subsequent point cloud stitching. However, since the point cloud data is collected by multiple lidars and is not in the same coordinate system, it is necessary to convert all lidar coordinate systems to the vehicle coordinate system so that the point clouds are in the same coordinate system before point cloud stitching can be performed.

[0059] Assume that the vehicle adopts a Cartesian coordinate system, with the center of rotation of the arm rotating the vehicle body as the origin of the vehicle body coordinate system, the direction of the vehicle head is the positive direction of the y-axis, and the right side of the direction perpendicular to the vehicle body is the positive direction of the x-axis, such as Figure 4 shown.

[0060] There are six laser radars installed on the car, so six coordinate system conversions are required. The six rotations of the laser radar coordinate system to the vehicle body coordinate system include the rotation around the z-axis to generate the yaw angle and the rotation around the x-axis to generate the pitch angle. The transformation of the XOY plane and YOZ plane coordinate systems must be considered at the same time.

[0061] First, consider the part where the laser radar coordinate system rotates around the z-axis to generate the yaw angle yaw. In this case, take the part where a laser radar generates a yaw angle transformation around the z-axis as an example. The coordinates of the point in the laser radar coordinate system before the coordinate system transformation are P(x, y), and the coordinates of the point in the vehicle coordinate system after the transformation are P'(x', y'), then:

[0062]

[0063] R is the rotation matrix, which converts the coordinates of point P in the rotated coordinate system to the coordinates of the coordinate system before rotation. The xyz coordinate system before rotation is the coordinate system of the laser radar, and after rotation it is the x'-y'-z' coordinate system, such as Figure 5 shown.

[0064] The transformation relationship of the coordinate system on the XOY plane is as follows Figure 6 As shown:

[0065] According to the angle θ before and after rotation, the following formula can be listed:

[0066]

[0067] Then the rotation matrix R is

[0068] If the three-dimensional coordinates (x0, y0, z0) of the laser radar relative to the vehicle coordinate origin are known, the translation matrix of the laser radar and the vehicle coordinate system can be obtained as:

[0069]

[0070] Assume that the coordinates of the point in the vehicle coordinate system are (x c ,y c ,z c ), the coordinates of the point in the laser radar coordinate system are (x l ,y l ,z l ), according to the rotation matrix R and translation matrix T calculated above, we can find:

[0071]

[0072] The pitch angle transformation is the same as above. The laser radar needs to rotate around the x-axis, and the YOZ plane coordinate system transformation relationship is as follows: Figure 7 .

[0073] The following formula can be listed:

[0074]

[0075] Then the rotation matrix R is

[0076] Similarly:

[0077]

[0078] Different rotation matrices R can be obtained by rotating the coordinate axes in different orders. Since the rotation strategy adopted is to rotate around a fixed axis each time, the left multiplication method is adopted:

[0079] p c =R yaw R pitch p l

[0080] Based on the above derivation, the relationship between the points in the radar coordinate system and the points in the vehicle coordinate system is:

[0081]

[0082] Subsequently, the point cloud data collected by the LiDAR can be uniformly converted to the vehicle coordinate system according to the above method and directly added.

[0083] ROS provides a convenient command line tool static_transform_publisher, which is used to define the relationship between two coordinate systems in a static way. It will continuously publish a fixed transformation message to the ROS TF tree. You can perform tf transformation directly through the command line:

[0084] rosrun tf2_ros static_transform_publisher xyz yaw pitch rollparent_frame child_frame

[0085] The first three values ​​in the parameter list in the command line represent the translation transformation of the coordinate system, which are the values ​​of x, y, and z in meters, and the last three values ​​represent the rotation transformation of the coordinate system, which are the values ​​of Euler angles roll, pitch, and yaw in radians; parent_frame and child_frame are the names of the two transformed coordinate systems, indicating the transformation from the parent_frame coordinate system to the child_frame coordinate system, which is actually the transformation from the livox coordinate system to the vehicle coordinate system in this invention. The specific processing flow is as follows Figure 8 .

[0086] Since multiple tf transformations are required in the usage scenario of this article, you can use the launch file to start multiple nodes at one time to complete multiple tf transformations.

[0087] The PCL library is an open source C++ library specifically used to process and analyze 3D point cloud data. It contains many modules related to point cloud stitching, including point cloud filtering module, feature extraction module, registration module, point cloud fusion and stitching module, etc. These modules can be used to achieve point cloud stitching. Fig. 9 As shown in the figure, we first use pcl::VoxelGrid in the PCL library to downsample the point cloud to reduce the number of point clouds and improve the subsequent calculation efficiency. Then we use the ICP algorithm pcl::IterativeClosestPoint to align the point cloud. Finally, according to the transformation matrix obtained by the alignment, we use pcl::ConcatenatePointCloud to transform the coordinates of each point cloud and align them to the same coordinate system to finally obtain a complete three-dimensional model of the boom. After the splicing is completed, the point cloud data is extracted from the pcl::PointCloud object as binary data and written into a BIN file, and the point cloud in pcd format is converted to BIN format to facilitate the subsequent point cloud annotation operation.

[0088] In order to detect and identify the various parts of the boom in the future, a large amount of data of various postures of the boom with annotation information is required. Therefore, after completing the point cloud stitching, the point cloud needs to be labeled. The Point Labeler tool is used to label the existing point cloud BIN file and divide the boom into several parts: base, first arm, second arm and joint.

[0089] S3. Boom area model segmentation

[0090] A semantic segmentation network for point clouds is used to classify each pixel in the image or point cloud as a label, accurately segmenting the boom area. In addition, the boom can be distinguished from other surrounding objects.

[0091] According to the above processing steps, after obtaining the labeled data set, the semantic segmentation model is trained. In this step, a network model is introduced. Based on the basic structure framework of 3D U-Net, the network model adds a cylindrical partitioning module to construct an asymmetric three-dimensional convolution. This network model can retain the three-dimensional structure information of the original data while still being able to deal with the problems caused by variable density and sparse point clouds. It has a good effect on the arm part that is in the unfolded state and far away from the lidar. The overall framework of the model is as follows Fig.10 shown.

[0092] The process of inputting the 3D point cloud data collected by the LiDAR into the trained point cloud segmentation model is as follows:

[0093] (1) The original point cloud data is fed into the MLP to obtain point-by-point features, and the generated features will be redistributed based on columnar partitioning;

[0094] (2) Feeding the voxel-based feature data into an asymmetric three-dimensional convolutional network to generate voxel-based classification results;

[0095] (3) A point-by-point refinement module is introduced to improve the output results and obtain the final accurate output.

[0096] Finally, the segmented point cloud is displayed through PCL to obtain the three-dimensional model of the boom in real time and highlight the various parts of the boom.

[0097] S4, double check of boom posture

[0098] During the boom operation, due to inherent problems such as boom deformation and calculation errors introduced by angle conversion, there may be a certain error between the measured value obtained by the angle sensor or length sensor and the actual posture of the boom. There is also a difference between the boom posture display reproduced based on this measurement value and the actual posture.

[0099] The real-time boom model obtained through laser radar point cloud processing and the fitting processing of the central axis of each boom in the model can determine the relationship between the length of each boom and the angle between the booms, and compare this value with the detection value of the angle sensor and length sensor installed on each boom to achieve double verification of the boom posture. Fig.11 When checking, a threshold is set according to the actual situation. When the threshold is exceeded, an alarm message and two measurement messages are output.

[0100] S5, Boom Action Safety Warning:

[0101] In order to ensure that the boom posture of aerial work vehicles meets safety requirements during operation and avoid vehicle instability or operation hazards due to abnormal posture, various safety restrictions are imposed on the boom, including telescopic boom length limit, boom deployment angle limit, boom height limit, etc., to ensure the stability and safety of the vehicle. After obtaining the boom posture angle according to the boom model, the boom length limit and deployment angle limit can be determined, but due to factors such as boom deformation under force, the height limit and amplitude limit need to be further calculated according to the model.

[0102] During the operation of the aerial work vehicle, the laser radar continuously obtains the real-time point cloud data of the boom, and calculates the horizontal projection distance D of the farthest node of the boom according to the three-dimensional coordinates of the point cloud at the end of the boom. x and vertical projection distance H x ,Right now:

[0103]

[0104] H x =z i -z0

[0105] Among them, (x0, y0, z0) are the coordinates of the vehicle base, (x i ,y i ,z i ) is the horizontal coordinate of the farthest point of the boom.

[0106] If the horizontal projection distance D of the farthest node of the boom is detected x Exceeding the maximum limit or vertical projection distance H x If the maximum height limit is exceeded, or the boom length or angle exceeds the limit, the posture is judged to not meet the safety verification conditions, an alarm signal is issued, and further extension or retraction of the boom is restricted. Fig.12 shown.

[0107] S6. Boom environment perception and anti-collision safety warning:

[0108] When the boom is moving, for safety reasons, it is necessary to avoid collision accidents. Currently, ultrasonic sensors are installed on all sides of the platform to avoid platform collision and ensure the safety of passengers. However, the sensors cannot completely cover the entire moving boom, and collisions can only be avoided by the operator's naked eye in the blind area of ​​the sensor.

[0109] When processing the point cloud model, we simultaneously obtain the point cloud model of the boom's surrounding environment by segmenting the boom's point cloud. When an object around the boom approaches the boom to a certain distance, an anti-collision safety warning is issued, and the operator is reminded of the warning location; when the object approaches the limit distance, an anti-collision safety alarm is issued, and the boom is restricted from moving toward that side. Fig.13 shown.

[0110] The present invention utilizes the long-range, high-precision and high-scanning frequency characteristics of Livox Avia solid-state laser radar, solves the problem of insufficient vertical field of view through front and rear complementary installation and layout of multiple laser radars, ensures that all parts of the boom can be fully scanned during the boom deployment and rotation process, and obtains comprehensive point cloud data. Subsequently, the PCL point cloud library is used to downsample, align and splice the collected point cloud data, and reconstruct the complete three-dimensional model of the boom as data preprocessing. The ICP algorithm is used in the alignment process to ensure that the data collected by multiple radars can be accurately fused. The point cloud training samples are labeled by the Point Labeler tool, combined with the semantic segmentation network based on the 3D U-Net framework, for variable density and sparse point cloud optimization processing, the cylindrical partition module is used to achieve accurate segmentation of the boom area and eliminate interfering objects. The segmented point cloud is displayed with the PCL tool, highlighting each part of the boom, and obtaining the boom model and the model of the surrounding environment. By acquiring the boom posture data in real time, the double posture verification is completed together with the sensor data, and it is determined whether it meets the boom safety limit requirements to ensure that the boom is always operating within a safe range. In addition, the distance between the surrounding environment and the boom is calculated based on the model, and when the anti-collision distance limit is reached, the boom anti-collision alarm is triggered to avoid collision accidents.

[0111] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features contained in other embodiments but not other features, the combination of features from different embodiments is also meant to be within the scope of protection of the present invention and to form different embodiments.

Claims

1. A method for dynamic monitoring and safety verification of the boom posture of an aerial work vehicle, characterized in that: The following steps are involved: (1) Install multiple laser radars so that the scanning range of the laser radar covers all spaces where the boom moves; (2) The arm point cloud data is collected through the laser radar. According to the position and posture calibration relationship between the laser radars, the point cloud data obtained by all laser radar scans are uniformly converted to the vehicle body coordinate system, and the point cloud data are merged and spliced ​​to form a complete three-dimensional point cloud of the vehicle's surrounding environment; (3) A deep neural network is used to perform semantic segmentation on the complete three-dimensional point cloud, identify all the boom sections and joints, and recognize the boom posture. Subsequently, the boom posture is dynamically monitored and safety verified based on the real-time spatial model of the boom.

2. The method for dynamic monitoring and safety verification of the boom posture of an aerial work vehicle according to claim 1, characterized in that: In the step (1), four laser radars are installed at four diagonal positions on the top of the vehicle cab, and one laser radar is installed on each of the left and right sides of the rear of the vehicle; the laser radar is a Livox Avia solid-state laser radar.

3. The method for dynamic monitoring and safety verification of the boom posture of an aerial work vehicle according to claim 1, characterized in that: In the step (2), the laser radar collects the boom point cloud data, downsamples the collected point cloud data using the PCL point cloud library, aligns the point cloud data using the ICP algorithm, and finally converts the laser radar coordinate system to the vehicle body coordinate system so that the point cloud data are in the same coordinate system. The point cloud data are spliced ​​to obtain a three-dimensional model of the boom.

4. A method for dynamic monitoring and safety verification of boom posture of aerial work vehicle according to claim 3, characterized in that: When the laser radar coordinate system is converted to the vehicle body coordinate system, the relationship between the points in the laser radar coordinate system and the points in the vehicle body coordinate system is: The coordinates of the point in the vehicle coordinate system are (x c ,y c ,z c ), the coordinates of the point in the laser radar coordinate system are (x l ,y l ,z l ).

5. The method for dynamic monitoring and safety verification of the boom posture of an aerial work vehicle according to claim 1, characterized in that: After the point cloud data in step (2) is spliced, it is labeled using the Point Labeler tool.

6. The method for dynamic monitoring and safety verification of the boom posture of an aerial work vehicle according to claim 1, characterized in that: In the step (3), the annotated point cloud data is combined with a semantic segmentation network based on a 3D U-Net framework, and a cylindrical partitioning module is used to achieve accurate segmentation of the arm area and eliminate interfering objects.

7. The method for dynamic monitoring and safety verification of boom posture of aerial work vehicle according to claim 1, characterized in that: The safety check in step (3) includes: fitting the central axis of each boom in the model according to the real-time spatial model of the boom, determining the relationship between the length of each boom and the angle between the booms, comparing the value with the detection value of the angle sensor and the length sensor installed on each boom, and realizing double check of the boom posture.

8. A method for dynamic monitoring and safety verification of boom posture of aerial work vehicle according to claim 7, characterized in that: During the operation of the aerial work vehicle, the laser radar continuously obtains the real-time point cloud data of the boom, and calculates the horizontal projection distance D of the farthest node of the boom according to the three-dimensional coordinates of the point cloud at the end of the boom. x and vertical projection distance H x ,Right now: H x =z i -z0 Among them, (x0, y0, z0) are the coordinates of the vehicle base, (x i ,y i ,z i ) is the horizontal coordinate of the farthest point of the boom; If the horizontal projection distance D of the farthest node of the boom is detected x Exceeding the maximum limit or vertical projection distance H x If the maximum height limit is exceeded, or the boom length or angle exceeds the limit, it is determined that the posture does not meet the safety verification conditions, an alarm signal is issued, and further extension or retraction operations of the boom are restricted.

9. A method for dynamic monitoring and safety verification of boom posture of aerial work vehicle according to claim 7 or 8, characterized in that: According to the real-time spatial model of the boom, when an object around the boom approaches a certain distance from the boom, an anti-collision safety warning is issued through the ultrasonic sensor installed on the side of the vehicle, and the operator is reminded of the warning position; When the object approaches to the limit distance, an anti-collision safety alarm is issued and the arm is restricted from moving toward that side.

Images