Snow edge detection method in snow shoveling process, medium and electronic equipment

Abstract

The invention discloses an accumulated snow edge detection method in a snow shoveling process, a medium and electronic equipment, and the method comprises the steps: detecting the longitudinal section two-dimensional point cloud data of one side, close to a snow shovel, of accumulated snow through a radar; establishing a radar local coordinate system for describing each frame of scanning plane of the radar; splicing radar scanning point cloud data and preprocessing the cloud data; a statistical filter in the three-dimensional point cloud data is utilized to select the three-dimensional point cloud data spliced by a plurality of frames of scanning plane point clouds to carry out abnormal point elimination, and the three-dimensional point cloud data after noise removal is obtained; carrying out plane normal vector solving on the group of three-dimensional point cloud data without the abnormal points; and calculating a group of three-dimensional point cloud data fitting plane normal vector included angles, and judging the position of the accumulated snow part scanned by the radar. According to the invention, the defects of many artifacts and low resolution of the millimeter wave radar can be overcome, and the problems caused by human eye observation errors in manual snow shoveling operation are avoided.

Description

Technical Field

[0001] The present invention belongs to the field of radar detection technology, and in particular relates to a method, medium and electronic equipment for detecting the edge of snow during snow shoveling. Background Art

[0002] Currently, snowplows in China primarily rely on manually controlled snowplows. During manual snowplowing, the driver visually analyzes the snow coverage on the road, divides lanes according to the specific snow conditions, and determines the snowplow's route. The driver then manually controls the snowplow's upward, downward, leftward, and floating movements. In this commonly used lane-by-lane snowplowing method, the snowplow is typically angled left or right to push the snow toward the roadside. If there are multiple lanes, multiple snowplows are required. When the first snowplow pushes snow to the second lane on one side of the road, the snowplow in the second lane then angles its snowplow to the adjacent lane in the same direction, and so on, until the snow is pushed to the roadside. Therefore, without detecting the edge of the snowplow and the snowplow angle to proactively adjust the snowplow angle, previously cleared snow may be pushed back into the original lane, significantly impacting snowplow efficiency.

[0003] In this process, the human eye is primarily responsible for determining the snow accumulation and shovel angle. Even in clear, sunny weather, the glare from the sun's reflection from the accumulated snow makes it difficult for drivers to accurately judge the position, often leading to the shovel deviating significantly from the edge of the snow. In poor lighting or in rainy, snowy, or foggy conditions, the human eye also struggles to discern the edge of the snow and the angle of the shovel. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide a method, medium and electronic device for detecting the edge of snow during snow shoveling, which overcome the above problems or at least partially solve the above problems.

[0005] To achieve the above-mentioned object, in a first aspect of the present application, a method for detecting the edge of snow during snow shoveling is provided, the method comprising:

[0006] The radar is used to detect the 2D point cloud data of the longitudinal section of the snow near the snow shovel side;

[0007] A radar local coordinate system is established to describe the radar scanning plane of each frame. The two-dimensional plane discrete points scanned by the radar are expanded into three-dimensional point cloud data based on the time series and the snow shovel swing angular velocity.

[0008] Stitching radar scan point cloud data and preprocessing the cloud data;

[0009] Using the statistical filter in the 3D point cloud data, the 3D point cloud data obtained by splicing several frames of scanned plane point clouds is selected to remove abnormal points, thus obtaining the 3D point cloud data after noise removal.

[0010] Solve the plane normal vector of a set of 3D point cloud data after removing abnormal points;

[0011] Calculate the included angle of the plane normal vector fitted to a set of 3D point cloud data to determine the position of the snow part scanned by the radar.

[0012] Optionally, the radar is installed on the left and right sides of the snow plow of the snowplow. The radar is a millimeter wave radar. The scanning field of view angle of the radar is a ±60 degree sector area within the sensing distance range of 0-70 meters and a ±10 degree sector area in the vertical direction of the beam.

[0013] Optionally, establishing a radar local coordinate system that describes the radar scanning plane for each frame includes:

[0014] The ray perpendicular to the radar plane from the center of the radar to the far end is the horizontal axis of the coordinate system, and the vertical ray drawn from the center of the radar in the radar scanning plane is the vertical axis of the radar local coordinate system;

[0015] Establish a three-dimensional coordinate system for the radar rotation process, and correct the moving distance of the scanning plane in the coordinate system at each moment according to the different angular velocities during movement;

[0016] Before snow plowing begins, the center of the radar where the snowplow is about to start moving forward is taken as the origin (0,0,0). The X and Z axes are the horizontal and vertical axes of the radar's local coordinate system at this time, respectively. The Y axis is the ray perpendicular to the XOZ plane from the origin, which is used to represent the relative distance between each corrected scan plane and the first scan plane.

[0017] Optionally, after preprocessing the cloud data, artifact points are removed based on quality information obtained by the radar.

[0018] Optionally, the preprocessing of cloud data includes:

[0019] By combining the band-limited filtering concept used in analog signals with the actual working conditions of snowplows, the point cloud data is restricted to a range within a depth of 15 meters in a local coordinate system with the radar center as the far point. This creates a plane field of view of an isosceles triangle with a top angle of 120 degrees, a base angle of 30 degrees, a waist length of 30 meters, and a base length of 52 meters.

[0020] Remove unreliable points in the scan by using false alarm probability, Doppler ambiguity, cluster uncertainty and radar cross section;

[0021] After integrating the angular velocity within each frame interval of the radar, the arc length between the two frames is obtained by combining the gyration radius, and the plane rectangular coordinate difference between the beginning and end of this arc is calculated as the correction value of the coordinate point of the next frame data.

[0022] The data is stitched together to obtain the 3D point cloud data for this time interval. Based on the principle of statistical filtering, the 3D Euclidean distances from all points in a set of data to other points are calculated. The adjacent points are selected, and the average distance from each point to the adjacent points is calculated.

[0023] The global average distance standard deviation of the neighboring points is used as the threshold, the average distance of the neighboring points minus the global average distance is compared with the set threshold, and the points exceeding the threshold are defined as outliers and screened out.

[0024] Optionally, solving a plane normal vector for a set of 3D point cloud data with outliers removed includes:

[0025] Assume that the fitting plane equation of this set of point clouds is ax+by+cz=d, (x, y, z) is the 3D point cloud data obtained by scanning, and (a, b, c) is the normal vector of the fitting plane;

[0026] First, to facilitate program operations, its matrix representation Right now

[0027]

[0028] Solving the unknowns (a, b, c) is the normal vector of the fitting plane At this time, the problem of finding the normal vector of the fitted plane is transformed into the problem of finding the optimal solution of an overdetermined set of equations, and the unknowns solved by the least squares method are selected as the parameters of the normal vector of the fitted plane.

[0029] Optionally, determining the position of the snow portion scanned by the radar includes:

[0030] Take the projection of the normal vector onto the coordinate plane XOZ and take the two-dimensional vector The angle between the fitting plane and the positive direction of the X-axis is obtained by calculating the supplementary angle;

[0031] Combined with the cantilever pitch angle and the correction of the angle between the radar and the stockpile plane, the real-time fitting angle α between the reclaiming surface and the horizontal plane is calculated.

[0032] The snow repose angle is used as the judgment threshold. When the fitting plane angle α is less than the snow repose angle, it is judged that the radar scanning surface is outside the snow. At this time, the position of the snow shovel when the vehicle moves forward can be predicted based on the distance between the radar and the snow shovel.

[0033] In a second aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program. When the computer program is loaded and executed by a processor, the method described in any one of the first aspects is adopted.

[0034] In a third aspect of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor loads and executes the computer program, the method described in any one of the first aspects is adopted.

[0035] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0036] This invention calculates three-dimensional point cloud data from a two-dimensional radar scan surface based on the radar's temporal position changes. Directly analyzing the spliced ​​three-dimensional point cloud data not only increases the amount of information, but also serves as a basis for outlier removal, overcoming the shortcomings of millimeter-wave radar, such as the high artifacts and low resolution. This also avoids the problems caused by human visual errors during manual snow shoveling. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic flow chart of a method for detecting snow edges during snow shoveling provided in an embodiment of the present application;

[0038] Figure 2 This is a schematic diagram of the radar installation location in the application embodiment. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0040] See also Figure 1 This embodiment provides a method for detecting snow edges during snow shoveling, including:

[0041] S1. The millimeter-wave radars mounted on the left and right sides of the snowplow cantilever detect the two-dimensional point cloud data of the longitudinal section of the snow near the snowplow side. That is, the relative position of the detected object and the millimeter-wave radar at that moment is obtained in the form of plane rectangular coordinates.

[0042] Since the metal structures on the left and right sides of the cantilever are slightly different, the radar should be installed at a location without any obstructions in front of the radar according to the actual situation. Figure 2As shown, the horizontal and vertical distances between the left and right radars and the center of the snow shovel are measured. The scanning field of view of the millimeter-wave radar used in this method is a sector-shaped area of ​​±60 degrees (i.e., a total of 120 degrees) within the sensing distance of 0-70 meters and a sector-shaped area of ​​±10 degrees in the vertical direction of the beam (due to the characteristics of the millimeter-wave radar, it does not have the ability to distinguish the vertical direction of the beam field of view, so the data obtained is the Euclidean distance and azimuth of the object being measured relative to the radar); the millimeter-wave radar is placed vertically. According to the characteristics of the beam scanning, the scanning process of each frame of the radar (a total of 72 milliseconds) is to scan upward from the radar itself at +60 degrees to -60 degrees in the horizontal direction (i.e., the vertical ray of the radar center point is 0 degrees). According to the characteristics of the radar beam scanning, the original data is in the polar coordinate system and needs to be converted to a plane rectangular coordinate system.

[0043] S2. Establish a radar local coordinate system that describes the radar scanning plane in each frame. Expand the two-dimensional plane discrete points scanned by the radar into three-dimensional point cloud data based on the time series and the snow shovel swing angular velocity. That is, it becomes a three-dimensional point cloud data constructed over time in a three-dimensional coordinate system.

[0044] The radar false alarm probability, Doppler ambiguity, cluster uncertainty and radar cross section are the key factors affecting the performance of radar signal processing and target analysis;

[0045] An in-depth analysis of the key factors mentioned above is conducted, and a reasonable comprehensive utilization strategy is applied to effectively improve the efficiency and accuracy of removing unreliable points, thereby improving the overall performance of radar signal processing and target analysis.

[0046] These factors are complex and mixed, and there is no order of precedence.

[0047] A local radar coordinate system is established, with the horizontal axis (i.e., the distance from the radar) extending perpendicular to the radar plane and extending from the radar center to the far end. A vertical axis is also established for the radar's local coordinate system. A three-dimensional radar coordinate system is established. Since the millimeter-wave radars are fixed to either side of the snowplow and continuously move forward with the snowplow, the distance traveled by the scanning plane in the coordinate system at each moment must be corrected based on the angular velocity during movement. The origin is (0,0,0), with the radar center at the moment the snowplow begins moving forward, just before snowplowing begins. The X-axis and Z-axis are the horizontal and vertical axes of the radar's local coordinate system at that moment, respectively. The Y-axis is a ray perpendicular to the XOZ plane extending from the origin, representing the relative distance of each scan plane from the first scan plane after correction.

[0048] S3. Stitch the radar scan point cloud data and pre-process the cloud data to limit the coordinate range of the X-axis and Z-axis of the point cloud data; remove artifact points based on the quality information obtained by the radar; then use the statistical filter commonly used in 3D point cloud data to select several frames of scan plane point cloud stitching 3D point cloud data to remove abnormal points and obtain the 3D point cloud data after noise removal.

[0049] First, by combining the band-limiting filtering concept used in analog signals with the actual working conditions of snowplows, the point cloud data is restricted to a depth of 15 meters in a local coordinate system with the radar center as the far point to avoid the impact of snow accumulation on off-highway lanes. This creates a plane field of view of an isosceles triangle with a vertex angle of 120 degrees, a base angle of 30 degrees, a waist length of 30 meters, and a base length of 52 meters. Unreliable points in the scan are then removed using data such as false alarm probability, Doppler ambiguity, cluster uncertainty, and radar cross section. Second, the angular velocity is integrated within each millimeter-wave radar frame interval. The arc length between the two frames is calculated by combining it with the radius of gyration. The difference in the plane rectangular coordinates between the beginning and end of this arc is then calculated as the coordinate correction value for the next frame. Finally, the data is spliced ​​to obtain the 3D point cloud data for this time interval (this group). Based on the principle of statistical filtering, the 3D Euclidean distances from all points in a group to other points are calculated. "Neighboring points" are selected according to a certain ratio, and the average distance from each point to its neighbors is calculated. At this time, all points in this group have an average distance to their neighboring points. The global average distance standard deviation of the neighboring points is used as the threshold (the threshold can be enlarged or reduced according to actual needs). The global average distance is subtracted from the average distance of the neighboring points and then compared with the set threshold. Points exceeding the threshold are defined as outliers and filtered out.

[0050] S4. Solve the plane normal vector of a set of three-dimensional point cloud data after removing abnormal points, where the fitting plane equation of this set of point clouds is set to ax+by+cz=d, (x, y, z) is the three-dimensional point cloud data obtained by scanning, (a, b, c) is the normal vector of the fitting plane, and the vector projection on the XOZ plane is obtained by the projection method.

[0051] First, to facilitate program operations, its matrix representation Right now

[0052]

[0053] Solving the unknowns (a, b, c) is the normal vector of the fitting plane At this time, the problem of finding the normal vector of the fitted plane is transformed into the problem of finding the optimal solution of an overdetermined set of equations, and the unknowns solved by the least squares method are selected as the parameters of the normal vector of the fitted plane.

[0054] Considering that the snow edge detection in snow shoveling operations requires real-time performance and at least sub-second feedback, the coefficient z in the plane equation is set to 1 to reduce the amount of calculation and rewritten as ax+by+d=z. Considering that step S5 is projected onto the XOZ plane, the normal vector is expressed as Therefore, the parameter to be sought is only one a. According to the derivation process of least square method for solving overdetermined equations, the original equation can be converted into The matrix form is:

[0055]

[0056] According to Cramer's rule, we can find the unknown variable a. According to the necessary and sufficient conditions of the least squares solution of overdetermined equations, we know that the above system of equations must have a unique solution. Therefore, the solution can be simplified to:

[0057]

[0058] At this point, the normal vector projection to the XOZ plane is calculated

[0059] S5. Calculate the included angle of the normal vector of the plane fitted by a set of three-dimensional point cloud data to determine the position of the snow part scanned by the radar.

[0060] The time interval of each set of point cloud data is set to 0.35 seconds. At such a small time interval, the fitting plane along the Y axis has basically no change. Therefore, the projection of the normal vector onto the coordinate plane XOZ is taken, and the two-dimensional vector The angle between the fitting plane and the positive direction of the X-axis is obtained by calculating the supplementary angle. Combined with the correction of the cantilever pitch angle and the angle between the radar and the snow ground plane, the real-time fitting angle α between the snow surface and the horizontal plane is calculated.

[0061]

[0062] This method uses the snow repose angle as the judgment threshold. If the radar scanning surface is in a snow pile, the cutting surface passing through the snow pile must be greater than its own repose angle. Therefore, when the fitting plane angle α is less than the snow repose angle, it is judged that the radar scanning surface is outside the snow. At this time, the distance between the radar and the snow shovel can predict the position where the snow shovel will shovel the snow when the vehicle moves forward.

[0063] This embodiment can calculate 3D point cloud data from the radar's 2D scanning surface based on the radar's temporal position changes. Directly analyzing the spliced ​​3D point cloud data not only increases the amount of information, but also serves as a basis for outlier removal, overcoming the shortcomings of millimeter-wave radar, such as the high artifact count and low resolution. This also avoids issues caused by human visual errors during manual snow shoveling.

[0064] In addition, it should be noted that: an embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the method described in the above embodiment is executed.

[0065] In addition, it should be noted that an embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor loads and executes the computer program, it executes the method described in the above embodiment.

[0066] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A method for detecting snow edge during snow shoveling, characterized in that: The method comprises: The radar is used to detect the 2D point cloud data of the longitudinal section of the snow near the snow shovel side; A radar local coordinate system is established to describe the radar scanning plane of each frame. The two-dimensional plane discrete points scanned by the radar are expanded into three-dimensional point cloud data based on the time series and the snow shovel swing angular velocity. Stitching radar scan point cloud data and preprocessing the cloud data; Using the statistical filter in the 3D point cloud data, the 3D point cloud data obtained by splicing several frames of scanned plane point clouds is selected to remove abnormal points, thus obtaining the 3D point cloud data after noise removal. Solve the plane normal vector of a set of 3D point cloud data after removing abnormal points; Calculate the included angle of the plane normal vector fitted to a set of 3D point cloud data to determine the position of the snow part scanned by the radar.

2. The method for detecting the edge of snow during snow shoveling according to claim 1, wherein: The radar is installed on the left and right sides of the snow plow of the snowplow. The radar is a millimeter wave radar. The scanning field of view of the radar is within the sensing distance of 0-70 meters within a ±60 degree sector area and the beam vertical direction field of view angle is ±10 degree sector area.

3. The method for detecting snow edge during snow shoveling according to claim 1, wherein: Establishing the radar local coordinate system that describes the radar scanning plane for each frame includes: The ray perpendicular to the radar plane from the center of the radar to the far end is the horizontal axis of the coordinate system, and the vertical ray drawn from the center of the radar in the radar scanning plane is the vertical axis of the radar local coordinate system; Establish a three-dimensional coordinate system for the radar rotation process, and correct the moving distance of the scanning plane in the coordinate system at each moment according to the different angular velocities during movement; Before snow plowing begins, the center of the radar where the snowplow is about to start moving forward is taken as the origin (0,0,0). The X and Z axes are the horizontal and vertical axes of the radar's local coordinate system at this time, respectively. The Y axis is the ray perpendicular to the XOZ plane from the origin, which is used to represent the relative distance between each corrected scan plane and the first scan plane.

4. The method for detecting snow edge during snow shoveling according to claim 1, wherein: After the cloud data is preprocessed, artifact points are removed based on the quality information obtained by the radar.

5. The method for detecting snow edge during snow shoveling according to claim 1, wherein: The preprocessing of cloud data includes: By combining the band-limited filtering concept used in analog signals with the actual working conditions of snowplows, the point cloud data is restricted to a range within a depth of 15 meters in a local coordinate system with the radar center as the far point. This creates a plane field of view of an isosceles triangle with a top angle of 120 degrees, a base angle of 30 degrees, a waist length of 30 meters, and a base length of 52 meters. The radar false alarm probability, Doppler ambiguity, cluster uncertainty and radar cross section are the key factors affecting the performance of radar signal processing and target analysis; Conduct in-depth analysis of the key factors and apply reasonable comprehensive utilization strategies; After integrating the angular velocity within each frame interval of the radar, the arc length between the two frames is obtained by combining the gyration radius, and the plane rectangular coordinate difference between the beginning and end of this arc is calculated as the correction value of the coordinate point of the next frame data. The data is stitched together to obtain the 3D point cloud data for this time interval. Based on the principle of statistical filtering, the 3D Euclidean distances from all points in a set of data to other points are calculated. The adjacent points are selected, and the average distance from each point to the adjacent points is calculated. The global average distance standard deviation of the neighboring points is used as the threshold, the average distance of the neighboring points minus the global average distance is compared with the set threshold, and the points exceeding the threshold are defined as outliers and screened out.

6. The method for detecting snow edge during snow shoveling according to claim 1, wherein: Solving the plane normal vector of a set of 3D point cloud data after removing outliers includes: Assume that the fitting plane equation of this set of point clouds is ax+by+cz=d, (x, y, z) is the 3D point cloud data obtained by scanning, and (a, b, c) is the normal vector of the fitting plane; First, to facilitate program operations, its matrix representation Right now Solving the unknowns (a, b, c) is the normal vector of the fitting plane At this time, the problem of finding the normal vector of the fitted plane is transformed into the problem of finding the optimal solution of an overdetermined set of equations, and the unknowns solved by the least squares method are selected as the parameters of the normal vector of the fitted plane.

7. The method for detecting snow edge during snow shoveling according to claim 1, wherein: Determining the location of the snow covered area scanned by the radar includes: Take the projection of the normal vector onto the coordinate plane XOZ and take the two-dimensional vector The angle between the fitting plane and the positive direction of the X-axis is obtained by calculating the supplementary angle; Combined with the cantilever pitch angle and the correction of the angle between the radar and the stockpile plane, the real-time fitting angle α between the reclaiming surface and the horizontal plane is calculated. The snow repose angle is used as the judgment threshold. When the fitting plane angle α is less than the snow repose angle, it is judged that the radar scanning surface is outside the snow. At this time, the position of the snow shovel when the vehicle moves forward can be predicted based on the distance between the radar and the snow shovel.

8. A computer-readable storage medium storing a computer program, wherein: When the computer program is loaded and executed by a processor, the method according to any one of claims 1 to 7 is adopted.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor loads and executes the computer program, the method according to any one of claims 1 to 7 is adopted.

Images