Method for estimating running speed of surface mine vehicle based on image movement characteristics

Abstract

The invention discloses a surface mine vehicle running speed estimation method based on image movement characteristics, which comprises the following steps: acquiring an image based on sensing equipment, and preprocessing the image; extracting a target vehicle in the preprocessed image; based on the characteristics of the distance from the center of mass to the boundary contour of the target vehicle, performing movement characteristic extraction on the target vehicle to obtain vector characteristics describing vehicle movement characteristics; and on the basis of a Markov chain and a deep network model DBN, predicting the future vehicle speed of the target vehicle under different conditions to obtain a final estimation result. According to the method, the moving direction and different driving states of the target vehicle can be well distinguished through the extracted vehicle motion features, and data support is provided for collision early warning of the front vehicle in the selected area unmanned driving system.

Description

Technical Field

[0001] The invention relates to the technical field of supervision information fusion of open-pit mines, and in particular to a method for estimating the driving speed of open-pit mine vehicles based on image movement features. Background Art

[0002] The analysis and identification of the changing characteristics of the vehicle ahead has become one of the most effective means of vehicle collision monitoring due to its advantage of directly reflecting the road dynamics. In recent years, in order to improve the real-time and accuracy of collision warning, the vehicle speed estimation technology has adopted a multi-step testing strategy, combined with a rolling optimization algorithm and a feedback correction mechanism. The application of these innovative ideas has enabled the vehicle speed prediction to achieve good real-time results in complex and changeable traffic environments, effectively reducing the risk of collision. However, these advanced estimation methods rely heavily on the precise positioning information provided by the GPS system or prior operating data, which limits their application in specific environments. For example, it is difficult for non-road vehicles in open-pit mine unmanned driving systems that are not equipped with positioning systems and advanced sensing equipment to directly adopt these technologies.

[0003] The shortcomings of existing technologies are that they are often based on specific hardware support (such as GPS) or preset conditions (such as prior working condition information), which limits the popularity and adaptability of the technology. Especially in complex non-road environments such as open-pit mines, due to the limited configuration of vehicle equipment, the traditional vehicle speed monitoring method that relies on image geometric features, although it can reflect the vehicle's motion state to a certain extent, lacks an accurate and stable correspondence with different vehicle speed states, resulting in insufficient accuracy in vehicle speed estimation and collision warning, and it is difficult to meet the high standards of safety and reliability of open-pit mine unmanned driving systems. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art. To achieve the above purpose, a method for estimating the driving speed of open-pit mine vehicles based on image movement characteristics is adopted to solve the problems raised in the above background technology.

[0005] A method for estimating the driving speed of open-pit mine vehicles based on image movement characteristics comprises the following steps:

[0006] Step S1, acquiring an image based on a sensing device and performing image preprocessing;

[0007] Step S2, extracting the target vehicle from the preprocessed image;

[0008] Step S3, based on the distance feature from the center of mass of the target vehicle to the boundary contour, extract the movement feature of the target vehicle to obtain a vector feature describing the vehicle movement feature;

[0009] Step S4: Based on the Markov chain and the deep network model DBN, the future speed of the target vehicle is predicted under different circumstances to obtain a final estimation result.

[0010] As a further solution of the present invention: the specific steps in step S1 include:

[0011] Step S11, acquiring an image through a vehicle-mounted camera;

[0012] Step S12, by measuring the angle between the image coordinate system and the IMU coordinate system, and then calculating the representation of the actual camera direction in the image coordinate system according to the angle;

[0013] Step S13, according to the position of the target vehicle in the IMU coordinate system in the image, select a moving unit direction vector of the target vehicle with the shooting position as the center;

[0014] Step S14, real-time acquisition of the target vehicle's moving direction, select a neighborhood with a radius of 1 laser radar scanning ROI. After obtaining the unidirectional vector in the IMU coordinate system, the conversion is performed according to the conversion formula, which is:

[0015]

[0016] In the formula, e A is the moving unit direction vector of the vehicle in the IMU coordinate system at the current moment, e I To transform to the unit direction vector in the image coordinate system, The conversion rotation matrix is:

[0017]

[0018] Among them, θ is obtained by calibration.

[0019] As a further solution of the present invention: the specific steps in step S2 include:

[0020] Step S21, extracting the target vehicle motion features in the image moving area, and extracting the ROI area of ​​the vehicle;

[0021] Step S22, after extracting the ROI area of ​​the vehicle, using the connected component analysis method to determine the target vehicle; wherein the connected component with the largest area and located at the center of mass of the vehicle is the connected component of the target vehicle;

[0022] Step S23, setting different scale thresholds for the area determined to have the target vehicle to perform binarization processing, and separating the target vehicle area from the image;

[0023] Step S24: By comparing the binary images under different thresholds, the most appropriate threshold combination is selected to ensure accurate extraction of the target vehicle, thereby obtaining a final target vehicle image.

[0024] As a further solution of the present invention: the specific steps in step S3 include:

[0025] Step S31, extracting the moving area and ROI area of ​​the target vehicle;

[0026] Step S32, using the intermediate scale threshold to determine the size of the ROI area, and then binarizing to obtain the ROI area;

[0027] Step S33, tracking the ROI region boundary based on the 8-connected neighborhood, extracting the ROI region boundary pixel points, and then expanding the distance from the ROI region centroid to the boundary;

[0028] Step S34: Take the intersection of the ROI area moving direction line and the ROI area boundary in multiple images as the starting point, and expand the distance information from the ROI area centroid to the boundary contour in a clockwise direction to obtain a vector feature D={d 1 ,d 2 ,d 3 ,…,d 36}, d i The calculation formula is:

[0029]

[0030] Among them, (x i ,y i ) is the coordinate of the pixel point of the boundary contour at the corresponding angle, (x 0 ,y 0 ) is the coordinate of the center of mass of the target vehicle.

[0031] As a further solution of the present invention: the specific steps in step S4 include:

[0032] Step S41, judging the variation range of the vector feature obtained and estimating the driving speed;

[0033] Step S42: When d i When the change is small, it means that the current target vehicle is running under stable conditions, and the Markov chain prediction method is used to predict the future vehicle speed;

[0034] Step S43: When d i If the change is large, it means that the current target vehicle is driving in a fast-changing condition. The deep network model DBN is used to predict the future speed of the target vehicle.

[0035] Where D = {d1 ,d 2 ,d 3 ,…,d 36} as the input to define the DBN network model, the output N of the DBN network model out The predicted vehicle speed for a period of time in the future.

[0036] Compared with the prior art, the present invention has the following technical effects:

[0037] By adopting the above technical solution,. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings:

[0039] Figure 1 This is a schematic diagram of the steps of the driving speed estimation method according to the embodiment disclosed in the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Please refer to Figure 1 In an embodiment of the present invention, a method for estimating the driving speed of an open-pit mine vehicle based on image movement features comprises the following steps:

[0042] Step S1: Acquire an image based on a sensor device and perform image preprocessing. The specific steps include:

[0043] Step S11, acquiring an image through a vehicle-mounted camera;

[0044] The basis of autonomous driving or assisted driving systems, the on-board camera acts as the "eyes" that can capture the road and environment in front of the vehicle in real time. The high-definition camera can capture rich details and provide a reliable data source for subsequent image processing and analysis;

[0045] First, determine the center of mass and moving direction of the target vehicle from the image captured by the on-board camera. Since the image is acquired by the on-board camera, the image coordinate system is constantly moving with the vehicle. The position of the target vehicle in the image coordinate system can be obtained by taking multiple measurements of the center position of the target vehicle and averaging them.

[0046] Step S12, by measuring the angle between the image coordinate system and the IMU coordinate system, and then calculating the representation of the actual camera direction in the image coordinate system according to the angle;

[0047] The IMU coordinate system provides the vehicle's attitude information, including pitch, yaw, and roll angles. By accurately measuring the angular relationship between the image coordinate system and the IMU coordinate system, it is possible to calculate how the camera's orientation in the real world is projected onto the image plane. This step is key to understanding the relationship between the image content and the actual road conditions, and helps the system accurately determine the vehicle's direction of travel and target position;

[0048] Step S13, according to the position of the target vehicle in the IMU coordinate system in the image, select a moving unit direction vector of the target vehicle with the shooting position as the center;

[0049] Step S14, real-time acquisition of the target vehicle's moving direction, select a neighborhood with a radius of 1 laser radar scanning ROI. After obtaining the unidirectional vector in the IMU coordinate system, the conversion is performed according to the conversion formula, which is:

[0050]

[0051] In the formula, e A is the moving unit direction vector of the vehicle in the IMU coordinate system at the current moment, e I To transform to the unit direction vector in the image coordinate system, The conversion rotation matrix is:

[0052]

[0053] Among them, θ is obtained by calibration.

[0054] After determining the position of the target vehicle in the image, combined with the vehicle's own posture information provided by the IMU, the target vehicle's moving direction relative to the vehicle itself can be calculated. This unit direction vector is an important basis for guiding the vehicle to perform path planning and obstacle avoidance operations. Through this step, the system can more accurately understand the dynamics of the target vehicle, thereby making more intelligent and safe driving decisions.

[0055] Step S2: extracting the target vehicle from the preprocessed image. The specific steps include:

[0056] Step S21, extracting the target vehicle motion features in the image moving area, and extracting the ROI area of ​​the vehicle;

[0057] Step S22, after extracting the ROI area of ​​the vehicle, using the connected component analysis method to determine the target vehicle; wherein the connected component with the largest area and located at the center of mass of the vehicle is the connected component of the target vehicle;

[0058] Step S23, setting different scale thresholds for the area determined to have the target vehicle to perform binarization processing, and separating the target vehicle area from the image;

[0059] Step S24: By comparing the binary images under different thresholds, the most appropriate threshold combination is selected to ensure accurate extraction of the target vehicle, thereby obtaining a final target vehicle image.

[0060] In this embodiment, in order to extract the motion characteristics of the target vehicle in the moving area, the vehicle ROI is extracted. The connected component analysis method is used for judgment. Among them, the target vehicle connected component has the largest area and is located at the center of mass of the vehicle. If there are any remaining connected components, the area size is used to determine whether it is outside the area. When the threshold is set to be small, the area close to the center of the ROI has a large grayscale value, and this area will be connected to the ROI area after binarization; when the threshold is set to be large, the area far away from the center of the ROI has a small grayscale value, and after binarization, it will become a black background with a grayscale value of 0, leaving only the ROI component. Therefore, a larger threshold is required for areas close to the ROI area, and a smaller threshold is required for areas far away from the ROI area. To address this problem, three scale thresholds of large, medium and small can be set for binary image processing, so as to accurately extract moving vehicles from the image.

[0061] Step S3: based on the distance feature from the center of mass of the target vehicle to the boundary contour, extract the moving feature of the target vehicle to obtain the vector feature describing the vehicle motion feature, and the specific steps include:

[0062] In this embodiment, during the estimation of the driving speed of open-pit mine vehicles, the target vehicle is in a continuous moving state, and its movement mainly includes moving along the scanning road and bumping up and down. The driving speed estimation error is also formed in the process. The movement of the target vehicle will cause the size and shape of the ROI in the image to change. Therefore, it is necessary to describe the movement of the target vehicle consistently through features that can fully describe the size and shape of the target vehicle. To address this problem, the distance feature from the target vehicle's center of mass to the target vehicle's boundary contour is used to effectively describe the moving target vehicle.

[0063] Step S31, extracting the moving area and ROI area of ​​the target vehicle;

[0064] Step S32: To make the size of the ROI as close as possible to the actual vehicle size, the size of the ROI region is determined by using the intermediate scale threshold, and then the ROI region is obtained by binarization;

[0065] Among them, the middle scale threshold is set to 0.5;

[0066] Step S33, tracking the ROI region boundary based on the 8-connected neighborhood, extracting the ROI region boundary pixel points, and then expanding the distance from the ROI region centroid to the boundary;

[0067] Step S34: To better compare the moving ROIs, the intersection of the moving direction lines of the ROI regions in multiple images and the ROI region boundaries is taken as the starting point, and the distance information from the ROI region centroid to the boundary contour is expanded in a clockwise direction to obtain a vector feature D with a fixed dimension of 36 = {d 1 ,d 2 ,d 3 ,…,d 36}, d i The calculation formula is:

[0068]

[0069] Among them, (x i ,y i ) is the coordinate of the pixel point of the boundary contour at the corresponding angle, (x 0 ,y 0 ) is the coordinate of the center of mass of the target vehicle.

[0070] Step S4: Based on the Markov chain and the deep network model DBN, the future speed of the target vehicle is predicted under different circumstances to obtain the final estimation result. The specific steps include:

[0071] Step S41, judging the variation range of the vector feature obtained and estimating the driving speed;

[0072] Step S42: When d i When the change is small, it means that the current target vehicle is running under stable conditions, and the Markov chain prediction method is used to predict the future vehicle speed;

[0073] Among them, the Markov chain prediction method can better capture this trend and make a relatively accurate prediction of the future vehicle speed. This method has the advantages of simple calculation and high efficiency, and is suitable for vehicle speed prediction under stable working conditions.

[0074] Step S43: When d i If the change is large, it means that the current target vehicle is driving in a fast-changing condition. The deep network model DBN is used to predict the future speed of the target vehicle.

[0075] Where D = {d 1 ,d 2 ,d 3 ,…,d 36} as the input to define the DBN network model, the output N of the DBN network model outThe predicted vehicle speed for a period of time in the future.

[0076] DBN is a neural network with multiple hidden layers that can capture complex features and nonlinear relationships in data. Under fast-changing conditions, the vehicle's driving speed changes dramatically, and a more sophisticated prediction model is needed to capture this change.

[0077] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents, and all should be included within the scope of protection of the present invention.

Claims

1. A method for estimating vehicle speed in an open-pit mine based on image motion features, characterized in that: The following steps are involved: Step S1, acquiring an image based on a sensing device and performing image preprocessing; Step S2, extracting the target vehicle from the preprocessed image; Step S3, based on the distance feature from the center of mass of the target vehicle to the boundary contour, extract the movement feature of the target vehicle to obtain a vector feature describing the vehicle movement feature; Step S4: Based on the Markov chain and the deep network model DBN, the future speed of the target vehicle is predicted under different circumstances to obtain a final estimation result.

2. The method for estimating the driving speed of open-pit mine vehicles based on image movement characteristics according to claim 1 is characterized in that: The specific steps in step S1 include: Step S11, acquiring an image through a vehicle-mounted camera; Step S12, by measuring the angle between the image coordinate system and the IMU coordinate system, and then calculating the representation of the actual camera direction in the image coordinate system according to the angle; Step S13, according to the position of the target vehicle in the IMU coordinate system in the image, select a moving unit direction vector of the target vehicle with the shooting position as the center; Step S14, real-time acquisition of the target vehicle's moving direction, select a neighborhood with a radius of 1 laser radar scanning ROI. After obtaining the unidirectional vector in the IMU coordinate system, the conversion is performed according to the conversion formula, which is: In the formula, e A is the moving unit direction vector of the vehicle in the IMU coordinate system at the current moment, e I To transform to the unit direction vector in the image coordinate system, The conversion rotation matrix is: Among them, θ is obtained by calibration.

3. The method for estimating the driving speed of open-pit mine vehicles based on image movement characteristics according to claim 1 is characterized in that: The specific steps in step S2 include: Step S21, extracting the target vehicle motion features in the image moving area, and extracting the ROI area of ​​the vehicle; Step S22, after extracting the ROI area of ​​the vehicle, using the connected component analysis method to determine the target vehicle; wherein the connected component with the largest area and located at the center of mass of the vehicle is the connected component of the target vehicle; Step S23, setting different scale thresholds for the area determined to have the target vehicle to perform binarization processing, and separating the target vehicle area from the image; Step S24: By comparing the binary images under different thresholds, the most appropriate threshold combination is selected to ensure accurate extraction of the target vehicle, thereby obtaining a final target vehicle image.

4. The method for estimating the driving speed of open-pit mine vehicles based on image movement characteristics according to claim 1 is characterized in that: The specific steps in step S3 include: Step S31, extracting the moving area and ROI area of ​​the target vehicle; Step S32, using the intermediate scale threshold to determine the size of the ROI area, and then binarizing to obtain the ROI area; Step S33, tracking the ROI region boundary based on the 8-connected neighborhood, extracting the ROI region boundary pixel points, and then expanding the distance from the ROI region centroid to the boundary; Step S34: Take the intersection of the ROI area moving direction line and the ROI area boundary in multiple images as the starting point, and expand the distance information from the ROI area centroid to the boundary contour in a clockwise direction to obtain a vector feature D with a fixed dimension of 36 = {d1, d2, d3, ..., d 36 }, d i The calculation formula is: Among them, (x i ,y i ) is the coordinate of the pixel point of the boundary contour at the corresponding angle, and (x0, y0) is the coordinate of the center of mass of the target vehicle.

5. The method for estimating vehicle speed in an open-pit mine based on image motion characteristics according to claim 1, characterized in that: The specific steps in step S4 include: Step S41, judging the variation range of the vector feature obtained and estimating the driving speed; Step S42: When d i When the change is small, it means that the current target vehicle is running under stable conditions, and the Markov chain prediction method is used to predict the future vehicle speed; Step S43: When d i If the change is large, it means that the current target vehicle is driving in a fast-changing condition. The deep network model DBN is used to predict the future speed of the target vehicle. Where D = {d1, d2, d3, ..., d 36 } as the input to define the DBN network model, the output N of the DBN network model out The predicted vehicle speed for a period of time in the future.

Images