Method for detecting size of obstacle in front of unmanned electric locomotive based on YOLOv8 and related device

Abstract

The invention belongs to the technical field of automatic driving, and discloses a YOLOv8-based unmanned electric locomotive front obstacle size detection method and a related device, and the method comprises the steps: carrying out the preprocessing of unmanned electric locomotive environment image data collected by a binocular camera, and obtaining a preprocessed image; performing obstacle category detection on the preprocessed image by using a trained YOLOv8 target detection algorithm, and determining whether there is an obstacle in the preprocessed image; when an obstacle exists in the preprocessed image, matching the left view and the right view by using an SGBM stereo matching algorithm to obtain obstacle distance information; and calculating the height and width of the obstacle according to the obstacle distance information. According to the invention, whether the obstacle exists in front of the unmanned electric locomotive or not and the type of the obstacle can be accurately detected, the height and width information of the obstacle in front of the unmanned electric locomotive can be accurately detected, a decision basis is provided for obstacle classification avoidance, and the operation efficiency of the unmanned electric locomotive is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and in particular relates to a method for detecting the size of an obstacle in front of an unmanned electric vehicle based on YOLOv8 and a related device. Background Art

[0002] The perception system of unmanned electric locomotives captures the surrounding environment information through sensors, processes and analyzes the data in combination with perception computing algorithms, and extracts obstacle information. There are various obstacles on the road of unmanned electric locomotives. Large obstacles will hinder the normal driving of the vehicle. Large obstacles that are lower than the chassis of the vehicle will not hinder the normal driving of the vehicle, but the obstacle avoidance strategy will still judge the danger level of such obstacles as high danger, thereby triggering the obstacle avoidance rules, which has a great impact on the vehicle's operating efficiency. Specifically, the safe driving of unmanned electric locomotives is closely related to factors such as the location of obstacles on the road, the size of obstacles, and the accuracy of perception algorithms. The performance of obstacle perception algorithms has a huge impact on the driving of unmanned electric locomotives. Efficient and accurate perception algorithms can maximize the vehicle's operating efficiency under the condition of accurately perceiving obstacle information. The current obstacle perception algorithm mainly focuses on detecting whether there are obstacles and the types of obstacles during the driving of unmanned electric locomotives, but fails to accurately detect the size information such as the height and width of the obstacles. When a low obstacle that cannot reach the chassis of the vehicle is detected, the vehicle will still trigger the obstacle avoidance mechanism, and the operating efficiency of the unmanned electric locomotive is limited. Summary of the invention

[0003] In order to solve the problems existing in the prior art, the purpose of the present invention is to propose a method and related device for detecting the size of obstacles in front of an unmanned electric locomotive based on YOLOv8. The present invention can not only accurately detect whether there is an obstacle in front of the unmanned electric locomotive and the type of obstacle, but also accurately detect the height and width information of the obstacle in front of the unmanned electric locomotive, which provides a decision-making basis for obstacle classification and avoidance, and improves the operating efficiency of the unmanned electric locomotive.

[0004] To achieve the above purpose, the technical solution adopted by the present invention is as follows: A method for detecting the size of obstacles in front of an unmanned electric vehicle based on YOLOv8 includes the following steps: Preprocessing the unmanned electric locomotive environment image data collected by the binocular camera to obtain a preprocessed image; Use the trained YOLOv8 target detection algorithm to detect obstacle categories in the preprocessed image to determine whether there are obstacles in the preprocessed image; When there are obstacles in the preprocessed image, the SGBM stereo matching algorithm is used to match the left view and the right view to obtain the obstacle distance information; The obstacle height and width are calculated based on the obstacle distance information.

[0005] Preferably, the unmanned electric locomotive environment image data collected by the binocular camera is preprocessed to obtain a preprocessed image, including: The unmanned electric locomotive environment image data is subjected to image grayscale conversion, Zero_DCE image enhancement and histogram equalization operations in sequence to obtain a preprocessed image.

[0006] Preferably, the training process of the YOLOv8 target detection algorithm includes: Collect image data of real tunnels and simulated tunnels in underground mines to obtain image data sets; Amplifying the image data set to obtain an amplified data set; Preprocessing the amplified data set to obtain a preprocessed image; The preprocessed images are used to train the YOLOv8 target detection algorithm. After the training is completed, the optimal weight is set for the YOLOv8 target detection algorithm to obtain the trained YOLOv8 target detection algorithm.

[0007] Preferably, the image data set is enlarged by flipping, cropping and blocking operations to obtain an enlarged data set.

[0008] Preferably, the left view and the right view are matched using the SGBM stereo matching algorithm to obtain obstacle distance information, including: Set up a commanding position on an obstacle The coordinates of , commanding heights Pixel coordinates projected onto the left camera's viewing axis for , the corresponding pixel coordinates on the right image plane for ; The binocular camera disparity is obtained by the SGBM stereo matching algorithm D as follows:

[0009] Using binocular geometry and binocular camera parallax, we can calculate the commanding heights. The distance between the projection on the left camera's boresight and the projection on the right camera's boresight , get the high ground Coordinates as follows:

[0010]

[0011]

[0012] Where: Represents the pixel horizontal coordinate of the object projection on the left camera image; Represents the pixel horizontal coordinate of the object projection on the right camera image; represents the focal length of the camera, is the distance between the optical centers of the cameras; For the commanding heights Parallax.

[0013] Preferably, calculating the obstacle height according to the obstacle distance information includes: Based on the commanding heights of obstacles Coordinates , calculate the commanding height Height relative to the ground :

[0014] in, It represents the angle between the optical axis and the horizontal plane, and H represents the height of the binocular camera relative to the ground.

[0015] Preferably, calculating the obstacle width according to the obstacle distance information includes: Use the left cut surface of the left view of the binocular camera and the initial point of the obstacle , and the right section of the right view of the binocular camera and the initial point of the obstacle , calculate the obstacle width as follows: .

[0016] The present invention also provides a system for detecting the size of obstacles in front of an unmanned electric vehicle based on YOLOv8, which is used to implement the above-mentioned method for detecting the size of obstacles in front of an unmanned electric vehicle based on YOLOv8 of the present invention. The system comprises: Preprocessing unit: used to preprocess the unmanned electric locomotive environment image data collected by the binocular camera to obtain a preprocessed image; Recognition unit: used to detect obstacle categories in pre-processed images using the trained YOLOv8 target detection algorithm to determine whether there are obstacles in the pre-processed images; The first calculation unit is used for matching the left view and the right view by using the SGBM stereo matching algorithm to obtain obstacle distance information when there is an obstacle in the preprocessed image; The second calculation unit is used to calculate the height and width of the obstacle according to the obstacle distance information.

[0017] The present invention also provides an electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for detecting the size of obstacles in front of an unmanned electric vehicle based on YOLOv8 as described above.

[0018] The present invention also provides a storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for detecting the size of an obstacle in front of an unmanned electric vehicle based on YOLOv8 as described above is implemented in the present invention.

[0019] The present invention has the following beneficial effects: The present invention pre-processes the image data of the unmanned electric locomotive environment collected by a binocular camera, and then uses the trained YOLOv8 target detection algorithm to perform real-time obstacle category detection on the processed data. Finally, on the basis of detecting the obstacle category, the SGBM stereo matching algorithm is used to match the left view and the right view and obtain the obstacle distance information, and the obstacle height information and width information are calculated based on the obtained obstacle distance information. The technical solution of the present invention can obtain the height information of the measured obstacle under the premise of ensuring the accurate detection of the obstacle category in front of the unmanned electric locomotive, and solves the problem of low operating efficiency caused by not considering the height information in the existing unmanned electric locomotive obstacle avoidance rules. The present invention provides a basis for the classification and avoidance of obstacles in front of the unmanned electric locomotive, improves the working efficiency of the unmanned electric locomotive, and can be popularized and applied in the field of unmanned transportation in mines, with great market development potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Flow chart of a method for detecting the size of an obstacle in front of an unmanned electric vehicle based on YOLOv8 in an embodiment of the present invention; Figure 2 This is a geometric principle diagram of binocular stereo matching in an embodiment of the present invention; Figure 3 It is the commanding height of the obstacle in the embodiment of the present invention. Schematic diagram of relative ground height; Figure 4 Schematic diagram of the steps for calculating the left and right cut planes of the left and right viewing angles of the binocular camera in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention is clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Example In this embodiment, the calibrated and corrected binocular camera is fixed on the unmanned electric locomotive at a height of 1.5 meters from the ground, and then the image data is transmitted to the image processing system during driving. The image data collected by the binocular camera is preprocessed by using image grayscale, Zero_DCE image enhancement, and histogram equalization methods in sequence to obtain a preprocessed image. Then, the trained YOLOv8 target detection algorithm is used to perform real-time obstacle category detection on the obtained preprocessed image. Then, based on the detected obstacle category, the SGBM stereo matching algorithm is used to match the left view and the right view and obtain obstacle distance information, and then the obstacle height information is calculated based on the obtained obstacle distance information.

[0023] Specifically, Figure 1 As shown, the obstacle size detection method in front of the unmanned electric vehicle based on YOLOv8 in this embodiment is to add the obstacle height measurement function to the existing binocular stereo matching technology, so as to obtain the size information of the obstacle, which mainly includes the following steps: Step 1: To solve the problem of unclear images caused by low illumination in underground tunnels, a binocular camera is used to perceive the environmental data of the unmanned electric locomotive and transmit the data to the image processing system for data preprocessing. The specific implementation method includes: after calibrating the binocular camera using a black and white chess board, the binocular camera is fixed at a height of 1.5 meters above the ground of the unmanned electric locomotive. During the driving of the unmanned electric locomotive, the collected image data is transmitted to the image processing system. The image processing system sequentially uses image grayscale, Zero_DCE image enhancement and histogram equalization operations to preprocess the image data to obtain a preprocessed image.

[0024] Step 2: Use the YOLOv8 target detection algorithm to detect obstacle categories on the image data (i.e., preprocessed image) obtained by preprocessing in step 1. The specific implementation method includes: step ①, collect the image data of the real tunnel and simulated tunnel of the underground mine, and use the operations of flipping, cropping and occluding to amplify the data set to obtain the amplified data set; the amplified data set is sequentially subjected to the operations of image grayscale, Zero_DCE image enhancement, and histogram equalization to obtain the preprocessed image. Step ②, input the preprocessed data in step ① into the YOLOv8 target detection algorithm for model training. After the training is completed, set the optimal weight for the YOLOv8 target detection algorithm to obtain the trained YOLOv8 target detection algorithm. Step ③, input the real-time image data taken by the binocular camera processed in step 1 into the trained YOLOv8 target detection algorithm for obstacle category detection to determine whether there are obstacles in the preprocessed image.

[0025] Step 3: In order to improve the safety efficiency of underground electric locomotives and their obstacle avoidance effect, when there are obstacles in the preprocessed image, the SGBM stereo matching algorithm is used to match the left view and the right view and obtain the obstacle distance information. The specific implementation method includes: Step ①, set the commanding height on the obstacle The coordinates of , the commanding heights The pixel coordinates projected on the visual axis of the left camera are , the corresponding pixel coordinates on the right image plane are Step ②, according to the model assumption, since the distance between the left and right cameras of the binocular camera is fixed, the distance between the optical centers of the cameras can be expressed as , the binocular camera disparity D can be obtained from the above stereo matching algorithm as follows:

[0026] Step ③, using binocular geometry principle (such as Figure 2 As shown in the figure) and obtain the binocular camera parallax, calculate the commanding height The distance between the two projections on the left camera's viewing axis and the right camera's viewing axis , that is, the parallax of the commanding height , according to the similarity of triangles, we can get the commanding heights coordinate as follows:

[0027] Where: Represents the pixel horizontal coordinate of the object projection on the left camera image; Represents the pixel horizontal coordinate of the object projection on the right camera image; Indicates the focal length of the camera.

[0028] Step 4: Calculate the height and width of the obstacle using the obstacle distance information. The specific implementation method includes: Step ①, the commanding height of the obstacle is known World coordinates , calculate the commanding height Height relative to the ground as follows:

[0029] In the formula, represents the angle between the optical axis and the horizontal plane, and H represents the height of the binocular camera relative to the ground. Step ②, when the height of the obstacle relative to the ground is calculated, Figure 4 As shown, the left cut surface of the left view of the binocular camera and the initial point of the obstacle , and the right section of the right view of the binocular camera and the initial point of the obstacle , calculate the obstacle width for:

[0030] The above scheme of the present invention is summarized as follows: The Zero_DCE image enhancement algorithm is: through the technology of low-light image enhancement, the visual quality of the image is improved without the need for additional labeled data, the image enhancement features are automatically learned using a deep learning model, and the unlabeled data is fully utilized (i.e., no additional high-quality images are required as training sets) to achieve enhancement. The YOLOv8 target detection algorithm is: the entire image is processed through a single neural network, and the bounding box and category probability are directly predicted, thereby achieving rapid target positioning and recognition. Binocular stereo matching algorithm: a technology for recovering the depth of a three-dimensional scene from images acquired by two cameras. By calculating the matching relationship between pixels of the same scene at different perspectives, the algorithm can estimate the depth information of each point in the scene.

[0031] The present invention also provides a YOLOv8-based unmanned electric vehicle obstacle size detection system for implementing the above detection method, the system comprising: Preprocessing unit: used to preprocess the unmanned electric locomotive environment image data collected by the binocular camera to obtain a preprocessed image; Recognition unit: used to detect obstacle categories in pre-processed images using the trained YOLOv8 target detection algorithm to determine whether there are obstacles in the pre-processed images; The first calculation unit is used for matching the left view and the right view by using the SGBM stereo matching algorithm to obtain obstacle distance information when there is an obstacle in the preprocessed image; The second calculation unit is used to calculate the height and width of the obstacle according to the obstacle distance information.

[0032] The embodiments of the present invention also provide corresponding electronic devices and computer-readable storage media for implementing the solutions provided by the embodiments of the present invention.

[0033] The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes the method for detecting the size of obstacles in front of an unmanned electric vehicle based on YOLOv8 as described in any embodiment of the present application.

[0034] The storage medium stores a computer program, wherein the computer program, when executed by the processor, implements the method for detecting the size of an obstacle in front of an unmanned electric vehicle based on YOLOv8 as described in any embodiment of the present application.

[0035] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0036] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting the size of obstacles in front of an unmanned electric vehicle based on YOLOv8, characterized in that: The process includes the following: Preprocessing the unmanned electric locomotive environment image data collected by the binocular camera to obtain a preprocessed image; Use the trained YOLOv8 target detection algorithm to detect obstacle categories in the preprocessed image to determine whether there are obstacles in the preprocessed image; When there are obstacles in the preprocessed image, the SGBM stereo matching algorithm is used to match the left view and the right view to obtain the obstacle distance information; The obstacle height and width are calculated based on the obstacle distance information.

2. According to the YOLOv8-based method for detecting the size of obstacles in front of an unmanned electric vehicle according to claim 1, it is characterized in that: The unmanned electric locomotive environment image data collected by the binocular camera is preprocessed to obtain a preprocessed image, including: The unmanned electric locomotive environment image data is subjected to image grayscale conversion, Zero_DCE image enhancement and histogram equalization operations in sequence to obtain a preprocessed image.

3. The method for detecting the size of obstacles in front of an unmanned electric vehicle based on YOLOv8 according to claim 1, characterized in that: The training process of the YOLOv8 target detection algorithm includes: Collect image data of real tunnels and simulated tunnels in underground mines to obtain image data sets; Amplifying the image data set to obtain an amplified data set; Preprocessing the amplified data set to obtain a preprocessed image; The preprocessed images are used to train the YOLOv8 target detection algorithm. After the training is completed, the optimal weight is set for the YOLOv8 target detection algorithm to obtain the trained YOLOv8 target detection algorithm.

4. The method for detecting the size of obstacles in front of an unmanned electric vehicle based on YOLOv8 according to claim 3 is characterized in that: The image data set is enlarged by using flipping, cropping and occluding operations to obtain an enlarged data set.

5. The method for detecting the size of obstacles in front of an unmanned electric vehicle based on YOLOv8 according to claim 1, characterized in that: The SGBM stereo matching algorithm is used to match the left view and the right view to obtain obstacle distance information, including: Set up a commanding position on an obstacle The coordinates of , commanding heights Pixel coordinates projected onto the left camera's viewing axis for , the corresponding pixel coordinates on the right image plane for ; The binocular camera disparity is obtained by the SGBM stereo matching algorithm D as follows: Using binocular geometry and binocular camera parallax, we can calculate the commanding heights. The distance between the projection on the left camera's boresight and the projection on the right camera's boresight , get the high ground Coordinates as follows: Where: Represents the pixel horizontal coordinate of the object projection on the left camera image; Represents the pixel horizontal coordinate of the object projection on the right camera image; represents the focal length of the camera, is the distance between the optical centers of the cameras; For the commanding heights Parallax.

6. The method for detecting the size of obstacles in front of an unmanned electric vehicle based on YOLOv8 according to claim 5, characterized in that: Obstacle height is calculated based on obstacle distance information, including: Based on the commanding heights of obstacles Coordinates , calculate the commanding height Height relative to the ground : in, It represents the angle between the optical axis and the horizontal plane, and H represents the height of the binocular camera relative to the ground.

7. The method for detecting the size of obstacles in front of an unmanned electric vehicle based on YOLOv8 according to claim 5, characterized in that: Obstacle width is calculated based on obstacle distance information, including: Use the left cut surface of the left view of the binocular camera and the initial point of the obstacle , and the right section of the right view of the binocular camera and the initial point of the obstacle , calculate the obstacle width as follows: 。 8. A YOLOv8-based obstacle size detection system for unmanned electric vehicles, characterized in that: include: Preprocessing unit: used to preprocess the unmanned electric locomotive environment image data collected by the binocular camera to obtain a preprocessed image; Recognition unit: used to detect obstacle categories in pre-processed images using the trained YOLOv8 target detection algorithm to determine whether there are obstacles in the pre-processed images; The first calculation unit is used for matching the left view and the right view by using the SGBM stereo matching algorithm to obtain obstacle distance information when there is an obstacle in the preprocessed image; The second calculation unit is used to calculate the height and width of the obstacle according to the obstacle distance information.

9. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for detecting the size of obstacles in front of an unmanned electric vehicle based on YOLOv8 as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: A computer program is stored thereon, wherein when the computer program is executed by a processor, the method for detecting the size of an obstacle in front of an unmanned electric vehicle based on YOLOv8 as described in any one of claims 1 to 7 is implemented.

Images