Method for acceptance of visual perception performance of autonomous vehicle

Abstract

The application discloses an acceptance method for visual perception performance of an automatic driving vehicle, and the method comprises the following steps: arranging a plurality of obstacles in a designated open-pit mine test area to construct an acceptance scene; driving an unmanned mine vehicle in the acceptance scene under different illumination environments, acquiring N frames of acceptance scene images through a visual perception device on the unmanned mine vehicle, and detecting the N frames of acceptance scene images by using a detection model to obtain first obstacle detection result data; re-inspecting the acquired N frames of acceptance scene images to obtain second obstacle detection result data; determining a detection recall rate and an identification accuracy rate based on the first obstacle detection result data and the second obstacle detection result data; if the detection recall rate is greater than a first preset value and the identification accuracy rate is greater than a second preset value, then determining that the visual perception performance of the unmanned mine vehicle is qualified; and the acceptance method realizes effective evaluation on the visual perception performance of the unmanned mine vehicle.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and more specifically, to a method for accepting the visual perception performance of autonomous vehicles. Background Technology

[0002] In recent years, with the continuous development of autonomous driving technology, driverless mining trucks have become an important component of intelligent mining solutions. In open-pit mines, material spillage, rockfalls, and axle crushing frequently occur, resulting in various obstacles in the driving area of ​​driverless mining trucks. These obstacles can affect the normal operation of the trucks. To prevent collisions with these obstacles, it is necessary to identify them and adjust the driving path accordingly. Currently, this is generally achieved by integrating visual perception devices into driverless mining trucks to sense the surrounding environment and detect potential obstacles; however, a standardized evaluation scheme for the visual perception performance of driverless mining trucks is lacking, making it impossible to effectively verify whether the visual perception performance of driverless mining trucks meets design requirements.

[0003] In summary, there is an urgent need to develop an acceptance method for the visual perception performance of autonomous vehicles to solve the aforementioned technical problems. Summary of the Invention

[0004] One objective of this invention is to provide a new technical solution for the acceptance method of the visual perception performance of autonomous vehicles.

[0005] This invention provides a method for accepting the visual perception performance of autonomous vehicles, the method comprising:

[0006] Step S1: Arrange multiple obstacles within the designated open-pit mine test area to construct an acceptance scenario;

[0007] Step S2: Under different lighting conditions, the unmanned mining truck drives in the acceptance scene, acquires N frames of acceptance scene images through the visual perception device on the unmanned mining truck, and uses the detection model to detect the N frames of acceptance scene images to obtain the first obstacle detection result data.

[0008] Step S3: Re-inspect the acquired N frames of acceptance scene images to obtain the second obstacle detection result data;

[0009] Step S4: Determine visual perception performance indicators based on the first obstacle detection result data and the second obstacle detection result data, wherein the visual perception performance indicators include detection recall and recognition accuracy;

[0010] Step S5: If the detection recall rate is greater than the first preset value and the recognition accuracy is greater than the second preset value, then the visual perception performance of the unmanned mining vehicle is determined to be qualified; otherwise, the visual perception performance of the unmanned mining vehicle is determined to be unqualified.

[0011] Optionally, in step S1, the obstacle is a falling rock.

[0012] Optionally, the minimum size of the falling rock is 30cm×30cm×30cm.

[0013] Optionally, the minimum spacing between two adjacent rocks is 50m.

[0014] Optionally, in step S2, the different lighting environments include daytime environments and nighttime environments.

[0015] Optionally, step S4 specifically includes:

[0016] Statistical analysis is performed on the first obstacle detection result data and the second obstacle detection result data to obtain the first frame number TP, the second frame number FN1, and the third frame number FN2; wherein, the first frame number TP represents the number of image frames in the N frames of the acceptance scene image where there are actually falling rocks and the detection model detects them as falling rocks, the second frame number FN1 represents the number of image frames in the N frames of the acceptance scene image where there are actually falling rocks but the detection model identifies them as other obstacles, and the third frame number FN2 represents the number of image frames in the N frames of the acceptance scene image where there are actually falling rocks but the detection model does not detect them;

[0017] The detection recall rate is calculated based on the first frame number TP, the second frame number FN1, and the third frame number FN2;

[0018] The recognition accuracy is calculated based on the first frame number TP and the second frame number FN1.

[0019] Optionally, the formula for calculating the detection recall rate is expressed as:

[0020]

[0021] Optionally, the formula for calculating the detection accuracy is expressed as:

[0022]

[0023] Optionally, the first preset value and the second preset value may be the same or different.

[0024] Optionally, both the first preset value and the second preset value are 85%.

[0025] According to an embodiment disclosed by the present invention, a method for accepting the visual perception performance of an autonomous driving vehicle of the present invention has the following beneficial effects:

[0026] A method for accepting the visual perception performance of an autonomous driving vehicle of the present invention first arranges a plurality of obstacles within a demarcated open-pit mine test area to construct an acceptance scenario; then, in different lighting environments, the driverless mining truck travels within the acceptance scenario, obtains N frames of acceptance scenario images through the visual perception device on the driverless mining truck, and uses a detection model to detect the N frames of acceptance scenario images to obtain first obstacle detection result data; then复检the obtained N frames of acceptance scenario images to obtain second obstacle detection result data; then determine the detection recall rate and recognition accuracy based on the first obstacle detection result data and the second obstacle detection result data; finally, if the detection recall rate is greater than a first preset value and the recognition accuracy is greater than a second preset value, it is determined that the visual perception performance of the driverless mining truck is qualified. The present invention provides a set of standardized methods for accepting the visual perception performance of autonomous driving vehicles, realizing an effective evaluation of the visual perception performance of driverless mining trucks.

[0027] Through the following detailed description of the exemplary embodiments of the present invention with reference to the accompanying drawings, other features and advantages of the present invention will become clear. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings incorporated in the specification and constituting a part of the specification illustrate embodiments of the present invention and, together with the description, are used to explain the principles of the present invention.

[0029] Figure 1 FIG. is a schematic flow chart of a method for accepting the visual perception performance of an autonomous driving vehicle provided according to an embodiment;

[0030] Figure 2 FIG. is a schematic diagram of obstacle arrangement provided according to an embodiment. DETAILED DESCRIPTION

[0031] Now, various exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that: unless otherwise specifically stated, the relative arrangements, numerical expressions, and numerical values of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0032] The following description of at least one exemplary embodiment is merely illustrative in nature and in no way serves as a limitation on the present invention or its application or use.

[0033] Known technologies, methods, and devices of those skilled in the relevant art may not be discussed in detail, but where appropriate, the said technologies, methods, and devices should be regarded as a part of the specification.

[0034] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0035] See Figure 1 As shown in the figure, this embodiment of the invention provides an acceptance method for the visual perception performance of an autonomous vehicle, the method comprising:

[0036] Step S1: Arrange multiple obstacles within the designated open-pit mine test area to construct an acceptance scenario.

[0037] It should be noted that in this embodiment, multiple obstacles can be arranged according to predetermined rules. For example, the preset rules can be arranged in a straight line, a broken line, a curve, etc. Figure 2 As shown; the types of multiple obstacles can be the same or different. For example, the types of obstacles can be people, auxiliary vehicles, rocks, mining trucks, bulldozers, electric shovels, retaining walls, water pits, etc., without specific limitations.

[0038] Preferably, in the method for accepting the visual perception performance of autonomous vehicles in this embodiment, the obstacle is a falling rock.

[0039] It should be noted that there are many obstacles in open-pit mines, including falling rocks of varying sizes, scattered throughout the mining area. When unmanned mining trucks run over small falling rocks, it can easily cause safety accidents such as component failure, tire blowouts, and rollovers. Therefore, in this embodiment, falling rocks are mainly selected as obstacles for acceptance testing.

[0040] Preferably, in the method for accepting the visual perception performance of autonomous vehicles in this embodiment, the minimum size of the falling rock is 30cm×30cm×30cm.

[0041] Preferably, in the method for accepting the visual perception performance of autonomous vehicles in this embodiment, the minimum arrangement distance between two adjacent falling rocks is 50m.

[0042] It should be noted that while the unmanned mining truck can easily detect large obstacles, it is prone to misperception of small obstacles. Therefore, in this embodiment, the size of falling rocks (i.e., small obstacles) is set to be greater than 30*30*30cm, and the effective distance for stable recognition is no less than 50m. Of course, other sizes and recognition distances can also be set, which will not be listed here.

[0043] Step S2: Under different lighting conditions, the unmanned mining truck drives in the acceptance scene, acquires N frames of acceptance scene images through the visual perception device on the unmanned mining truck, and uses the detection model to detect the N frames of acceptance scene images to obtain the first obstacle detection result data.

[0044] In this embodiment, the unmanned mining truck integrates a visual perception device, which may include one or more camera devices capable of capturing images of the acceptance scene under different lighting conditions. For example, the device includes two camera devices, namely a visible light camera and an infrared camera.

[0045] In this embodiment, the acquired acceptance scene images can be used for obstacle detection using a pre-established detection model. This detection model can be integrated into the software control system of the unmanned mining truck or into a remote driving cockpit; no specific limitation is made here. In this embodiment, the detection model can be a convolutional neural network model, trained using a pre-made training image set. Of course, in this embodiment, the detection model can also be a model using traditional image processing and detection algorithms; no specific limitation is made here.

[0046] It should be noted that in this embodiment, the number of acceptance image frames N is greater than or equal to 1000. For example, it can be 1000 frames, 1500 frames, 2000 frames, etc., which will not be listed here.

[0047] It should be noted that the first obstacle detection result data in this embodiment refers to the detection log result corresponding to each frame of the acceptance scene image, wherein each frame of the image contains at most one stone or no stone.

[0048] Optionally, in step S2 of the method for accepting the visual perception performance of autonomous vehicles in this embodiment, different lighting environments include daytime environments and nighttime environments.

[0049] Since the ambient light is weak at night, which has a significant impact on visual perception, the method in this embodiment adopts the method of conducting acceptance tests on the visual perception performance of the unmanned mining truck during the day and at night respectively. For example, the unmanned mining truck can drive in the acceptance scene in the daytime and nighttime environments within 48 hours to obtain 1,000 acceptance scene images.

[0050] Step S3: Re-inspect the acquired N frames of acceptance scene images to obtain the second obstacle detection result data.

[0051] In this embodiment, the N frames of acceptance scene images obtained can be re-inspected by manually reviewing the images. Of course, other methods can also be used for re-inspection, and no specific limitation is made here.

[0052] Step S4: Determine visual perception performance indicators based on the first obstacle detection result data and the second obstacle detection result data, wherein the visual perception performance indicators include detection recall rate and recognition accuracy rate.

[0053] Optionally, step S4 in the acceptance method for the visual perception performance of autonomous vehicles in this embodiment specifically includes:

[0054] Statistical analysis was performed on the first obstacle detection result data and the second obstacle detection result data to obtain the first frame number TP, the second frame number FN1, and the third frame number FN2. Among them, the first frame number TP represents the number of image frames in the N frames of the acceptance scene image where there are actually falling rocks and the detection model detects them as falling rocks; the second frame number FN1 represents the number of image frames in the N frames of the acceptance scene image where there are actually falling rocks but the detection model identifies them as other obstacles; and the third frame number FN2 represents the number of image frames in the N frames of the acceptance scene image where there are actually falling rocks but the detection model does not detect them.

[0055] The detection recall rate is calculated based on the first frame number TP, the second frame number FN1, and the third frame number FN2;

[0056] The recognition accuracy is calculated based on the first frame number TP and the second frame number FN1.

[0057] Optionally, the formula for calculating the recall rate in the acceptance method for the visual perception performance of autonomous vehicles in this embodiment is expressed as follows:

[0058]

[0059] Optionally, the calculation formula for the detection accuracy in the acceptance method for the visual perception performance of autonomous vehicles in this embodiment is expressed as:

[0060]

[0061] It should be noted that in this embodiment, statistical analysis of the first obstacle detection result data and the second obstacle detection result data can also yield a fourth frame number FP. The fourth frame number FP represents the number of image frames in the N frames of the acceptance scene image where there are no actual falling rocks but the detection model detects falling rocks. The false detection rate can be calculated using the third frame number FN2 and the fourth frame number FP.

[0062] Step S5: If the detection recall rate is greater than the first preset value and the recognition accuracy is greater than the second preset value, then the visual perception performance of the unmanned mining vehicle is determined to be qualified; otherwise, the visual perception performance of the unmanned mining vehicle is determined to be unqualified.

[0063] Optionally, in the acceptance method for the visual perception performance of autonomous vehicles in this embodiment, the first preset value and the second preset value may be the same or different. It should be noted that in this embodiment, the values ​​of the first preset value and the second preset value are determined according to the acceptance criteria set according to the specific project situation.

[0064] Preferably, in the acceptance method for the visual perception performance of autonomous vehicles in this embodiment, both the first preset value and the second preset value are 85%.

[0065] In summary, the acceptance method for the visual perception performance of the autonomous driving vehicle in the embodiment of the present invention first arranges a plurality of obstacles in the designated open-pit mine test area to construct an acceptance scenario; then, in different lighting environments, the driverless mining truck travels in the acceptance scenario, obtains N frames of acceptance scenario images through the visual perception device on the driverless mining truck, and uses the detection model to detect the N frames of acceptance scenario images to obtain the first obstacle detection result data; then复检the obtained N frames of acceptance scenario images to obtain the second obstacle detection result data; then determine the detection recall rate and recognition accuracy based on the first obstacle detection result data and the second obstacle detection result data; finally, if the detection recall rate is greater than the first preset value and the recognition accuracy is greater than the second preset value, it is determined that the visual perception performance of the driverless mining truck is qualified. The embodiment of the present invention provides a set of standardized acceptance methods for the visual perception performance of autonomous driving vehicles, realizing an effective evaluation of the visual perception performance of driverless mining trucks.

[0066] Please note that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combinations of these technical features do not conflict, they should be considered as the scope described in this specification. The above embodiments only represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

[0067] The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0068] Although some specific embodiments of the present invention have been described in detail by way of examples, those skilled in the art should understand that the above examples are only for the purpose of illustration and not for the purpose of limiting the scope of the present invention. Those skilled in the art should understand that the above embodiments can be modified without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A method for accepting the visual perception performance of an autonomous vehicle, characterized in that, The method include: Step S1: Arrange multiple obstacles within the designated open-pit mine test area to construct an acceptance scenario; Step S2: Under different lighting conditions, the unmanned mining truck drives in the acceptance scene, acquires N frames of acceptance scene images through the visual perception device on the unmanned mining truck, and uses the detection model to detect the N frames of acceptance scene images to obtain the first obstacle detection result data. Step S3: Re-inspect the acquired N frames of acceptance scene images to obtain the second obstacle detection result data; Step S4: Determine visual perception performance indicators based on the first obstacle detection result data and the second obstacle detection result data, wherein the visual perception performance indicators include detection recall and recognition accuracy; Step S5: If the detection recall rate is greater than the first preset value and the recognition accuracy is greater than the second preset value, then the visual perception performance of the unmanned mining vehicle is determined to be qualified; otherwise, the visual perception performance of the unmanned mining vehicle is determined to be unqualified.

2. The method for accepting the visual perception performance of autonomous vehicles according to claim 1, characterized in that, In step S1, the obstacle is a falling rock.

3. The method for accepting the visual perception performance of autonomous vehicles according to claim 2, characterized in that, The minimum size of the falling rock is 30cm×30cm×30cm.

4. The method for accepting the visual perception performance of autonomous vehicles according to claim 3, characterized in that, The minimum spacing between two adjacent falling rocks is 50m.

5. The method for accepting the visual perception performance of autonomous vehicles according to claim 1, characterized in that, In step S2, the different lighting environments include daytime environments and nighttime environments.

6. The method for accepting the visual perception performance of autonomous vehicles according to claim 1, characterized in that, Step S4 specifically includes: Statistical analysis is performed on the first obstacle detection result data and the second obstacle detection result data to obtain the first frame number TP, the second frame number FN1, and the third frame number FN2; wherein, the first frame number TP represents the number of image frames in the N frames of the acceptance scene image where there are actually falling rocks and the detection model detects them as falling rocks, the second frame number FN1 represents the number of image frames in the N frames of the acceptance scene image where there are actually falling rocks but the detection model identifies them as other obstacles, and the third frame number FN2 represents the number of image frames in the N frames of the acceptance scene image where there are actually falling rocks but the detection model does not detect them; The detection recall rate is calculated based on the first frame number TP, the second frame number FN1, and the third frame number FN2; The recognition accuracy is calculated based on the first frame number TP and the second frame number FN1.

7. The method for accepting the visual perception performance of autonomous vehicles according to claim 6, characterized in that, The formula for calculating the detection recall rate is expressed as follows:

8. The method for accepting the visual perception performance of an autonomous vehicle according to claim 6, characterized in that, The formula for calculating the detection accuracy is as follows:

9. The method for accepting the visual perception performance of an autonomous vehicle according to claim 1, characterized in that, The first preset value and the second preset value may be the same or different.

10. The method for accepting the visual perception performance of an autonomous vehicle according to claim 9, characterized in that, Both the first preset value and the second preset value are 85%.

Images