Obstacle Detection Method, Vehicle Control Method, Device, Vehicle and Medium

Abstract

Embodiments of the present invention disclose an obstacle detection method, a vehicle control method, a device, a vehicle, and a medium. The method includes: obtaining initial image data collected by an in-vehicle camera; inputting the initial image data into a trained semantic segmentation model to obtain a semantic segmentation result; determining in-distribution image data corresponding to the semantic segmentation result; and inputting the initial image data and the in-distribution image data into a trained out-of-distribution detection model to obtain out-of-distribution obstacle information. This solves the problem that existing obstacle detection methods cannot detect out-of-distribution obstacles.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of image processing, and in particular, to an obstacle detection method, a vehicle control method, a device, a vehicle, and a medium. Background Art

[0002] Currently, artificial intelligence algorithms related to autonomous driving are being widely used. Among them, image semantic segmentation technology can help the in-vehicle computer perceive the surrounding environment and achieve pixel-level class prediction for the images collected by the camera. Current semantic segmentation algorithms, such as U-Net, DeepLab, etc., can make accurate classification predictions for objects within the dataset (in-distribution). However, when objects outside the dataset (out-of-distribution) appear in the image to be detected, these algorithms often misclassify, for example, classifying an obstacle as a road.

[0003] In summary, the existing obstacle detection methods have the problem of being unable to recognize out-of-distribution obstacles. Summary of the Invention

[0004] Embodiments of the present invention provide an obstacle detection method, a vehicle control method, a device, a vehicle, and a medium, which solve the problem that the existing obstacle detection methods are unable to recognize out-of-distribution obstacles.

[0005] In a first aspect, embodiments of the present invention provide an out-of-distribution obstacle detection method, which includes:

[0006] Obtain initial image data collected by an in-vehicle camera;

[0007] Input the initial image data into a trained semantic segmentation model to obtain a semantic segmentation result, and the trained semantic segmentation model is used to determine the semantic classification labels of in-distribution objects;

[0008] Determine the in-distribution image data corresponding to the semantic segmentation result;

[0009] Input the initial image data and the in-distribution image data into a trained out-of-distribution detection model to obtain out-of-distribution obstacle information.

[0010] In a second aspect, embodiments of the present invention also provide an out-of-distribution obstacle detection device, which includes:

[0011] An acquisition module, configured to obtain initial image data collected by an in-vehicle camera;

[0012] A semantic segmentation module, configured to input the initial image data into a trained semantic segmentation model to obtain a semantic segmentation result;

[0013] An in-distribution image determination module, configured to determine the in-distribution image data corresponding to the semantic segmentation result;

[0014] An out-of-distribution obstacle information determination module, configured to input the initial image data and in-distribution image data into a trained out-of-distribution detection model to obtain out-of-distribution obstacle information, where the out-of-distribution obstacle information includes the position information of out-of-distribution obstacles.

[0015] In a third aspect, an embodiment of the present invention provides a vehicle control method, the method including:

[0016] Inputting the initial image data into a trained in-distribution detection model to obtain in-distribution obstacle information, and determining out-of-distribution obstacle information by using the out-of-distribution obstacle detection method described in the embodiment of the present invention;

[0017] Controlling the vehicle to drive around obstacles, where the obstacles include in-distribution obstacles corresponding to the in-distribution obstacle information, and / or, out-of-distribution obstacles corresponding to the out-of-distribution obstacle information.

[0018] In a fourth aspect, an embodiment of the present invention further provides a vehicle control device, the device including:

[0019] An obstacle detection module, configured to input the initial image data into a trained in-distribution detection model to obtain in-distribution obstacle information, and determine out-of-distribution obstacle information according to the out-of-distribution obstacle detection method described in the embodiment of the present invention;

[0020] A vehicle control module, configured to control the vehicle to drive around obstacles, where the obstacles include in-distribution obstacles corresponding to the in-distribution obstacle information, and / or, out-of-distribution obstacles corresponding to the out-of-distribution obstacle information.

[0021] In a fifth aspect, an embodiment of the present invention further provides a vehicle, the vehicle including:

[0022] One or more processors;

[0023] A storage device, configured to store one or more programs;

[0024] When the one or more programs are executed by the one or more processors, enabling the one or more processors to implement the out-of-distribution obstacle detection method described in any embodiment of the present invention or the vehicle control method described in any embodiment of the present invention.

[0025] In a sixth aspect, an embodiment of the present invention further provides a storage medium including computer-executable instructions, where the computer-executable instructions are used to execute the out-of-distribution obstacle detection method described in any embodiment of the present invention or the vehicle control method described in any embodiment of the present invention when executed by a computer processor.

[0026] The technical solution of the obstacle detection method provided by the embodiment of the present invention obtains the initial image data collected by the vehicle-mounted camera, inputs the initial image data into the trained semantic segmentation model to obtain the semantic segmentation result, determines the in-distribution image data corresponding to the semantic segmentation result, and inputs the initial image data and the in-distribution image data into the trained out-of-distribution detection model to obtain the out-of-distribution obstacle information. By determining the in-distribution image data containing in-distribution objects through the trained semantic segmentation model, and analyzing the initial image data and the in-distribution image data through the trained out-of-distribution detection model, the trained out-of-distribution detection model can analyze the difference information between the initial image data and the in-distribution image data with the in-distribution image data as a reference, so as to obtain the out-of-distribution obstacle information, and the out-of-distribution obstacle information has high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0028] Figure 1 It is a flowchart of an out-of-distribution obstacle detection method provided by Embodiment 1 of the present invention;

[0029] Figure 2 It is a structural block diagram of an out-of-distribution obstacle detection device provided by Embodiment 2 of the present invention;

[0030] Figure 3 It is a flowchart of a vehicle control method provided by Embodiment 3 of the present invention;

[0031] Figure 4 It is a structural block diagram of a vehicle control device provided by Embodiment 4 of the present invention;

[0032] Figure 5 It is a schematic structural diagram of a vehicle provided by Embodiment 5 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will, with reference to the drawings in the embodiments of the present invention, clearly and completely describe the technical solutions of the present invention through implementation manners. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0034] Embodiment 1

[0035] Conventional detection methods can detect conventional obstacles within the distribution, but for out-of-distribution obstacles that appear in some scenarios, conventional detection methods are usually difficult to detect.

[0036] Figure 1 FIG. 4 is a flowchart of a method for detecting out-of-distribution obstacles provided in the first embodiment of the present invention. The technical solution of this embodiment is applicable to the situation where out-of-distribution obstacles may exist in the vehicle driving scenario. This method can be executed by an out-of-distribution obstacle detection device provided by the embodiments of the present invention. The device can be implemented in software and / or hardware and configured to be applied in a vehicle processor. The method specifically includes the following steps:

[0037] S110. Obtain initial image data collected by an in-vehicle camera.

[0038] Among them, in-vehicle cameras are mainly divided into front-view cameras, surround-view cameras, rear-view cameras, side-view cameras, and built-in cameras. In the embodiments of the present application, a front-view camera is used. The installation angle and field of view of the front-view camera should meet the following conditions: all parts of the forward road of the target vehicle are captured, and the farthest distance of the forward road covered by each original image in the road condition video is covered.

[0039] In one embodiment, the situation of the current driving road where the target vehicle is located is photographed and recorded by the front-view camera to obtain a road condition video, and frame images in the road condition video are sequentially extracted as initial image data; alternatively, the real-time image collected by the front-view camera is directly obtained as the initial image data.

[0040] In one embodiment, the collected initial image data is preprocessed by using an existing image preprocessing method to update the initial image data. Through image preprocessing, the quality of the initial image data can be improved, thereby improving the accuracy of the image processing results obtained in subsequent image processing steps.

[0041] It can be understood that when the vehicle is driving in a scenario that may contain out-of-distribution obstacles, the initial image data may contain in-distribution obstacle information and / or out-of-distribution obstacle information. Among them, in-distribution obstacles are obstacles that can be recognized by the current obstacle recognition means, and out-of-distribution obstacles are obstacles that cannot be recognized by the current obstacle recognition means.

[0042] S120. Input the initial image data into a trained semantic segmentation model to obtain a semantic segmentation result. The trained semantic segmentation model is used to determine the semantic classification labels of in-distribution objects.

[0043] The semantic segmentation model is used to determine the semantic classification labels of each object in the image. It can be understood that the trained semantic segmentation model can determine the semantic classification labels of the in-distribution objects in the initial image; the trained semantic segmentation model cannot recognize the out-of-distribution objects, so it will classify the out-of-distribution objects into the in-distribution objects where they are located or connected. Exemplarily, the road is an in-distribution object, and the sand pile on the road is an out-of-distribution object. The trained semantic segmentation model cannot recognize the sand pile, so it classifies the sand pile as the road and assigns it a road label.

[0044] In one embodiment, the initial image corresponding to the initial image data includes a road, a vehicle, a tree, a flower bed, and a sand pile on the road, where the sand pile is an out-of-distribution object. The initial image data is input into the trained semantic segmentation model to obtain a semantic segmentation result. The size of the semantic segmentation result and the in-distribution image corresponding to the semantic segmentation result are the same as the size of the initial image, and each pixel value is the class label value corresponding to the corresponding pixel in the initial image. Exemplarily, the class label value of the road surface is set to purple, the class label value of the vehicle is set to blue, and the class label value of the tree is set to green. Since the trained semantic segmentation model cannot recognize the sand pile, the area where it is located is classified as the road, that is, its class label value is set to purple.

[0045] S130. Determine the in-distribution image data corresponding to the semantic segmentation result.

[0046] Among them, all the objects included in the in-distribution image data are objects that the trained semantic segmentation model can recognize.

[0047] Determining the in-distribution image data corresponding to the semantic segmentation result may include: inputting the semantic segmentation result into the trained image reconstruction model to obtain the in-distribution image data.

[0048] Among them, the trained image reconstruction model is used to reconstruct the input semantic segmentation result to obtain the in-distribution image data. Since the input image of the trained reconstruction model is the semantic segmentation result, no unknown obstacles will appear in the in-distribution image data.

[0049] S140. Input the initial image data and the in-distribution image data into the trained out-of-distribution detection model to obtain out-of-distribution obstacle information.

[0050] Among them, the out-of-distribution obstacle information includes objects that the trained semantic segmentation model cannot recognize, and the position information of the objects. The trained out-of-distribution detection model is trained by the out-of-distribution detection model based on the in-distribution image data randomly added with out-of-distribution obstacle information, and its function is to determine the out-of-distribution obstacle information in the initial image data. Exemplarily, the randomly added out-of-distribution obstacles include overturned vehicles, vehicle parts, construction materials, falling rocks, etc.

[0051] It is understandable that inputting the initial image data and the in-distribution image data into the trained out-of-distribution detection model enables the trained out-of-distribution detection model to analyze the difference information between the initial image data and the in-distribution image data with the in-distribution image data as a reference, so as to obtain the out-of-distribution obstacle information. Exemplarily, an initial image captured by an on-vehicle camera of a current vehicle includes a vehicle and a wardrobe placed on the vehicle. Among them, the vehicle is an in-distribution object, and the wardrobe is an out-of-distribution object. After the trained out-of-distribution detection model detects the wardrobe, if the wardrobe will affect the detour of the current vehicle, for example, the wardrobe extends beyond the vehicle bed in the lateral direction of the vehicle bed, it is determined that the wardrobe is an out-of-distribution obstacle; if the wardrobe will not affect the detour of the current vehicle, for example, the wardrobe does not extend beyond the vehicle bed in the lateral direction of the vehicle bed, it is determined that the wardrobe is not an out-of-distribution obstacle. It can be seen that the introduction of the in-distribution image data can significantly improve the accuracy of the out-of-distribution obstacle information output by the trained out-of-distribution detection model.

[0052] In one embodiment, the out-of-distribution image data is determined according to the initial image data and the in-distribution image data, and the out-of-distribution image data is input into the trained out-of-distribution detection model to obtain the out-of-distribution obstacle information. Directly inputting the out-of-distribution image data into the trained out-of-distribution detection model can reduce the data analysis amount of the trained out-of-distribution detection model, thereby improving the speed of the out-of-distribution obstacle information.

[0053] In one embodiment, the initial image, the semantic segmentation result, and the in-distribution image data are input into the trained out-of-distribution detection model to obtain the out-of-distribution obstacle information. Inputting the semantic segmentation result and the in-distribution image data into the trained out-of-distribution detection model enables the trained out-of-distribution detection model to analyze the difference information between the initial image data and the in-distribution image data with the semantic segmentation result and the in-distribution image data as references, so as to obtain the out-of-distribution obstacle information, which helps to improve the accuracy of the prediction of the trained out-of-distribution detection model, that is, to improve the accuracy of the determination of the out-of-distribution obstacle information.

[0054] The technical solution of this embodiment is to obtain the initial image data collected by the vehicle-mounted camera; input the initial image data into the trained semantic segmentation model to obtain the semantic segmentation result, and the trained semantic segmentation model is used to determine the semantic classification labels of in-distribution objects; determine the in-distribution image data corresponding to the semantic segmentation result; input the initial image data and the in-distribution image data into the trained out-of-distribution detection model to obtain the out-of-distribution obstacle information. In the embodiment of the present invention, the initial image data is semantically segmented to obtain the semantic segmentation result, and the in-distribution image data corresponding to the semantic segmentation result is determined to determine the in-distribution object information, and the in-distribution image data is used as a reference to compare the differences between the initial image data and the in-distribution image data to obtain the out-of-distribution obstacle information, achieving the technical effect of accurately detecting out-of-distribution obstacles.

[0055] Embodiment 2

[0056] Figure 2 It is a structural block diagram of an out-of-distribution obstacle detection device in Embodiment 2 of the present invention. This device is used to execute the out-of-distribution obstacle detection method provided in any of the above embodiments, and this device can be implemented by a vehicle processor. The device includes:

[0057] An acquisition module 210, configured to acquire the initial image data collected by the vehicle-mounted camera;

[0058] A semantic segmentation module 220, configured to input the initial image data into the trained semantic segmentation model to obtain the semantic segmentation result;

[0059] An in-distribution image determination module 230, configured to determine the in-distribution image data corresponding to the semantic segmentation result;

[0060] An obstacle information determination module 240, configured to input the initial image data and the in-distribution image data into the trained out-of-distribution detection model to obtain the out-of-distribution obstacle information, and the out-of-distribution obstacle information includes the position information of the out-of-distribution obstacle.

[0061] Optionally, the in-distribution image determination module 230 is specifically configured to input the semantic segmentation result into the trained image reconstruction model to obtain the in-distribution image data.

[0062] Optionally, the obstacle information determination module 240 is specifically configured to input the initial image, the semantic segmentation result, and the in-distribution image data into the trained out-of-distribution detection model to obtain the out-of-distribution obstacle information.

[0063] Optionally, the obstacle information determination module 240 is further configured to determine the out-of-distribution image data according to the initial image data and the in-distribution image data, and input the out-of-distribution image data into the trained out-of-distribution detection model to obtain the out-of-distribution obstacle information.

[0064] Optionally, the obstacle information determination module 240 is further configured to determine out-of-distribution image data based on the initial image data and in-distribution image data, and input the out-of-distribution image data and the semantic segmentation result into a trained out-of-distribution detection model to obtain out-of-distribution obstacle information.

[0065] In the technical solution of the control device of the intelligent terminal provided in the second embodiment of the present invention, the semantic segmentation module performs semantic segmentation on the initial image data to obtain a semantic segmentation result, and the in-distribution image determination module determines the in-distribution image data corresponding to the semantic segmentation result to determine the in-distribution object information. The obstacle information determination module is controlled to use the in-distribution image data as a reference, compare the differences between the initial image data and the in-distribution image data to obtain out-of-distribution obstacle information, achieving the technical effect of accurately detecting out-of-distribution obstacles.

[0066] An out-of-distribution obstacle detection device provided by an embodiment of the present invention can execute an out-of-distribution obstacle detection method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution of the method.

[0067] Embodiment Three

[0068] Figure 3 It is a flowchart of a vehicle control method provided by the third embodiment of the present invention. Based on any of the above embodiments, the present invention adds a vehicle control method in this embodiment. The method of this embodiment includes:

[0069] S310: Input the initial image data into a trained in-distribution detection model to obtain in-distribution obstacle information, and determine out-of-distribution obstacle information according to the out-of-distribution obstacle detection method described in any of the above embodiments.

[0070] During the driving process of the vehicle, various obstacles will be encountered, including in-distribution obstacles and out-of-distribution obstacles. Inputting the initial image data into a trained in-distribution detection model can obtain in-distribution obstacle information, and out-of-distribution obstacle information can be determined according to the out-of-distribution obstacle detection method described in any of the above embodiments.

[0071] In one embodiment, at least two obstacle detection modes are configured for the vehicle, and the at least two obstacle detection modes include a conventional obstacle detection mode and a comprehensive obstacle detection mode. Among them, the comprehensive obstacle detection mode includes the conventional obstacle detection mode and the out-of-distribution obstacle detection mode; the obstacle detection mode can be automatically selected by the user or automatically switched by the vehicle according to the scene where the vehicle is currently located. For example, when it is detected that the vehicle is driving on a highway, the conventional obstacle detection mode is used for obstacle detection, and when it is detected that the vehicle is in a complex traffic scene such as a provincial road, the comprehensive obstacle detection mode is used for obstacle detection.

[0072] In one embodiment, different out-of-distribution obstacle detection methods are set for different scenarios. For example, corresponding trained out-of-distribution obstacle detection models are set for provincial roads and rural roads respectively. When it is detected that the vehicle is driving on a highway, a conventional obstacle detection mode is adopted, and this conventional obstacle detection mode corresponds to a conventional obstacle detection method; when it is detected that the vehicle travels to a provincial road, an obstacle detection method combining an out-of-distribution obstacle detection mode and a conventional obstacle detection mode is adopted for obstacle detection, and the current out-of-distribution obstacle detection mode uses the first trained out-of-distribution obstacle detection model for obstacle detection; when it is detected that the vehicle travels to a rural road, an obstacle detection method combining an out-of-distribution obstacle detection mode and a conventional obstacle detection mode is adopted for obstacle detection, and the current out-of-distribution obstacle detection mode uses the second trained out-of-distribution obstacle detection model for obstacle detection. Among them, the training samples corresponding to the first trained out-of-distribution obstacle detection model and the second trained in-distribution obstacle detection model are different. The training samples corresponding to the former include out-of-distribution obstacles that may appear on provincial roads, and the training samples corresponding to the latter include in-distribution obstacles that may appear on rural roads.

[0073] S320. Control the vehicle to drive around the obstacle. The obstacle includes in-distribution obstacles corresponding to in-distribution obstacle information and / or out-of-distribution obstacles corresponding to out-of-distribution obstacle information.

[0074] Control the vehicle to drive around the obstacle, where the obstacle includes in-distribution obstacles corresponding to in-distribution obstacle information and / or out-of-distribution obstacles corresponding to out-of-distribution obstacle information.

[0075] The vehicle control method provided by the embodiment of the present invention determines the in-distribution obstacle information included in the initial image data through a trained in-distribution detection model, and determines the out-of-distribution obstacle information included in the initial image data through a trained out-of-distribution obstacle detection model. Controlling the vehicle to avoid in-distribution obstacles corresponding to in-distribution obstacle information and / or out-of-distribution obstacles corresponding to out-of-distribution obstacle information can enable the vehicle to avoid all obstacles and improve the driving safety of the vehicle.

[0076] Embodiment 4

[0077] Figure 4 It is a structural block diagram of a vehicle control device in Embodiment 4 of the present invention. On the basis of implementing the out-of-distribution obstacle detection method provided in any of the above embodiments, the device adds a vehicle control module, and the device can be implemented by a vehicle processor. The device includes:

[0078] An obstacle detection module 410 is configured to input initial image data into a trained in-distribution detection model to obtain in-distribution obstacle information, and determine out-of-distribution obstacle information according to the out-of-distribution obstacle detection method described in any of the above embodiments;

[0079] A vehicle control module 420 is configured to control the vehicle to drive around obstacles, where the obstacles include in-distribution obstacles corresponding to the in-distribution obstacle information and / or out-of-distribution obstacles corresponding to the out-of-distribution obstacle information.

[0080] The vehicle control device provided by the embodiments of the present invention can obtain in-distribution obstacle information and out-of-distribution obstacle information according to the initial image data, and control the vehicle to avoid in-distribution obstacles corresponding to the in-distribution obstacle information and / or out-of-distribution obstacles corresponding to the out-of-distribution obstacle information. By identifying and detecting obstacle information in advance and controlling the vehicle to avoid obstacles corresponding to the obstacle information, the safety of vehicle control is improved.

[0081] A vehicle control device provided by an embodiment of the present invention can execute a vehicle control method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution of the method.

[0082] Embodiment 5

[0083] Figure 5 is a schematic structural diagram of a vehicle provided for Embodiment 5 of the present invention. As Figure 5 shown, the vehicle includes a processor 501, a memory 502, an input device 503, and an output device 504; the number of processors 501 in the vehicle can be one or more, Figure 5 and one processor 501 is taken as an example here; the processor 501, the memory 502, the input device 503, and the output device 504 in the vehicle can be connected through a bus or other means, Figure 5 and taking the connection through a bus as an example here.

[0084] The memory 502, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as modules corresponding to the out-of-distribution obstacle detection method in the embodiments of the present invention (for example, the acquisition module 210, the semantic segmentation module 220, the in-distribution image determination module 230, and the obstacle information determination module 240) or modules corresponding to the vehicle control method in the embodiments of the present invention (for example, the obstacle detection module 410 and the vehicle control module 420). The processor 501 executes various functional applications and data processing of the vehicle by running the software programs, instructions, and modules stored in the memory 502, that is, implements the above out-of-distribution obstacle detection method or vehicle control method.

[0085] The memory 502 may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 502 may further include a memory remotely provided with respect to the processor 501, and these remote memories may be connected to the vehicle through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0086] The input device 503 may be used to receive input digital or character information, and generate key signal inputs related to user settings and function controls of the vehicle.

[0087] The output device 504 may include a display device such as a display screen, for example, the display screen of a user terminal.

[0088] Embodiment Six

[0089] The embodiment of the present invention also provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute an out-of-distribution obstacle detection method or a vehicle control method when executed by a computer processor. The method includes:

[0090] Obtain initial image data collected by an in-vehicle camera;

[0091] Input the initial image data into a trained semantic segmentation model to obtain a semantic segmentation result, and the trained semantic segmentation model is used to determine the semantic classification labels of in-distribution objects;

[0092] Determine the in-distribution image data corresponding to the semantic segmentation result;

[0093] Input the initial image data and the in-distribution image data into a trained out-of-distribution detection model to obtain out-of-distribution obstacle information.

[0094] Or execute a vehicle control method, the method includes:

[0095] Input the initial image data into a trained in-distribution detection model to obtain in-distribution obstacle information, and determine out-of-distribution obstacle information according to the out-of-distribution obstacle detection method described in any of the above embodiments;

[0096] Control the vehicle to drive around the obstacles. The obstacles include in-distribution obstacles corresponding to the in-distribution obstacle information and / or out-of-distribution obstacles corresponding to the out-of-distribution obstacle information.

[0097] Of course, for a storage medium containing computer-executable instructions provided by an embodiment of the present invention, the computer-executable instructions are not limited to the method operations described above, and can also execute relevant operations in the out-of-distribution obstacle detection method or vehicle control method provided by any embodiment of the present invention.

[0098] From the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software and necessary general-purpose hardware. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk, or optical disc of a computer, etc., including several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the out-of-distribution obstacle detection method or vehicle control method described in various embodiments of the present invention.

[0099] It should be noted that in the embodiments of the above out-of-distribution obstacle detection method or vehicle control method, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0100] Note that the above is only the preferred embodiment of the present invention and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described here, and various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more other equivalent embodiments can be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An out-of-distribution obstacle detection method, characterized in that Including: Obtaining initial image data collected by a vehicle-mounted camera; Inputting the initial image data into a trained semantic segmentation model to obtain a semantic segmentation result, where the trained semantic segmentation model is used to determine semantic classification labels of in-distribution objects; the trained semantic segmentation model cannot recognize out-of-distribution objects; Inputting the semantic segmentation result into a trained image reconstruction model to obtain in-distribution image data; the in-distribution image data includes objects that the trained semantic segmentation model can recognize; Inputting the initial image data and the in-distribution image data into a trained out-of-distribution detection model to obtain out-of-distribution obstacle information; the out-of-distribution obstacle information includes unknown objects that the trained semantic segmentation model cannot recognize and the position information of such objects.

2. The method according to claim 1, wherein The step of inputting the initial image and the in-distribution image data into a trained out-of-distribution detection model to obtain out-of-distribution obstacle information includes: Inputting the initial image, the semantic segmentation result, and the in-distribution image data into a trained out-of-distribution detection model to obtain out-of-distribution obstacle information.

3. The method according to any one of claims 1-2, characterized in that, The step of inputting the initial image data and the in-distribution image data into a trained out-of-distribution detection model to obtain out-of-distribution obstacle information includes: Determining out-of-distribution image data based on the initial image data and the in-distribution image data, Inputting the out-of-distribution image data into a trained out-of-distribution detection model to obtain out-of-distribution obstacle information.

4. The method according to claim 2, wherein The step of inputting the initial image, the semantic segmentation result, and the in-distribution image data into a trained out-of-distribution detection model to obtain out-of-distribution obstacle information includes: Determining out-of-distribution image data based on the initial image data and the in-distribution image data, Inputting the out-of-distribution image data and the semantic segmentation result into a trained out-of-distribution detection model to obtain out-of-distribution obstacle information.

5. An out-of-distribution obstacle detection device, characterized in that, Including: An acquisition module for obtaining initial image data collected by a vehicle-mounted camera; A semantic segmentation module for inputting the initial image data into a trained semantic segmentation model to obtain a semantic segmentation result; the trained semantic segmentation model cannot recognize out-of-distribution objects; An in-distribution image determination module for inputting the semantic segmentation result into a trained image reconstruction model to obtain in-distribution image data; the in-distribution image data includes objects that the trained semantic segmentation model can recognize; An obstacle information determination module for inputting the initial image data and the in-distribution image data into a trained out-of-distribution detection model to obtain out-of-distribution obstacle information, where the out-of-distribution obstacle information includes unknown objects that the trained semantic segmentation model cannot recognize and the position information of such objects.

6. A vehicle control method, characterized in that, Including: Inputting initial image data into a trained in-distribution detection model to obtain in-distribution obstacle information, and determining out-of-distribution obstacle information according to the out-of-distribution obstacle detection method according to any one of claims 1-4; Controlling the vehicle to drive around obstacles, where the obstacles include in-distribution obstacles corresponding to the in-distribution obstacle information and / or out-of-distribution obstacles corresponding to the out-of-distribution obstacle information.

7. A vehicle control device, characterized in that, Including: An obstacle detection module, configured to input initial image data into a trained in-distribution detection model to obtain in-distribution obstacle information, and determine out-of-distribution obstacle information according to the out-of-distribution obstacle detection method described in any one of claims 1-4; A vehicle control module, configured to control the vehicle to drive around obstacles, where the obstacles include in-distribution obstacles corresponding to the in-distribution obstacle information, and / or out-of-distribution obstacles corresponding to the out-of-distribution obstacle information.

8. A vehicle, characterized in that, The vehicle includes: One or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the out-of-distribution obstacle detection method described in any one of claims 1-4 or the vehicle control method described in claim 6.

9. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to execute the out-of-distribution obstacle detection method described in any one of claims 1-4 or the vehicle control method described in claim 6.

Images