Ramp target detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a ramp target detection method and device, electronic equipment and a storage medium, and the method comprises the steps: detecting a to-be-detected image through a target detection model obtained based on pre-training, determining the position information of a front vehicle, and obtaining the linear distance between the front vehicle and a vehicle; moreover, the to-be-detected image can be detected based on a ramp detection model obtained through pre-training, an included angle between a ramp and a horizontal road surface included in the to-be-detected image and a self-vehicle distance from the self-vehicle to an uphill point are determined, and furthermore, the to-be-detected image can be detected based on a triangle formed by the self-vehicle, a front vehicle and the uphill point. The actual road distance between the vehicle and the front vehicle is calculated according to the linear distance, the vehicle distance and the included angle, the distance is the accurate distance between the vehicle and the front vehicle, and therefore the accuracy of ramp target detection can be improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and more specifically, to a ramp target detection method, device, electronic device and storage medium. Background Art

[0002] At present, autonomous driving environment perception mainly relies on sensors such as cameras to obtain data, and then uses algorithms to process, extract, and model the sensor data to obtain information about the vehicle's surrounding environment.

[0003] However, in the above environmental perception process, the imaging principle of the camera lacks 3D information, which causes the relative positions of objects in different 3D positions on the image to be close or even overlap. This leads to the problem of inaccurate slope detection in autonomous vehicles, that is, when the autonomous vehicle is driving on a slope, due to the inaccurate 3D position of the perception result of the vehicle in front, it will be significantly closer. Figure 1 As shown in the figure, the vehicle 10 perceives that the distance between it and the vehicle 20 in front is 101, while the actual distance between the vehicle and the vehicle in front should be the sum of the road distances 102 and 103. From the triangular relationship, it can be seen that the distance 101 is smaller than the sum of the road distances 102 and 103. In other words, the perception result is obviously too close, which may cause the system to misjudge that the vehicle in front is a rapidly decelerating or stationary vehicle, and ultimately cause the automatic driving vehicle to brake incorrectly, or even cause serious safety problems such as rear-end collision. Therefore, how to accurately detect ramp targets has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides a ramp target detection method, device, electronic device and storage medium to improve the accuracy of ramp target detection. The specific technical solution is as follows.

[0005] In a first aspect, an embodiment of the present application provides a ramp target detection method, the method comprising:

[0006] Obtain the image to be detected collected by the front-view camera of the autonomous driving vehicle;

[0007] The image to be detected is processed based on a pre-trained target detection model to determine the position information of the vehicle in front of the autonomous driving vehicle; the target detection model is trained based on a first sample image with vehicle position information annotated;

[0008] The image to be detected is processed based on a pre-trained ramp detection model to determine the angle between the ramp and the horizontal road surface included in the image, and the self-driving vehicle distance from the intersection of the ramp and the horizontal road surface; the ramp detection model is trained based on a second sample image that is labeled with whether there is a ramp, the true value of the ramp angle, and the true value of the self-driving vehicle distance;

[0009] Obtaining the current position of the autonomous driving vehicle, and calculating the straight-line distance between the autonomous driving vehicle and the preceding vehicle based on the current position of the autonomous driving vehicle and the position information of the preceding vehicle;

[0010] According to the straight-line distance, the angle, and the self-vehicle distance, based on a preset trigonometric function relationship, the actual road distance between the autonomous driving vehicle and the preceding vehicle is calculated.

[0011] Optionally, the step of calculating the actual road distance between the autonomous driving vehicle and the leading vehicle based on a preset trigonometric function relationship according to the straight-line distance, the angle, and the self-vehicle distance includes:

[0012] Construct a triangle with the straight-line distance as the longest side, the own vehicle distance and the front vehicle distance as the other two sides, and the included angle being the angle between the own vehicle distance and the front vehicle distance; the front vehicle distance being the distance between the front vehicle and the intersection;

[0013] Taking the straight-line distance, the distance from the vehicle itself, and the angle as known quantities, the distance to the vehicle ahead is calculated based on the trigonometric cosine theorem;

[0014] The sum of the self-vehicle distance and the front vehicle distance is taken as the actual road distance between the autonomous driving vehicle and the front vehicle.

[0015] Optionally, the step of processing the image to be detected based on the pre-trained ramp detection model to determine the angle between the ramp included in the image and the horizontal road surface includes:

[0016] Based on the pre-trained ramp detection model, the depth information of each road point in the image to be detected is detected to identify the intersection between the ramp and the horizontal road surface included in the image;

[0017] Based on the intersection, a preset mean square error regression method is used to calculate the angle between the ramp and the horizontal road surface.

[0018] Optionally, the training process of the target detection model includes:

[0019] Build an initial target detection model;

[0020] Acquire first sample images and first annotation results of each of the first sample images; the first annotation results include vehicle position information; the first sample images include ramp scenes;

[0021] Processing each labeled first sample image based on the initial target detection model to obtain a target detection result corresponding to each first sample image, and determining whether the accuracy of the initial target detection model is greater than a first preset threshold according to the target detection result and the corresponding first labeling result;

[0022] When the accuracy of the initial target detection model is not greater than the first preset threshold, adjusting the parameters of the initial target detection model, and returning to execute the step of processing each labeled first sample image based on the initial target detection model;

[0023] When the accuracy of the initial target detection model is greater than the first preset threshold, the current initial target detection model is used as the target detection model.

[0024] Optionally, the method further includes:

[0025] Collecting test sample images corresponding to different scenes, and evaluating the target detection results of the target detection model based on the test sample images;

[0026] The target scene category corresponding to the target detection result whose statistical error rate is greater than a preset threshold;

[0027] Collect target sample images corresponding to the target scene category, and update the target detection model based on the target sample images.

[0028] Optionally, the training process of the ramp detection model includes:

[0029] Construct an initial ramp detection model;

[0030] Acquire second sample images and second annotation results of each of the second sample images, wherein the second annotation results include whether there is a ramp, a true value of the ramp angle, and a true value of the self-vehicle distance;

[0031] Processing each labeled second sample image based on the initial ramp detection model to obtain a prediction result corresponding to each second sample image, and determining whether the accuracy of the initial ramp detection model is greater than a second preset threshold according to the prediction result and the corresponding second annotation result;

[0032] When the accuracy of the initial ramp detection model is not greater than the second preset threshold, adjusting the parameters of the initial ramp detection model, and returning to execute the step of processing each labeled second sample image based on the initial ramp detection model;

[0033] When the accuracy of the initial ramp detection model is greater than the second preset threshold, the current initial ramp detection model is used as the ramp detection model.

[0034] Optionally, before processing the image to be detected based on the pre-trained target detection model, the method further includes:

[0035] The image to be detected is preprocessed, and the preprocessing includes at least one of the following: image enhancement, image cropping, and cylindrical transformation.

[0036] In a second aspect, an embodiment of the present invention provides a ramp target detection device, the device comprising:

[0037] An image acquisition module, used to acquire the image to be detected collected by the front-view camera of the autonomous driving vehicle;

[0038] A target detection module, used to process the image to be detected based on a pre-trained target detection model to determine the position information of the vehicle in front of the autonomous driving vehicle; the target detection model is trained based on a first sample image with vehicle position information annotated;

[0039] a ramp detection module, configured to process the image to be detected based on a pre-trained ramp detection model, and determine an angle between a ramp and a horizontal road surface included in the image, and a self-vehicle distance of the autonomous driving vehicle from an intersection of the ramp and the horizontal road surface; the ramp detection model is trained based on a second sample image on which a ramp, a true value of the ramp angle, and a true value of the self-vehicle distance are annotated;

[0040] a distance calculation module, used to obtain the current position of the autonomous driving vehicle, and calculate the straight-line distance between the autonomous driving vehicle and the preceding vehicle according to the current position of the autonomous driving vehicle and the position information of the preceding vehicle;

[0041] The distance calculation module is also used to calculate the actual road distance between the autonomous driving vehicle and the leading vehicle based on the preset trigonometric function relationship according to the straight-line distance, the angle, and the self-vehicle distance.

[0042] Optionally, the distance calculation module includes:

[0043] A triangle construction submodule is used to construct a triangle with the straight-line distance as the longest side, the own vehicle distance and the front vehicle distance as the other two sides, and the included angle being the angle between the own vehicle distance and the front vehicle distance; the front vehicle distance is the distance between the front vehicle and the intersection;

[0044] A preceding vehicle distance calculation submodule, configured to calculate the preceding vehicle distance based on the trigonometric cosine theorem by taking the straight-line distance, the self-vehicle distance, and the angle as known quantities;

[0045] The road distance calculation submodule is used to use the sum of the self-vehicle distance and the front vehicle distance as the actual road distance between the autonomous driving vehicle and the front vehicle.

[0046] Optionally, the ramp detection module includes:

[0047] An intersection recognition submodule is used to detect the depth information of each road point in the image to be detected based on a pre-trained ramp detection model, and identify the intersection between the ramp and the horizontal road surface included in the image;

[0048] The angle calculation submodule is used to calculate the angle between the ramp and the horizontal road surface based on the intersection using a preset mean square error regression device.

[0049] Optionally, the device further includes: a target detection model training module, wherein the target detection model training module is specifically used to:

[0050] Build an initial target detection model;

[0051] Acquire first sample images and first annotation results of each of the first sample images; the first annotation results include vehicle position information; the first sample images include ramp scenes;

[0052] Processing each labeled first sample image based on the initial target detection model to obtain a target detection result corresponding to each first sample image, and determining whether the accuracy of the initial target detection model is greater than a first preset threshold according to the target detection result and the corresponding first labeling result;

[0053] When the accuracy of the initial target detection model is not greater than the first preset threshold, adjusting the parameters of the initial target detection model, and returning to execute the step of processing each labeled first sample image based on the initial target detection model;

[0054] When the accuracy of the initial target detection model is greater than the first preset threshold, the current initial target detection model is used as the target detection model.

[0055] Optionally, the device further comprises:

[0056] A model evaluation module, used to collect test sample images corresponding to different scenes, and evaluate the target detection results of the target detection model based on the test sample images;

[0057] A scene category statistics module, used to count the target scene categories corresponding to the target detection results whose error rate is greater than a preset threshold;

[0058] The model updating module is used to collect target sample images corresponding to the target scene category and update the target detection model based on the target sample images.

[0059] Optionally, the device further includes: a ramp detection model training module, wherein the ramp detection model training module is specifically used to:

[0060] Construct an initial ramp detection model;

[0061] Acquire second sample images and second annotation results of each of the second sample images, wherein the second annotation results include whether there is a ramp, a true value of the ramp angle, and a true value of the self-vehicle distance;

[0062] Processing each labeled second sample image based on the initial ramp detection model to obtain a prediction result corresponding to each second sample image, and determining whether the accuracy of the initial ramp detection model is greater than a second preset threshold according to the prediction result and the corresponding second annotation result;

[0063] When the accuracy of the initial ramp detection model is not greater than the second preset threshold, adjusting the parameters of the initial ramp detection model, and returning to execute the step of processing each labeled second sample image based on the initial ramp detection model;

[0064] When the accuracy of the initial ramp detection model is greater than the second preset threshold, the current initial ramp detection model is used as the ramp detection model.

[0065] Optionally, the device further comprises:

[0066] The image preprocessing module is used to preprocess the image to be detected, and the preprocessing includes at least one of the following: image enhancement, image cropping, and cylindrical transformation.

[0067] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the ramp target detection method as described in the first aspect above is implemented.

[0068] In a fourth aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0069] one or more processors;

[0070] The processor is coupled to a storage device, and the storage device is used to store one or more programs.

[0071] When the one or more programs are executed by the one or more processors, the electronic device implements the ramp target detection method described in the first aspect above.

[0072] In a fifth aspect, an embodiment of the present application provides a vehicle, wherein the vehicle includes the electronic device as described in the fourth aspect above.

[0073] In a sixth aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the ramp target detection method described in the first aspect.

[0074] It can be seen from the above content that a ramp target detection method, device, electronic device and storage medium provided in the embodiments of the present application can detect the image to be detected based on the target detection model obtained by pre-training, determine the position information of the front vehicle, and then obtain the straight-line distance between the front vehicle and the own vehicle; and, based on the ramp detection model obtained by pre-training, detect the image to be detected, determine the angle between the ramp and the horizontal road surface included in the image to be detected and the distance of the own vehicle from the uphill point; further, based on the triangle formed by the three points of the own vehicle, the front vehicle, and the uphill point, the actual road distance between the own vehicle and the front vehicle can be calculated according to the above straight-line distance, the own vehicle distance, and the angle, and this distance is the accurate spacing between the own vehicle and the front vehicle, thereby improving the accuracy of ramp target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work.

[0076] Figure 1 It is a schematic diagram of the distance detection result between the front vehicle and the vehicle in the known technology;

[0077] Figure 2 A schematic diagram of a process flow of a ramp target detection method provided in an embodiment of the present application;

[0078] Figure 3 This is a schematic diagram of the distance detection result between the front vehicle and the vehicle in an uphill scenario in an embodiment of the present application;

[0079] Figure 4 This is a schematic diagram of the distance detection result between the front vehicle and the vehicle in a downhill scenario in an embodiment of the present application;

[0080] Figure 5 A schematic diagram of the structure of a ramp target detection device provided in an embodiment of the present application;

[0081] Figure 6A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0082] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0083] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0084] The embodiment of the present application discloses a ramp target detection method, device, electronic device and storage medium, which can improve the accuracy of ramp target detection. The embodiment of the present application is described in detail below.

[0085] Figure 2 A flowchart of a ramp target detection method provided in an embodiment of the present application is shown. The method can be applied to an autonomous driving vehicle, and specifically can be applied to a processor in an autonomous driving vehicle, such as Figure 2 As shown, the method may include the following steps:

[0086] S110: Acquire an image to be detected captured by a front-view camera of the autonomous driving vehicle.

[0087] During road driving, the autonomous vehicle can obtain the environmental image of its surrounding environment in real time to perceive the environment and plan the path according to the perception results. It can be understood that the vehicle in front (also called the front vehicle) is the vehicle that has a greater impact on the autonomous vehicle (also called the self-driving vehicle). Its operating state, such as the current speed of the front vehicle and the distance between the front vehicle and the self-driving vehicle, determines the subsequent operating state of the self-driving vehicle, such as acceleration and speed, to the greatest extent.

[0088] In the embodiment of the present application, for the scene of ramp target detection, the vehicle can obtain the image captured by its front-view camera as the image to be detected. The image to be detected must include the front vehicle, so by detecting the image to be detected, the distance between the front vehicle and the vehicle can be determined.

[0089] S120: Processing the image to be detected based on the pre-trained target detection model to determine the position information of the vehicle in front of the autonomous driving vehicle; the target detection model is trained based on the first sample image with the vehicle position information annotated.

[0090] In an embodiment of the present application, in order to accurately detect the image to be detected, a target detection model can be pre-trained. The target detection model is a neural network model that can detect the target of the input image, and its output result is the position information of the vehicle in the input image. Specifically, the target detection model can extract high-dimensional features of the input image, and then perform a final detection to determine the position information of the vehicle in the image, that is, the position information of the preceding vehicle.

[0091] In one implementation, the training process of the above-mentioned target detection model may include the following steps: constructing an initial target detection model; obtaining a first sample image and a first annotation result of each first sample image; the first annotation result includes vehicle location information; the first sample image includes a ramp scene; processing each annotated first sample image based on the initial target detection model to obtain a target detection result corresponding to each first sample image, and determining whether the accuracy of the initial target detection model is greater than a first preset threshold based on the target detection result and the corresponding first annotation result; when the accuracy of the initial target detection model is not greater than the first preset threshold, adjusting the parameters of the initial target detection model, and returning to execute the step of processing each annotated first sample image based on the initial target detection model; when the accuracy of the initial target detection model is greater than the first preset threshold, using the current initial target detection model as the target detection model.

[0092] That is to say, the parameters of the initial target detection model can be adjusted multiple times based on the first sample image, and finally a target detection model that meets the accuracy requirements is obtained.

[0093] Among them, the above-mentioned initial target detection model may include a resnet-18 network model. The first sample image may include an environmental image captured by a camera installed on an autonomous driving vehicle. The labeling process of the first sample image may be performed manually. The above-mentioned preset threshold value may be specifically set according to the number of first sample images or the accuracy requirement of the model, and the embodiment of the present application does not limit its specific value.

[0094] It is understandable that the accuracy of target detection may be different for different ramp scenes, such as viaducts, ramps, underground garage entrances, underground garage exits, etc. For example, compared with viaducts and ramps, underground garage entrances or underground garage exits may have lower target detection accuracy due to larger changes in ambient brightness. In the embodiment of the present application, in order to improve the accuracy of the target detection model, the target detection model can be corrected for special scenes.

[0095] In one embodiment, the target detection model can be iteratively updated online or offline. Specifically, test sample images corresponding to different scenes can be collected, and the target detection results of the target detection model can be evaluated based on the test sample images; then, the target scene categories corresponding to the target detection results with an error rate greater than a preset threshold can be counted; finally, the target sample images corresponding to the target scene categories are collected, and the target detection model is updated based on the target sample images.

[0096] Among them, when evaluating the target detection results of the target detection model based on the test sample images, for example, the test sample images can be manually annotated in advance. After the target detection model performs target detection on each test sample image, the detection results and the annotation results are compared to determine the detection accuracy of each test sample image.

[0097] S130: Processing the image to be detected based on the pre-trained ramp detection model to determine the angle between the ramp and the horizontal road surface included in the image, and the distance of the autonomous driving vehicle from the intersection of the ramp and the horizontal road surface; the ramp detection model is trained based on a second sample image that is labeled with whether there is a ramp, the true value of the ramp angle, and the true value of the distance from the vehicle.

[0098] In the embodiment of the present application, in order to accurately detect the ramp target, a ramp detection model can be pre-trained, and the model can detect the input image. When the image includes a ramp, the angle between the ramp and the horizontal road surface included in the image and the distance of the vehicle from the intersection of the ramp and the horizontal road surface can be detected, which can be called the vehicle distance. For example, the ramp detection model can extract high-dimensional features of the input image, and then perform a final detection to obtain the angle between the ramp and the horizontal road surface included in the image, and the vehicle distance.

[0099] In one implementation, when detecting the angle, the depth information of each road point in the image to be detected can be first detected based on the ramp detection model to identify the intersection between the ramp and the horizontal road surface included in the image; then, based on the intersection, a preset MSE (mean squared error) regression method can be used to obtain the angle between the ramp and the horizontal road surface.

[0100] In one implementation, the training process of the ramp detection model may include the following steps: constructing an initial ramp detection model; obtaining a second sample image and a second annotation result of each second sample image, the second annotation result including whether there is a ramp, a true value of the ramp angle, and a true value of the vehicle distance; processing each annotated second sample image based on the initial ramp detection model to obtain a prediction result corresponding to each second sample image, and determining whether the accuracy of the initial ramp detection model is greater than a second preset threshold value based on the prediction result and the corresponding second annotation result; when the accuracy of the initial ramp detection model is not greater than the second preset threshold value, adjusting the parameters of the initial ramp detection model, and returning to execute the step of processing each annotated second sample image based on the initial ramp detection model; when the accuracy of the initial ramp detection model is greater than the second preset threshold value, using the current initial ramp detection model as the ramp detection model.

[0101] Among them, the ramp detection model can first detect whether there is a ramp based on the following function:

[0102] FL(pt)=-(1-pt) γ log(pt)

[0103]

[0104] Among them, p represents the probability of y=1, y represents the true label, pt represents the intermediate conversion variable, FL(pt) represents the loss h function, and γ represents the pre-set adjustment parameter.

[0105] The ramp detection model can regress the ramp slope according to the following formula to obtain the angle between the ramp and the horizontal road surface:

[0106]

[0107] Wherein, m represents the sample size, that is, the number of the second sample images; SSE represents the sum variance; i represents the sample number; y represents the predicted value of the angle between the ramp and the horizontal road surface; Represents the true value of the angle between the ramp and the horizontal road surface.

[0108] Among them, the above-mentioned initial ramp detection model may include network models such as ResNet-18 or MobileNetV2. The second sample image may include any image, such as an environmental image captured by a camera installed on an autonomous driving vehicle. The annotation process of the second sample image may be performed manually. The above-mentioned second preset threshold may be specifically set according to the number of second sample images or the accuracy requirements of the model, and the embodiment of the present application does not limit its specific value.

[0109] Furthermore, the first sample image and the second sample image may be the same or different, and the second preset threshold and the first preset threshold may be the same or different, which are all possible and are not limited in this embodiment of the present application.

[0110] S140: Obtain the current position of the autonomous driving vehicle, and calculate the straight-line distance between the autonomous driving vehicle and the vehicle in front based on the current position of the autonomous driving vehicle and the position information of the vehicle in front.

[0111] For example, the vehicle can be located based on automatic positioning technologies such as GPS (Global Positioning System) to determine its current position, that is, its three-dimensional coordinates at the current moment. Then, based on the three-dimensional coordinates of the vehicle and the three-dimensional coordinates of the vehicle in front, the straight-line distance between the vehicle and the vehicle in front can be calculated. Figure 1 For example, the distance 101 in the figure is the straight-line distance between the vehicle and the front vehicle.

[0112] S150: Calculate the actual road distance between the autonomous driving vehicle and the vehicle ahead based on the straight-line distance, the angle, and the self-vehicle distance and a preset trigonometric function relationship.

[0113] In one implementation, the actual road distance between the vehicle and the front vehicle can be calculated according to the following steps: construct a triangle with the straight-line distance as the longest side, the vehicle distance and the front vehicle distance as the other two sides, and the angle between the vehicle distance and the front vehicle distance; the front vehicle distance is the distance between the front vehicle and the intersection; the straight-line distance, the vehicle distance, and the angle are used as known quantities, and the front vehicle distance is calculated based on the trigonometric cosine theorem; the sum of the vehicle distance and the front vehicle distance is used as the actual road distance between the autonomous driving vehicle and the front vehicle.

[0114] For example, Figure 3 As shown in the figure, for the uphill scene, the position of the front vehicle is obtained based on the target detection model, and the straight-line distance a is calculated. The distance b and the angle C are obtained by slope recognition. The distance c can be calculated based on the following cosine theorem:

[0115] c 2 =a 2 +b 2 -2abCOS(C)

[0116] The sum of distance b and distance c is the actual road distance between the vehicle and the front vehicle.

[0117] like Figure 4 As shown in the figure, for the downhill scenario, the position of the vehicle in front is obtained based on the target detection model, and the straight-line distance a is calculated. The distance b and the angle C are obtained by slope recognition. The distance c can be calculated based on the following cosine theorem:

[0118] c 2=a 2 +b 2 -2abCOS(C)

[0119] The sum of distance b and distance c is the actual road distance between the vehicle and the front vehicle.

[0120] A ramp target detection method provided in an embodiment of the present application can detect an image to be detected based on a pre-trained target detection model to determine the position information of the leading vehicle, and further obtain the straight-line distance between the leading vehicle and the own vehicle; and, based on the pre-trained ramp detection model, the image to be detected can be detected to determine the angle between the ramp and the horizontal road surface included in the image to be detected and the distance of the own vehicle from the uphill point. Furthermore, based on a triangle formed by the own vehicle, the leading vehicle, and the uphill point, the actual road distance between the own vehicle and the leading vehicle can be calculated according to the above-mentioned straight-line distance, the own vehicle distance, and the angle. This distance is the exact spacing between the own vehicle and the leading vehicle, thereby improving the accuracy of ramp target detection.

[0121] It is understandable that the image quality of the image to be detected will affect the accuracy of the recognition result of the network model for the image. As an implementation method of the present application, before the image to be detected is processed based on the pre-trained target detection model, the image to be detected can also be preprocessed, and the preprocessing includes at least one of the following: image enhancement, image cropping, and cylindrical transformation.

[0122] Image enhancement, also known as image sharpening, is often caused by averaging or integration of the image. Therefore, if a differential operation is performed on it, details such as edges can be made clear, which is image sharpening in the spatial domain. In this way, the clarity of the image to be detected can be improved, thereby improving the accuracy of ramp target detection.

[0123] Image cropping is to crop the image to a uniform size so that the size of the image to be detected can match the network model and improve the recognition accuracy of the model.

[0124] Cylindrical transformation is to project the image onto a cylinder. In order to maintain the consistency between the spatial constraints and vision in the image, cylindrical projection is required. Otherwise, the farther the image is from the center image, the greater the deformation will be after splicing. By cropping the image, the image quality of the image to be detected can be improved, thereby improving the accuracy of ramp target detection.

[0125] Please refer to Figure 5 , which shows a schematic diagram of the structure of a ramp target detection device provided in an embodiment of the present application, such as Figure 5 As shown, the ramp target detection device comprises:

[0126] An image acquisition module 510 is used to acquire an image to be detected collected by a front-view camera of the autonomous driving vehicle;

[0127] The target detection module 520 is used to process the image to be detected based on a pre-trained target detection model to determine the position information of the vehicle in front of the autonomous driving vehicle; the target detection model is trained based on the first sample image with the vehicle position information annotated;

[0128] A ramp detection module 530 is used to process the image to be detected based on a pre-trained ramp detection model to determine the angle between the ramp and the horizontal road surface included in the image, and the self-driving vehicle's distance from the intersection of the ramp and the horizontal road surface; the ramp detection model is trained based on a second sample image that is labeled with whether there is a ramp, a true value of the ramp angle, and a true value of the self-driving vehicle distance;

[0129] A distance calculation module 540 is used to obtain the current position of the autonomous driving vehicle and calculate the straight-line distance between the autonomous driving vehicle and the preceding vehicle according to the current position of the autonomous driving vehicle and the position information of the preceding vehicle;

[0130] The distance calculation module 540 is also used to calculate the actual road distance between the autonomous driving vehicle and the leading vehicle based on the preset trigonometric function relationship according to the straight-line distance, the angle, and the self-vehicle distance.

[0131] Optionally, the distance calculation module 540 includes:

[0132] A triangle construction submodule is used to construct a triangle with the straight-line distance as the longest side, the own vehicle distance and the front vehicle distance as the other two sides, and the included angle being the angle between the own vehicle distance and the front vehicle distance; the front vehicle distance is the distance between the front vehicle and the intersection;

[0133] A preceding vehicle distance calculation submodule, configured to calculate the preceding vehicle distance based on the trigonometric cosine theorem by taking the straight-line distance, the self-vehicle distance, and the angle as known quantities;

[0134] The road distance calculation submodule is used to use the sum of the self-vehicle distance and the front vehicle distance as the actual road distance between the autonomous driving vehicle and the front vehicle.

[0135] Optionally, the ramp detection module 530 includes:

[0136] An intersection recognition submodule is used to detect the depth information of each road point in the image to be detected based on a pre-trained ramp detection model, and identify the intersection between the ramp and the horizontal road surface included in the image;

[0137] The angle calculation submodule is used to calculate the angle between the ramp and the horizontal road surface based on the intersection using a preset mean square error regression device.

[0138] Optionally, the device further includes: a target detection model training module, wherein the target detection model training module is specifically used to:

[0139] Build an initial target detection model;

[0140] Acquire first sample images and first annotation results of each of the first sample images; the first annotation results include vehicle position information; the first sample images include ramp scenes;

[0141] Processing each labeled first sample image based on the initial target detection model to obtain a target detection result corresponding to each first sample image, and determining whether the accuracy of the initial target detection model is greater than a first preset threshold according to the target detection result and the corresponding first labeling result;

[0142] When the accuracy of the initial target detection model is not greater than the first preset threshold, adjusting the parameters of the initial target detection model, and returning to execute the step of processing each labeled first sample image based on the initial target detection model;

[0143] When the accuracy of the initial target detection model is greater than the first preset threshold, the current initial target detection model is used as the target detection model.

[0144] Optionally, the device further comprises:

[0145] A model evaluation module, used to collect test sample images corresponding to different scenes, and evaluate the target detection results of the target detection model based on the test sample images;

[0146] A scene category statistics module, used to count the target scene categories corresponding to the target detection results whose error rate is greater than a preset threshold;

[0147] The model updating module is used to collect target sample images corresponding to the target scene category and update the target detection model based on the target sample images.

[0148] Optionally, the device further includes: a ramp detection model training module, wherein the ramp detection model training module is specifically used to:

[0149] Construct an initial ramp detection model;

[0150] Acquire second sample images and second annotation results of each of the second sample images, wherein the second annotation results include whether there is a ramp, a true value of the ramp angle, and a true value of the self-vehicle distance;

[0151] Processing each labeled second sample image based on the initial ramp detection model to obtain a prediction result corresponding to each second sample image, and determining whether the accuracy of the initial ramp detection model is greater than a second preset threshold according to the prediction result and the corresponding second annotation result;

[0152] When the accuracy of the initial ramp detection model is not greater than the second preset threshold, adjusting the parameters of the initial ramp detection model, and returning to execute the step of processing each labeled second sample image based on the initial ramp detection model;

[0153] When the accuracy of the initial ramp detection model is greater than the second preset threshold, the current initial ramp detection model is used as the ramp detection model.

[0154] Optionally, the device further comprises:

[0155] The image preprocessing module is used to preprocess the image to be detected, and the preprocessing includes at least one of the following: image enhancement, image cropping, and cylindrical transformation.

[0156] A ramp target detection device provided in an embodiment of the present application can detect an image to be detected based on a pre-trained target detection model to determine the position information of the leading vehicle, and further obtain the straight-line distance between the leading vehicle and the own vehicle; and, based on the pre-trained ramp detection model, the image to be detected can be detected to determine the angle between the ramp and the horizontal road surface included in the image to be detected and the distance of the own vehicle from the uphill point. Furthermore, based on a triangle formed by the own vehicle, the leading vehicle, and the uphill point, the actual road distance between the own vehicle and the leading vehicle can be calculated according to the above-mentioned straight-line distance, the own vehicle distance, and the angle. This distance is the exact spacing between the own vehicle and the leading vehicle, thereby improving the accuracy of ramp target detection.

[0157] The above device embodiment corresponds to the method embodiment and has the same technical effect as the method embodiment. For specific description, please refer to the method embodiment. The device embodiment is obtained based on the method embodiment. For specific description, please refer to the method embodiment part, which will not be repeated here.

[0158] The present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following Figure 2 The ramp target detection method.

[0159] like Figure 6 As shown, the present application provides an electronic device, the electronic device comprising:

[0160] one or more processors 40;

[0161] The processor 40 is coupled to a storage device 41, and the storage device 41 is used to store one or more programs.

[0162] When the one or more programs are executed by the one or more processors 40, the electronic device implements the following Figure 2 The ramp target detection method.

[0163] The present application embodiment provides a vehicle, wherein the vehicle comprises: Figure 6 The electronic device.

[0164] The present application provides a computer program product, the computer program product comprising a computer program, the computer program when executed by a processor to achieve the following Figure 2 The ramp target detection method.

[0165] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of one embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present application.

[0166] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A ramp target detection method, characterized in that: The method comprises: Obtain the image to be detected collected by the front-view camera of the autonomous driving vehicle; The image to be detected is processed based on a pre-trained target detection model to determine the position information of the vehicle in front of the autonomous driving vehicle; the target detection model is trained based on a first sample image with vehicle position information annotated; The image to be detected is processed based on a pre-trained ramp detection model to determine the angle between the ramp and the horizontal road surface included in the image, and the self-driving vehicle distance from the intersection of the ramp and the horizontal road surface; the ramp detection model is trained based on a second sample image that is labeled with whether there is a ramp, the true value of the ramp angle, and the true value of the self-driving vehicle distance; Obtaining the current position of the autonomous driving vehicle, and calculating the straight-line distance between the autonomous driving vehicle and the preceding vehicle based on the current position of the autonomous driving vehicle and the position information of the preceding vehicle; According to the straight-line distance, the angle, and the self-vehicle distance, based on a preset trigonometric function relationship, the actual road distance between the autonomous driving vehicle and the preceding vehicle is calculated.

2. The method according to claim 1, characterized in that The step of calculating the actual road distance between the autonomous driving vehicle and the preceding vehicle based on the straight-line distance, the angle, and the self-vehicle distance and a preset trigonometric function relationship comprises: Construct a triangle with the straight-line distance as the longest side, the own vehicle distance and the front vehicle distance as the other two sides, and the included angle being the angle between the own vehicle distance and the front vehicle distance; the front vehicle distance being the distance between the front vehicle and the intersection; Taking the straight-line distance, the distance from the vehicle itself, and the angle as known quantities, the distance to the vehicle ahead is calculated based on the trigonometric cosine theorem; The sum of the self-vehicle distance and the front vehicle distance is taken as the actual road distance between the autonomous driving vehicle and the front vehicle.

3. The method according to claim 1, characterized in that The step of processing the image to be detected based on the pre-trained ramp detection model to determine the angle between the ramp included in the image and the horizontal road surface comprises: Based on the pre-trained ramp detection model, the depth information of each road point in the image to be detected is detected to identify the intersection between the ramp and the horizontal road surface included in the image; Based on the intersection, a preset mean square error regression method is used to calculate the angle between the ramp and the horizontal road surface.

4. The method according to claim 1, characterized in that The training process of the target detection model includes: Build an initial target detection model; Acquire first sample images and first annotation results of each of the first sample images; the first annotation results include vehicle position information; the first sample images include ramp scenes; Processing each labeled first sample image based on the initial target detection model to obtain a target detection result corresponding to each first sample image, and determining whether the accuracy of the initial target detection model is greater than a first preset threshold according to the target detection result and the corresponding first labeling result; When the accuracy of the initial target detection model is not greater than the first preset threshold, adjusting the parameters of the initial target detection model, and returning to execute the step of processing each labeled first sample image based on the initial target detection model; When the accuracy of the initial target detection model is greater than the first preset threshold, the current initial target detection model is used as the target detection model.

5. The method according to claim 4, characterized in that The method further comprises: Collecting test sample images corresponding to different scenes, and evaluating the target detection results of the target detection model based on the test sample images; The target scene category corresponding to the target detection result whose statistical error rate is greater than a preset threshold; Collect target sample images corresponding to the target scene category, and update the target detection model based on the target sample images.

6. The method according to claim 1, characterized in that The training process of the ramp detection model includes: Construct an initial ramp detection model; Acquire second sample images and second annotation results of each of the second sample images, wherein the second annotation results include whether there is a ramp, a true value of the ramp angle, and a true value of the self-vehicle distance; Processing each labeled second sample image based on the initial ramp detection model to obtain a prediction result corresponding to each second sample image, and determining whether the accuracy of the initial ramp detection model is greater than a second preset threshold according to the prediction result and the corresponding second annotation result; When the accuracy of the initial ramp detection model is not greater than the second preset threshold, adjusting the parameters of the initial ramp detection model, and returning to execute the step of processing each labeled second sample image based on the initial ramp detection model; When the accuracy of the initial ramp detection model is greater than the second preset threshold, the current initial ramp detection model is used as the ramp detection model.

7. The method according to any one of claims 1 to 6, characterized in that: Before processing the image to be detected based on the pre-trained target detection model, the method further includes: The image to be detected is preprocessed, and the preprocessing includes at least one of the following: image enhancement, image cropping, and cylindrical transformation.

8. A ramp target detection device, characterized in that: The device comprises: An image acquisition module, used to acquire the image to be detected collected by the front-view camera of the autonomous driving vehicle; A target detection module, used to process the image to be detected based on a pre-trained target detection model to determine the position information of the vehicle in front of the autonomous driving vehicle; the target detection model is trained based on a first sample image with vehicle position information annotated; a ramp detection module, configured to process the image to be detected based on a pre-trained ramp detection model, and determine an angle between a ramp and a horizontal road surface included in the image, and a self-vehicle distance of the autonomous driving vehicle from an intersection of the ramp and the horizontal road surface; the ramp detection model is trained based on a second sample image on which a ramp, a true value of the ramp angle, and a true value of the self-vehicle distance are annotated; a distance calculation module, used to obtain the current position of the autonomous driving vehicle, and calculate the straight-line distance between the autonomous driving vehicle and the preceding vehicle according to the current position of the autonomous driving vehicle and the position information of the preceding vehicle; The distance calculation module is also used to calculate the actual road distance between the autonomous driving vehicle and the leading vehicle based on the preset trigonometric function relationship according to the straight-line distance, the angle, and the self-vehicle distance.

9. The device according to claim 8, characterized in that The distance calculation module comprises: A triangle construction submodule is used to construct a triangle with the straight-line distance as the longest side, the own vehicle distance and the front vehicle distance as the other two sides, and the included angle being the angle between the own vehicle distance and the front vehicle distance; the front vehicle distance is the distance between the front vehicle and the intersection; A preceding vehicle distance calculation submodule, configured to use the straight-line distance, the self-vehicle distance, and the angle as known quantities, and calculate the preceding vehicle distance based on the trigonometric cosine theorem; The road distance calculation submodule is used to use the sum of the self-driving vehicle distance and the front vehicle distance as the actual road distance between the autonomous driving vehicle and the front vehicle.

10. The device according to claim 8, characterized in that The ramp detection module comprises: An intersection recognition submodule is used to detect the depth information of each road point in the image to be detected based on a pre-trained ramp detection model, and identify the intersection between the ramp and the horizontal road surface included in the image; The angle calculation submodule is used to calculate the angle between the ramp and the horizontal road surface based on the intersection using a preset mean square error regression device.

11. The device according to claim 8, characterized in that The device further includes: a target detection model training module, wherein the target detection model training module is specifically used to: Build an initial target detection model; Acquire first sample images and first annotation results of each of the first sample images; the first annotation results include vehicle position information; the first sample images include ramp scenes; Processing each labeled first sample image based on the initial target detection model to obtain a target detection result corresponding to each first sample image, and determining whether the accuracy of the initial target detection model is greater than a first preset threshold according to the target detection result and the corresponding first labeling result; When the accuracy of the initial target detection model is not greater than the first preset threshold, adjusting the parameters of the initial target detection model, and returning to execute the step of processing each labeled first sample image based on the initial target detection model; When the accuracy of the initial target detection model is greater than the first preset threshold, the current initial target detection model is used as the target detection model.

12. The device according to claim 11, characterized in that The device also includes: A model evaluation module, used to collect test sample images corresponding to different scenes, and evaluate the target detection results of the target detection model based on the test sample images; A scene category statistics module, used to count the target scene categories corresponding to the target detection results whose error rate is greater than a preset threshold; The model updating module is used to collect target sample images corresponding to the target scene category and update the target detection model based on the target sample images.

13. The device according to claim 8, characterized in that The device further includes: a ramp detection model training module, wherein the ramp detection model training module is specifically used to: Construct an initial ramp detection model; Acquire second sample images and second annotation results of each of the second sample images, wherein the second annotation results include whether there is a ramp, a true value of the ramp angle, and a true value of the self-vehicle distance; Processing each labeled second sample image based on the initial ramp detection model to obtain a prediction result corresponding to each second sample image, and determining whether the accuracy of the initial ramp detection model is greater than a second preset threshold according to the prediction result and the corresponding second annotation result; When the accuracy of the initial ramp detection model is not greater than the second preset threshold, adjusting the parameters of the initial ramp detection model, and returning to execute the step of processing each labeled second sample image based on the initial ramp detection model; When the accuracy of the initial ramp detection model is greater than the second preset threshold, the current initial ramp detection model is used as the ramp detection model.

14. The device according to any one of claims 8 to 13, characterized in that: The device also includes: The image preprocessing module is used to preprocess the image to be detected, and the preprocessing includes at least one of the following: image enhancement, image cropping, and cylindrical transformation.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the ramp target detection method according to any one of claims 1 to 7 is implemented.

16. An electronic device, characterized in that: The electronic device comprises: one or more processors; The processor is coupled to a storage device, and the storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the electronic device implements the ramp target detection method as described in any one of claims 1-7.

17. A vehicle, characterized in that: The vehicle includes the electronic device as claimed in claim 16.

Images