Vehicle obstacle detection method, system, device, medium and program

Abstract

The application provides a vehicle obstacle detection method, system, device, medium and program, the method comprising: obtaining a projection image formed by a projection light ray on the ground under the current state of the vehicle; comparing the projection image with a preset projection image to determine an obstacle in the projection image and a position area of the obstacle; and determining the type of the obstacle according to the line change generated by the position area of the obstacle in the projection image to realize obstacle detection, wherein the type of the obstacle includes a protrusion and a fault, the application is not affected by the ambient brightness, and obstacle detection can be performed through the projection light ray even in a parking lot with relatively dark light, so that the detection accuracy of the obstacle is improved, an additional sensor is not needed for assistance, and the obstacle detection cost is greatly reduced.

Description

Technical Field

[0001] This application relates to the field of vehicle safety inspection, specifically to a method, system, equipment, medium, and procedure for vehicle obstacle detection. Background Technology

[0002] During fully automated parking, once the occupants leave the vehicle, the vehicle needs to find and park itself in a parking space. To ensure safe automated parking, accurate detection of obstacles in the current parking environment is necessary.

[0003] In related technologies, cameras are widely used sensors in automated parking systems. They receive light emitted from targets, form images, and then, through relevant algorithms, detect whether the targets in the images are obstacles. However, this method is a 2D image-based detection approach, which has weak detection capabilities for targets that are not trained or lack texture. Therefore, current vision-based obstacle detection methods have low accuracy and cannot meet the requirements of obstacle detection in existing vehicles.

[0004] Application content

[0005] In view of the shortcomings of the prior art described above, this application provides a vehicle obstacle detection method, system, device, medium and program to solve the problem of low detection accuracy caused by using a camera for obstacle detection.

[0006] In a first aspect, this application provides a vehicle obstacle detection method, comprising:

[0007] Acquire a projection image of the projected light rays on the ground in the current state of the vehicle, wherein the current state includes a driving state or a stationary state;

[0008] The projected image is compared with a preset projected image to determine the obstacles in the projected image and the location area of ​​the obstacles;

[0009] Obstacle detection is achieved by determining the type of obstacle based on the line changes generated by the obstacle in the location area of ​​the projection map. The types of obstacles include protrusions and faults.

[0010] In one embodiment of this application, after determining the type of obstacle and implementing obstacle detection, the method further includes:

[0011] If the obstacle is a protrusion, the height of the protruding obstacle is calculated based on the projection angle of the vehicle's projected light at the farthest point and the travel distance; wherein, the travel distance and the projection angle are determined by the moment when the deformation of the projected image occurs and the moment when the deformation is eliminated.

[0012] If the obstacle is a fault, the height of the fault obstacle is calculated based on the principle of similar triangles, combined with the projection of the vehicle and the distance of the acquired projection image in the horizontal and vertical directions.

[0013] In one embodiment of this application, comparing the projected image with a preset projected image to determine obstacles in the projected image and the location regions of the obstacles includes:

[0014] The projected image is compared with a preset projected image;

[0015] If the projected image has the same shape as the preset projected image, then it is determined that there are no obstacles in the projected image;

[0016] If the projected image differs from the preset projected image, the obstacle and its location region are determined based on the graphic offset between the projected image and the preset projected image.

[0017] In one embodiment of this application, obstacle detection is achieved by determining the type of the obstacle based on the line changes generated by the obstacle in the position area of ​​the projection image, including:

[0018] If the furthest distance of a line in the projected image is greater than the furthest distance of a line in the preset projected image, then the type of road obstacle is determined to be a fault.

[0019] If the furthest distance of a line in the projected image is closer than the furthest distance of a line in a preset projected image, then the type of road obstacle is determined to be a protrusion.

[0020] In one embodiment of this application, after determining the height of the obstacle, the method further includes:

[0021] The height of the obstacle is determined based on a preset height.

[0022] If the height of the obstacle is lower than the preset height, then it is determined that the vehicle can pass through the obstacle at that height;

[0023] If the height of the obstacle is not lower than the preset height, it is determined that the vehicle cannot pass through the obstacle at that height, and an alarm is issued to notify the vehicle.

[0024] In one embodiment of this application, after determining that the vehicle cannot pass through an obstacle of that height, the method further includes:

[0025] Obtain the closest distance between the vehicle and the obstacle;

[0026] The nearest distance is compared with a preset safe distance threshold. If the nearest distance is less than or equal to the preset safe distance threshold, an obstacle alarm is issued; if the nearest distance is greater than the preset safe distance threshold, no action is taken.

[0027] In one embodiment of this application, after determining the type of obstacle and implementing obstacle detection, the method further includes:

[0028] Acquire ultrasonic radar data, millimeter-wave radar data, and lidar data of the vehicle in its current state;

[0029] The lidar data, ultrasonic radar data, and millimeter-wave radar data are fused to determine environmental perception information;

[0030] A training set is generated by labeling the environmental perception information and obstacle detection results.

[0031] The pre-built convolutional neural network is trained based on the training set to obtain an intelligent decision-making model;

[0032] The environmental perception information and obstacle detection results are input into the intelligent decision-making model to output the corresponding control strategy.

[0033] In one embodiment of this application, after determining the type of obstacle and thus completing obstacle detection, the method further includes:

[0034] Based on visual deep learning algorithms, parking space identification is performed on the visual data of the current vehicle scene to determine visual parking space information and mark the first time point;

[0035] Based on machine learning algorithms, ultrasonic radar data of the current vehicle scene is used to identify parking spaces, determine spatial parking space information, and mark the second time point.

[0036] By fusing spatial parking space information and visual parking space information associated with the first and second time points at the same time, the target parking space information is obtained.

[0037] The system assists in automatic parking of the vehicle based on the target parking space information and the obstacle detection results between the target parking space information and the vehicle.

[0038] In a second aspect, this application provides a vehicle obstacle detection system, comprising:

[0039] The image acquisition module is used to acquire the projected image of the vehicle's current state on the ground, wherein the current state includes a driving state or a stationary state.

[0040] An image comparison module is used to compare the projected image with a preset projected image to determine the obstacles in the projected image and the location area of ​​the obstacles;

[0041] An obstacle detection module is used to determine the type of obstacle based on the line changes generated by the obstacle in the position area of ​​the projection map, wherein the type of obstacle includes protrusions and faults.

[0042] In a third aspect, this application provides an electronic device comprising:

[0043] One or more processors;

[0044] A storage device for storing one or more programs that, when executed by one or more processors, enable the electronic device to implement the vehicle obstacle detection method described above.

[0045] In a fourth aspect, this application provides a vehicle device that includes the aforementioned electronic equipment.

[0046] In a fifth aspect, this application provides a computer-readable storage medium storing computer-readable instructions that, when executed by a computer's processor, cause the computer to perform the aforementioned vehicle obstacle detection method.

[0047] In a sixth aspect, this application provides a computer program product or computer program, the computer program product or computer program including computer instructions stored in a computer-readable storage medium, a processor of an electronic device reading the computer instructions from the computer-readable storage medium, and the processor executing the computer instructions to cause the electronic device to perform the above-described vehicle obstacle detection method.

[0048] The beneficial effects of this application are as follows: This application acquires a projected image of the vehicle's current state projected light onto the ground, compares the projected image with a preset projected image, and determines the obstacles in the projected image and the location area of ​​the obstacles; based on the line changes produced by the obstacles in the location area of ​​the projected image, the type of the obstacle is determined, thus completing obstacle detection. Compared with other visual obstacle detection methods, this application is not affected by ambient brightness. Even in a dimly lit parking lot, obstacle detection can be performed using projected light, which not only improves the accuracy of obstacle detection and eliminates the need for additional sensors, but also greatly reduces the cost of obstacle detection.

[0049] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0051] Figure 1 This is a schematic diagram illustrating the implementation environment of a vehicle obstacle detection method according to an exemplary embodiment of this application;

[0052] Figure 2 This is a flowchart illustrating a vehicle obstacle detection method in an exemplary embodiment of this application;

[0053] Figure 3 This is a flowchart illustrating the obstacle height determination process in a vehicle obstacle detection method according to an exemplary embodiment of this application;

[0054] Figure 4 This is a flowchart illustrating a vehicle obstacle detection method for parking in an exemplary embodiment of this application;

[0055] Figure 5 This is a flowchart illustrating a vehicle obstacle detection method for low-speed driving, as shown in an exemplary embodiment of this application.

[0056] Figure 6 This is a flowchart illustrating the interrupted layer height of a vehicle obstacle detection method according to an exemplary embodiment of this application;

[0057] Figure 7 This is a flowchart illustrating the protrusion height in a vehicle obstacle detection method according to an exemplary embodiment of this application;

[0058] Figure 8 This is a structural block diagram of a vehicle obstacle detection system illustrated in an exemplary embodiment of this application;

[0059] Figure 9 This is a schematic diagram illustrating the principle structure of a vehicle obstacle detection system according to an exemplary embodiment of this application;

[0060] Figure 10 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0061] The embodiments of this application will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be understood that the preferred embodiments are only for illustrating this application and are not intended to limit the scope of protection of this application.

[0062] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0063] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0064] Please see Figure 1 This is a schematic diagram illustrating the environmental application of a vehicle obstacle detection method provided in one embodiment of this application. Figure 1 As shown, the application network architecture of this implementation environment may include device side 11 and vehicle 10 (vehicle terminal cluster). The vehicle terminal cluster may include one or more vehicle terminals; the number of vehicle terminals is not limited here. Specifically, it may include vehicle terminal a, vehicle terminal b, vehicle terminal c, ..., vehicle terminal n. Vehicle terminal a, vehicle terminal b, vehicle terminal c, ..., vehicle terminal n can each connect to the device side via a network, so that the vehicle terminals can interact with device side 11 through this network connection. The specific connection method of this network connection is not limited here; for example, it can be directly or indirectly connected via wireless communication.

[0065] It should be noted that, in one approach, the device 11 can also be integrated into the vehicle terminal to collect vehicle data, perform obstacle detection on the collected data, and identify obstacles around the vehicle. In another approach, the vehicle terminal is responsible for collecting vehicle data and synchronizing it to the device side (cloud) for obstacle detection, thus completing the obstacle detection around the vehicle.

[0066] In this embodiment, the vehicle 10 can be a powered automobile, such as a truck, dump truck, off-road vehicle, sedan, bus, tractor-trailer, semi-trailer, or special-purpose vehicle. Trucks are primarily used for transporting goods, and some can also tow full trailers. Dump trucks are mainly used for transporting goods and have a tilting cargo box, suitable for driving on rough or unpaved roads, and are often used in national defense, forestry, and mining areas. Off-road vehicles are all-wheel-drive vehicles with high passability, suitable for driving on rough or unpaved roads, and are often used in national defense, forestry, and mining areas. Sedans are four-wheeled vehicles used to carry passengers and their belongings, with seats arranged between the two axles. Based on engine displacement, they can be divided into microcars (below 1L), standard sedans (1-1.6L), and mid-size sedans (1.6-2.5L). L), mid-to-high-end sedans (2.5-4L), luxury sedans (4L and above); buses are automobiles with rectangular carriages, mainly used for carrying passengers and their personal belongings, and can be divided into long-distance buses, group buses, city buses and tourist buses according to their purpose; tractor trucks and semi-trailer tractor trucks are automobiles mainly used for towing trailers or semi-trailers, and can be divided into semi-trailer tractor trucks and full-trailer tractor trucks according to the different trailers they tow; special vehicles are automobiles equipped with special equipment and have special functions, used to undertake special transportation tasks or special operations, such as fire trucks, ambulances, tanker trucks, bulletproof vehicles, engineering vehicles, etc., which are not limited here.

[0067] like Figure 1 As shown, in this embodiment, the device side 11 can be the server corresponding to the application client. The device side 11 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0068] Please see Figure 2 The flowchart illustrating a vehicle obstacle detection method as an exemplary embodiment of this application includes:

[0069] Step S210: Obtain the projection image of the projected light on the ground in the current state of the vehicle, wherein the current state includes a driving state or a stationary state;

[0070] The vehicle's current state includes both stationary and moving states, with the moving state primarily referring to low-speed driving, such as a speed below 40 kilometers per hour. The projected light originates from the vehicle's headlights, welcome lights, or additional projection lights (projection devices). This projected light can be visible light, specifically electromagnetic waves with frequencies between 380 and 750 THz and wavelengths between 780 and 400 nm, perceptible to the human eye. For example, in strong sunlight outdoors, light waves with fewer frequency components in the solar spectrum can be projected to mitigate the effects of sunlight. These waves are received by sensors of the corresponding frequency, resulting in a projected image on the ground.

[0071] It should be noted that the projected light can also be non-visible light, such as ultraviolet, infrared, and far-infrared light. A light-receiving sensor that matches the projected light emitted by the projection device is used to obtain the projected image.

[0072] Step S220: Compare the projected image with a preset projected image to determine the obstacles in the projected image and the location area of ​​the obstacles;

[0073] Specifically, the preset projection image is a projection image of the ground in the current scene without any obstacles. The projection image is compared with the preset projection image, mainly comparing the graphic in the two images. This graphic is formed by the projection light emitted by the projection device to project the pattern.

[0074] Step S230: Based on the line changes generated by the obstacle in the position area of ​​the projection map, determine the type of the obstacle to achieve obstacle detection, wherein the type of obstacle includes protrusions and faults.

[0075] Specifically, obstacles in the projected image are identified by the changes in the lines produced by the location of the obstacle in the projected image.

[0076] For example, if the furthest distance of a line in the projected image is greater than the furthest distance of a line in a preset projected image, then the type of road obstacle is determined to be a fault.

[0077] If the furthest distance of a line in the projected image is closer than the furthest distance of a line in a preset projected image, then the type of road obstacle is determined to be a protrusion.

[0078] The above methods can effectively determine the type of obstacle in the projected image, making it easier to understand the specific situation of the fault.

[0079] In this embodiment, by acquiring the projected image of the vehicle's current state projected light onto the ground, and comparing the projected image with a preset projected image, obstacles in the projected image and the location area of ​​the obstacles are determined. Based on the line changes generated by the obstacle in the location area of ​​the projected image, the type of obstacle is determined, and obstacle detection is completed. Compared with other visual obstacle detection methods, this application is not affected by ambient brightness. Even in a dimly lit parking lot, obstacle detection can be performed using projected light, which not only improves the accuracy of obstacle detection and eliminates the need for additional sensors, but also greatly reduces the cost of obstacle detection.

[0080] Please see Figure 3 The flowchart illustrates an obstacle height determination process in a vehicle obstacle detection method, as shown in an exemplary embodiment of this application. After determining the type of obstacle and performing obstacle detection, the method further includes:

[0081] Step S320: If the obstacle is a protrusion, calculate the height of the protruding obstacle based on the projection angle of the vehicle's projected light at the farthest point and the travel distance; wherein, the travel distance and the projection angle are determined by the moment when the deformation of the projected image occurs and the moment when the deformation is eliminated.

[0082] For example, the time when the deformation of the projected image occurs and the time when the deformation is eliminated are determined by the projected image to obtain a time period. The distance L that the vehicle has traveled in the current scene is determined based on the vehicle's current speed and the time period. The height H of the obstacle for the current vehicle is H = L * tan(α) by using the angle α formed between the farthest point (farthest projection point) formed by the projection pipeline on the ground and the ground.

[0083] See details Figure 7 The height of the obstacle in the current projected image can be determined by the angle formed between the projected light emitted by the projection device and the obstacle.

[0084] Step S330: If the obstacle is a fault, calculate the height of the fault obstacle based on the principle of similar triangles, combined with the projection generated by the vehicle and the distance of the acquired projection image in the horizontal and vertical directions.

[0085] For details, see Figure 6 Let h be the height of the fault to be calculated, and L2 be the distance from the projection point on the fault to the camera. H3' is a point on a similar triangle introduced for calculation convenience. We can obtain: H3 / H3' = L3 / (L1+L4-L3)

[0086] For example, the device that generates the projection (light) is the projection device, and the device that captures the image is the camera. It should be noted that the camera's light sensor is located inside the camera and is a certain distance away from the camera lens.

[0087] L1: The theoretical distance between the farthest point of illumination on the ground and the projection device when the ground is flat;

[0088] L3: Distance from the camera lens to the camera's light sensor;

[0089] L4: Distance from the projection device to the image sensor;

[0090] H1: Camera height;

[0091] H2: Height of the projection device;

[0092] H3: The theoretical distance between the image point of the farthest irradiated point on the ground on the camera sensor when the ground is flat, and the actual distance between the image point of the farthest irradiated point on the camera sensor.

[0093] According to the principle of similar triangles, we can obtain:

[0094] h / (h+H2)=(L2-L1-L4) / (L2-L4)

[0095] (h+H3') / (h+H1)=(L2-L1-L4) / L2

[0096] Therefore, the height h of the fault can be calculated.

[0097] In this embodiment, different calculation methods are used to calculate the height of obstacles according to their different types. This method can accurately calculate the height of obstacles without the need for complex algorithms. At the same time, it does not require any additional modifications to the vehicle or assistance from ultrasonic radar, lidar, etc., and can accurately determine the height of obstacles. In addition, compared with other methods of detecting obstacles using cameras, the detection accuracy is higher.

[0098] In other embodiments, the projected image is compared with a preset projected image to determine obstacles in the projected image and the location region of the obstacles, including:

[0099] The projected image is compared with a preset projected image;

[0100] If the projected image has the same shape as the preset projected image, then it is determined that there are no obstacles in the projected image;

[0101] If the projected image differs from the preset projected image, the obstacle and its location region are determined based on the graphic offset between the projected image and the preset projected image.

[0102] For example, when the graphic offset exceeds a preset threshold, i.e., the graphic offset is determined, after determining that the projected image is different from the preset projected image, the location area of ​​the obstacle and the obstacle are determined according to the degree of image offset.

[0103] The above method can quickly determine the obstacle and its location area in the projected image, facilitating the accurate positioning of the obstacle.

[0104] Based on the above embodiments, image recognition can also be performed based on the graphic offset and outline of obstacles in the projected image to determine the type of obstacle, such as stones, debris, faults formed by small pits, etc.

[0105] Please see Figure 4 The following is a detailed flowchart illustrating a vehicle obstacle detection method for parking, as shown in an exemplary embodiment of this application:

[0106] Step S401: When starting the parking maneuver, the controller controls the projection device to project around the vehicle body according to the preset pattern and position. Based on the projected pattern and position, the controller calculates the theoretical pattern captured on the flat ground.

[0107] In step S402, the surround-view camera captures the projected image, and the controller compares the captured pattern with the theoretical pattern. If the pattern matches the theoretical pattern, there is no target within the projected pattern area; otherwise, proceed to the next step.

[0108] Step S403: Compare the outer lines of the theoretical image with the actual captured image;

[0109] Step S404: When the offset exceeds the threshold, it is considered that there is an obstacle. The position and range of the non-collision area are calculated based on the position of the line offset.

[0110] Step S405: During the parking process, avoid entering areas where driving is prohibited.

[0111] In this embodiment, the controller controls the projection device to project light rays around the vehicle body onto the ground. The distance between the light rays and the vehicle body can be calibrated; generally, the higher the vehicle speed, the farther the light rays are from the vehicle body. When there are no obstacles, the light rays are projected onto a flat surface, and the shape of the light rays seen by the surround-view camera remains unchanged regardless of whether the vehicle is moving. When the light rays encounter an obstacle, the shape of the light rays changes. If the change exceeds a threshold, the controller determines whether the obstacle is above ground level or a break in the ground. The location where the light rays change is then recorded, indicating a point where passage is impossible, and this location is avoided during parking. Furthermore, it can be fused with sensors such as ultrasonic radar and surround-view cameras to more accurately obtain detailed information about obstacles and perform obstacle detection more precisely.

[0112] Please see Figure 5 The following is a flowchart illustrating a vehicle obstacle detection method at low speeds, as shown in an exemplary embodiment of this application:

[0113] Step S501: When starting the parking lot at low speed, the controller controls the projection device to project around the vehicle body according to the preset pattern and position. Since the vehicle speed is faster than when parking, the projection range from the vehicle is larger. Based on the projected pattern and position, the controller calculates the theoretical pattern captured on the flat ground.

[0114] In step S502, the front camera or the surround view camera captures the projected image. The controller compares the captured pattern with the theoretical pattern. If the pattern matches the theoretical pattern, there is no target within the projected pattern area. If the pattern does not match the theoretical pattern, proceed to the next step.

[0115] Step S503: Compare the outer lines of the theoretical image with the actual captured image;

[0116] Step S504: When the offset exceeds the threshold, it is considered that there is an obstacle. The position and range of the obstacle are calculated based on the position of the line offset.

[0117] Step S505: Determine if it is a fault.

[0118] If the camera is higher than the projection device, and the farthest range of the captured image is farther than the theoretical image, then a fault line below the road surface appears ahead. The height difference of this fault line is calculated. Conversely, if the camera is lower than the projection device, and the farthest range of the captured image is closer than the theoretical image, then a fault line below the road surface appears ahead. The height difference of this fault line is calculated, and based on this height difference, it is determined whether the vehicle can pass.

[0119] Step S506: Calculate the height of the obstacle.

[0120] If it is not a tomographic event, record the time point when the image shows deformation and the time point when the deformation disappears at the corresponding location in the image. Based on the distance the vehicle travels during this time, calculate the height of the obstacle and determine whether the vehicle can pass.

[0121] In this embodiment, during actual driving, the projected light is positioned further away from the vehicle. The vehicle's front-view camera, rear-view camera, and surround-view camera can be used to monitor the light projected onto the ground.

[0122] The height of an obstacle can be calculated based on the duration of the light change and the vehicle speed. When there are no obstacles, the light beams projected onto a flat surface remain unchanged in shape as seen by the camera, regardless of vehicle movement. When the light beams encounter an obstacle, their shape changes. If the change exceeds a threshold, the controller determines whether the obstacle is above ground level or a fault line. If it's a fault line, the height is determined based on the light change; if it's an obstacle, the height is calculated based on the start and end times of the light change and the distance the vehicle travels during that time, determining whether it's a small, passable obstacle.

[0123] In other embodiments, after determining the height of the obstacle, the method further includes:

[0124] The height of the obstacle is determined based on a preset height.

[0125] If the height of the obstacle is lower than the preset height, then it is determined that the vehicle can pass through the obstacle at that height;

[0126] If the height of the obstacle is not lower than the preset height, it is determined that the vehicle cannot pass through the obstacle at that height, and an alarm is issued to notify the vehicle.

[0127] In this embodiment, a threshold method can be used to accurately determine whether a vehicle can pass through an obstacle of a certain height, so as to prevent safety accidents during the autonomous driving process and improve the safety performance of automatic parking.

[0128] Based on the above embodiments, after determining that the vehicle cannot pass through an obstacle of that height, the method further includes:

[0129] Obtain the closest distance between the vehicle and the obstacle;

[0130] Specifically, by locating the center point of the obstacle, the shortest distance between the vehicle and the obstacle is determined by the distance between the center point and the front, rear, left and right sides of the obstacle.

[0131] The nearest distance is compared with a preset safe distance threshold. If the nearest distance is less than or equal to the preset safe distance threshold, an obstacle alarm is issued; if the nearest distance is greater than the preset safe distance threshold, no action is taken.

[0132] By using the above methods, the distance between the vehicle and obstacles can be accurately controlled during vehicle operation, thus preventing accidents caused by driving reasons (human factors or vehicle autopilot factors) when the vehicle knows it cannot cross the obstacle, and avoiding unnecessary losses to the vehicle.

[0133] In other embodiments, after determining the type of obstacle and performing obstacle detection, the method further includes:

[0134] Acquire ultrasonic radar data, millimeter-wave radar data, and lidar data of the vehicle in its current state;

[0135] Specifically, environmental data is collected using the aforementioned multiple radar sensors to determine the accuracy of the environmental data surrounding the vehicle from multiple dimensions.

[0136] The lidar data, ultrasonic radar data, and millimeter-wave radar data are fused to determine environmental perception information;

[0137] Specifically, ultrasonic radar sensors, together with vision sensors, can create parking maps and dynamically plan parking paths in real time, guiding the target vehicle to automatically control the steering wheel and drive into the parking position. Millimeter-wave radar can detect long distances around the clock. Through preset millimeter-wave radar, the blind spot range of the rearview mirror can be detected. LiDAR can detect long distances and measure lateral position. Vision sensors are inexpensive. The system combines vision sensors, millimeter-wave radar sensors, and lidar sensors.

[0138] A training set is generated by labeling the environmental perception information and obstacle detection results.

[0139] Specifically, labels for environmental perception information and obstacle detection results can be determined through manual annotation to form a training set.

[0140] The pre-built convolutional neural network is trained based on the training set to obtain an intelligent decision-making model;

[0141] Specifically, a convolutional neural network is trained using an unsupervised training method to obtain an intelligent decision-making model, which can determine whether a vehicle can pass through an obstacle.

[0142] The environmental perception information and obstacle detection results are input into the intelligent decision-making model to output the corresponding control strategy.

[0143] Specifically, it determines whether the vehicle can pass through the obstacle and whether it needs to go around the obstacle.

[0144] In this embodiment, multiple preset radar data are fused to detect and perceive the driving environment of the target vehicle, determine environmental perception information, collect environmental parameters, and improve environmental simulation capabilities. The intelligent decision-making module receives and processes the environmental perception information, formulates corresponding adaptive control strategies, transmits the adaptive control strategies to a preset human-machine interaction system for intelligent processing, generates corresponding intelligent decision commands, and performs intelligent and flexible control of the vehicle.

[0145] In other embodiments, after determining the type of obstacle and thus completing obstacle detection, the method further includes:

[0146] Visual deep learning algorithms (such as convolutional neural networks, YOLO (You Only Look Once) algorithm, etc.) are used to identify parking spaces based on visual data of the current vehicle scene, determine visual parking space information, and mark the first time point;

[0147] Specifically, the first time point is the time when the image of the vehicle scene to be identified is captured. The visual parking space information includes any one or more of the following: visual parking space corner information, visual parking space parking attributes, visual parking space type, and visual parking space size. The visual parking space parking attributes include whether the space is available for parking or not. The visual parking space corner information includes the coordinates and angle of the visual parking space corner. The visual parking space type includes perpendicular parking spaces, horizontal parking spaces, and angled parking spaces. The visual parking space size includes the visual parking space length and the visual parking space depth. The visual parking space length is the length of the first parking space side along the road direction in the parking space line, and the visual parking space depth is the length of the second parking space side intersecting the first parking space side in the parking space line. There are two second parking space sides, and their lengths may be the same or different.

[0148] Based on machine learning algorithms (support vector machine, feedforward neural network, recurrent neural network, etc.), the ultrasonic radar data of the current vehicle scene is used to identify parking spaces, determine spatial parking space information, and mark the second time point;

[0149] Specifically, the second time point is the time when the ultrasonic radar receives the echo signal from the ultrasonic radar data to be identified. The parking space information includes one or more of the following: parking space corner information, parking space attributes, parking space type, and parking space size. The parking space attributes all include whether a space is available for parking or not. The parking space corner information includes the corner coordinates and angle of the parking space. The parking space type includes perpendicular parking spaces, horizontal parking spaces, and angled parking spaces. The parking space size includes the length and depth of the parking space.

[0150] It should be noted that visual parking space information and spatial parking space information are essentially the same; the difference in name is simply due to the different methods of identification used to distinguish between them.

[0151] By fusing spatial parking space information and visual parking space information associated with the first and second time points at the same time, the target parking space information is obtained.

[0152] Specifically, spatial parking space information and visual parking space information are collected simultaneously at specific time points. For example, if a point in the visual parking space information is unclear or ambiguous, it is fused with the spatial parking space information. Alternatively, if there are discrepancies between the spatial and visual parking space information, the spatial information is prioritized, and the two are fused to obtain more accurate target parking space information.

[0153] It should be noted that the fusion methods include, but are not limited to, weighted average method, image pyramid fusion algorithm, Gaussian pyramid fusion algorithm, Laplacian pyramid fusion algorithm, etc.

[0154] The system assists in automatic parking of the vehicle based on the target parking space information and the obstacle detection results between the target parking space information and the vehicle.

[0155] Specifically, after obtaining the target parking space information, the system combines the obstacle detection results between the target parking space information and the vehicle to assist automatic parking. For example, when the vehicle is reversing into the parking space, it can determine whether the vehicle needs to change its driving trajectory to bypass the obstacle, thereby ensuring the safety of the vehicle entering the parking space.

[0156] In this embodiment, not only can visual parking space recognition be achieved based on machine vision recognition methods and spatial parking space recognition be achieved based on ultrasonic radar, but the two recognition results can also be fused to obtain more accurate parking space recognition results. Thus, compared with manual parking space recognition, it can achieve the effect of automatic, fast and accurate parking space recognition.

[0157] Figure 8 This is a structural block diagram illustrating a vehicle obstacle detection system according to an exemplary embodiment of this application. The system can be applied to... Figure 1 The implementation environment shown is specifically configured on the vehicle terminal and the cloud server. This system can also be applied to other exemplary implementation environments, specifically configured on the vehicle terminal or the cloud server. This embodiment does not limit the implementation environment to which the system is applicable.

[0158] like Figure 8 As shown, this exemplary vehicle obstacle detection system includes:

[0159] The image acquisition module 801 is used to acquire the projected image of the projected light rays on the ground in the current state of the vehicle, wherein the current state includes a driving state or a stationary state.

[0160] The image comparison module 802 is used to compare the projected image with a preset projected image to determine the obstacles in the projected image and the location area of ​​the obstacles.

[0161] The obstacle detection module 803 is used to determine the type of obstacle based on the line changes generated by the obstacle in the position area of ​​the projection map, and to realize obstacle detection. The type of obstacle includes protrusions and faults.

[0162] For details, see Figure 9 A light projection device (i.e., a projection device) emits a specific pattern to form a projected image on the ground, and a light sensing device (i.e., a camera) acquires the projected image on the ground. The controller controls the light projection device to emit the projected image of the specific pattern, compares it with the acquired projected image, and thus determines obstacles in the projected image and the location area of ​​the obstacles; based on the line changes produced by the obstacle in its location area in the projected image, the type of obstacle is determined, completing obstacle detection.

[0163] This exemplary vehicle obstacle detection system acquires a projected image of the vehicle's current state projected light onto the ground, compares the projected image with a preset projected image, and determines the obstacles in the projected image and the location area of ​​the obstacles. Based on the line changes produced by the obstacle in the location area of ​​the projected image, the type of obstacle is determined, thus completing obstacle detection. Compared with other visual obstacle detection methods, this application is not affected by ambient brightness. Even in a dimly lit parking lot, obstacle detection can be performed using projected light, which not only improves the accuracy of obstacle detection and eliminates the need for additional sensors, but also significantly reduces the cost of obstacle detection.

[0164] It should be noted that the vehicle obstacle detection system and the vehicle obstacle detection method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the vehicle obstacle detection system provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0165] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the vehicle obstacle detection method provided in the above embodiments.

[0166] Figure 10 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 10 The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0167] like Figure 10 As shown, the computer system 100 includes a Central Processing Unit (CPU) 101, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 102 or programs loaded from Storage Unit 108 into Random Access Memory (RAM) 103, such as performing the methods described in the above embodiments. The RAM 103 also stores various programs and data required for system operation. The CPU 101, ROM 102, and RAM 103 are interconnected via a bus 104. An Input / Output (I / O) interface 105 is also connected to the bus 104.

[0168] The following components are connected to I / O interface 105: an input section 106 including a keyboard, mouse, etc.; an output section 107 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 108 including a hard disk, etc.; and a communication section 109 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. Drive 1010 is also connected to I / O interface 105 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 108 as needed.

[0169] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 109, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 101, it performs various functions defined in the system of this application.

[0170] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0171] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0172] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0173] Another aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle obstacle detection method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0174] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the vehicle obstacle detection method provided in the various embodiments described above.

[0175] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for detecting vehicle obstacles, characterized in that, include: Acquire a projection image of the projected light rays on the ground in the current state of the vehicle, wherein the current state includes a driving state or a stationary state; The projected image is compared with a preset projected image to determine the obstacles in the projected image and the location area of ​​the obstacles; Obstacle detection is achieved by determining the type of obstacle based on the line changes generated by the obstacle in the position area of ​​the projection map. The types of obstacles include protrusions and faults. If the obstacle is a protrusion, the height of the protruding obstacle is calculated based on the projection angle of the vehicle's projected light at the farthest point and the travel distance; wherein, the travel distance is determined by the moment when the deformation of the projected image occurs and the moment when the deformation is eliminated; If the obstacle is a fault, the height of the fault obstacle is calculated based on the principle of similar triangles, combined with the distances in the horizontal and vertical directions of the projection generated by the vehicle and the equipment that collects the projection.

2. The vehicle obstacle detection method according to claim 1, characterized in that, Comparing the projected image with a preset projected image to determine the obstacles in the projected image and the location regions of the obstacles includes: The projected image is compared with a preset projected image; If the projected image has the same shape as the preset projected image, then it is determined that there are no obstacles in the projected image; If the projected image differs from the preset projected image, the obstacle and its location region are determined based on the graphic offset between the projected image and the preset projected image.

3. The vehicle obstacle detection method according to claim 1, characterized in that, Obstacle detection is achieved by determining the type of obstacle based on the line changes generated by the obstacle in the location area of ​​the obstacle in the projection image, including: If the furthest distance of a line in the projected image is greater than the furthest distance of a line in the preset projected image, then the type of road obstacle is determined to be a fault. If the furthest distance of a line in the projected image is closer than the furthest distance of a line in a preset projected image, then the type of road obstacle is determined to be a protrusion.

4. The vehicle obstacle detection method according to claim 1, characterized in that, After determining the height of the obstacle, the process also includes: The height of the obstacle is determined based on a preset height. If the height of the obstacle is lower than the preset height, then it is determined that the vehicle can pass through the obstacle at that height; If the height of the obstacle is not lower than the preset height, it is determined that the vehicle cannot pass through the obstacle at that height, and an alarm is issued to notify the vehicle.

5. The vehicle obstacle detection method according to claim 4, characterized in that, After determining that the vehicle cannot pass through an obstacle of that height, the process also includes: Obtain the closest distance between the vehicle and the obstacle; The nearest distance is compared with a preset safe distance threshold. If the nearest distance is less than or equal to the preset safe distance threshold, an obstacle alarm is issued; if the nearest distance is greater than the preset safe distance threshold, no action is taken.

6. The vehicle obstacle detection method according to any one of claims 1 to 5, characterized in that, After determining the type of obstacle and implementing obstacle detection, the method further includes: Acquire ultrasonic radar data, millimeter-wave radar data, and lidar data of the vehicle in its current state; The lidar data, ultrasonic radar data, and millimeter-wave radar data are fused to determine environmental perception information; A training set is generated by labeling the environmental perception information and obstacle detection results. The pre-built convolutional neural network is trained based on the training set to obtain an intelligent decision-making model; The environmental perception information and obstacle detection results are input into the intelligent decision-making model to output the corresponding control strategy.

7. The vehicle obstacle detection method according to any one of claims 1 to 5, characterized in that, After determining the type of obstacle and implementing obstacle detection, the method further includes: Based on visual deep learning algorithms, parking space identification is performed on the visual data of the current vehicle scene to determine visual parking space information and mark the first time point; Based on machine learning algorithms, ultrasonic radar data of the current vehicle scene is used to identify parking spaces, determine spatial parking space information, and mark the second time point. By fusing spatial parking space information and visual parking space information associated with the first and second time points at the same time, the target parking space information is obtained. The system assists in automatic parking of the vehicle based on the target parking space information and the obstacle detection results between the target parking space information and the vehicle.

8. A vehicle obstacle detection system, characterized in that, include: The image acquisition module is used to acquire the projected image of the vehicle's current state on the ground, wherein the current state includes a driving state or a stationary state. An image comparison module is used to compare the projected image with a preset projected image to determine the obstacles in the projected image and the location area of ​​the obstacles; An obstacle detection module is used to determine the type of obstacle based on the line changes generated by the obstacle in the position area of ​​the projection image, thereby achieving obstacle detection. The types of obstacles include protrusions and faults. If the obstacle is a protrusion, the height of the protruding obstacle is calculated based on the projection angle of the vehicle's projection light at the farthest point and the travel distance. The travel distance is determined by the time when the deformation of the projection image occurs and the time when the deformation is eliminated. If the obstacle is a fault, the height of the fault obstacle is calculated based on the principle of similar triangles, combined with the distances in the horizontal and vertical directions between the vehicle's projection and the device that acquires the projection image.

9. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to perform the method of any one of claims 1 to 7.

10. A vehicle device, characterized in that, Includes the electronic device as described in claim 9.

11. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by the computer's processor, cause the computer to perform the method of any one of claims 1 to 7.

12. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, wherein a processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions to cause the electronic device to perform the method of any one of claims 1 to 7.

Images