A visual occlusion solution, apparatus, device, and medium

Abstract

The present application discloses a solution, device, equipment, and medium for visual occlusion, relating to the field of autonomous driving, including: identifying surrounding obstacle information during the driving process of the vehicle; matching the corresponding target obstacle from the obstacle information using the obstacle attention type determined based on the current driving path and the preset planned path; continuously detecting the status of the target obstacle, and if the target obstacle undergoes a preset disappearance event, determining whether the target obstacle is obscured based on the corner position information of the preceding vehicle; if obscured, storing the target obstacle and the state information of the target obstacle before disappearance in a preset occlusion array, so that the vehicle can perform path planning based on the preset occlusion array; and releasing the information stored in the preset occlusion array when a preset release condition is met. The present application can improve the reliability and safety of path planning when visual occlusion occurs during vehicle driving.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic driving, and in particular to a method and device for solving visual occlusion, equipment and medium. BACKGROUND

[0002] In the urban automatic driving scenario, the current visual perception technology has obvious defects, which can only record the objects perceived at the current time, and lacks the ability to continuously record objective objects that exist in the environment all the time but may be temporarily occluded.

[0003] For example, at the intersection, large vehicles often block the traffic lights. Since the visual perception system of the autonomous vehicle cannot retain the traffic light information before being occluded, the vehicle cannot make correct driving plans in the state of not being able to see the traffic light. This situation is likely to cause the vehicle to run a red light and other serious violations of traffic rules, which not only poses a great threat to traffic safety, but also affects the reliability and practicality of autonomous driving technology, and becomes a key problem to be solved in the development of urban autonomous driving.

[0004] In summary, when visual occlusion occurs during vehicle driving, how to improve the reliability and safety of path planning is a problem to be solved at present. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a method and device for solving visual occlusion, which can improve the reliability and safety of path planning when visual occlusion occurs during vehicle driving. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses a method for solving visual occlusion, comprising:

[0007] During the driving of the ego vehicle, identifying the surrounding obstacle information;

[0008] According to the current driving path and the preset planning path, determining the obstacle attention type, and matching the corresponding target obstacle from the obstacle information based on the obstacle attention type;

[0009] Continuously detecting the state of the target obstacle, and if the target obstacle occurs a preset disappearance event, determining whether the target obstacle is occluded based on the corner point position information of the preceding vehicle;

[0010] If it is occluded, storing the target obstacle and the state information of the target obstacle before disappearance in a preset occlusion array, so that the ego vehicle plans the path based on the preset occlusion array;

[0011] When the preset release condition is met, releasing the information stored in the preset occlusion array.

[0012] Optionally, the process of identifying the surrounding obstacle information further comprises:

[0013] classifying the surrounding obstacle information; wherein the obstacle classes include vehicles, pedestrians, traffic signs and traffic lights;

[0014] comparing the speed of each surrounding obstacle with a preset speed threshold to determine the type of each surrounding obstacle; wherein the obstacle types include static obstacles and dynamic obstacles.

[0015] Optionally, the determining of the obstacle attention type based on the current driving path and the preset planning path to match the corresponding target obstacle from the obstacle information based on the obstacle attention type comprises:

[0016] determining the current driving path and obtaining the preset planning path from a navigation system;

[0017] judging whether the ego vehicle needs to perform a lane change operation based on the positional relationship between the current driving path and the preset planning path;

[0018] if the ego vehicle does not need to perform a lane change operation, judging whether the ego vehicle is located at an intersection position to determine a first type of obstacle attention type based on the intersection position judgment result, and then matching the corresponding target obstacle from the obstacle information based on the first type of obstacle attention type;

[0019] if the ego vehicle needs to perform a lane change operation, obtaining a second type of obstacle attention type based on the first type of obstacle attention type and the type of obstacle on the target lane change side, and then matching the corresponding target obstacle from the obstacle information based on the second type of obstacle attention type.

[0020] Optionally, the determining of the first type of obstacle attention type based on the intersection position judgment result comprises:

[0021] if the ego vehicle is not located at the intersection position, determining the front vehicle on the ego lane as the first type of obstacle attention type;

[0022] if the ego vehicle is located at the intersection position, determining the target dynamic obstacle and the target static obstacle around the intersection position as the first type of obstacle attention type; the target dynamic obstacle includes pedestrians and non-motor vehicles on both sides of the intersection, and the target static obstacle includes traffic lights and traffic signs.

[0023] Optionally, the judging of whether the target obstacle is blocked based on the corner point position information of the front vehicle comprises:

[0024] establish a three-dimensional coordinate system with a camera of a vehicle as an origin, and determine position information of each corner point of a front vehicle in the three-dimensional coordinate system according to position information and length, width and height information of the front vehicle;

[0025] connect each corner point of the front vehicle with the origin based on the position information of each corner point to obtain a corresponding ray, and determine an occlusion space based on a region formed by the rays;

[0026] determine whether the target obstacle is located in the occlusion space according to state information of the target obstacle before disappearing, and if yes, determine that the target obstacle is occluded.

[0027] Optionally, the determining whether the target obstacle is located in the occlusion space according to the state information of the target obstacle before disappearing includes:

[0028] project the occlusion space into a height space and a ground space respectively, wherein the ground space is a two-dimensional space parallel to a vehicle driving plane, and the height space is a two-dimensional space perpendicular to the ground;

[0029] obtain a spatial position coordinate of a last frame before the target obstacle disappears, and determine whether a height position coordinate in the spatial position coordinate is located in the height space;

[0030] if yes, determine whether a ground position coordinate in the spatial position coordinate is located in the ground space, and if yes, determine that the target obstacle is located in the occlusion space.

[0031] Optionally, before the determining whether the target obstacle is located in the occlusion space according to the state information of the target obstacle before disappearing, the method further includes:

[0032] if the target obstacle is a static obstacle, determining whether the target obstacle is in a preset stable state according to state information of several frames before the target obstacle disappears;

[0033] if no, prohibiting the step of determining whether the target obstacle is located in the occlusion space according to the state information of the target obstacle before disappearing.

[0034] Optionally, the storing the target obstacle and the state information of the target obstacle before disappearing into a preset occlusion array includes:

[0035] if the target obstacle is a traffic light, storing the traffic light and display state and second information of the traffic light before disappearing into a preset occlusion array;

[0036] If the target obstacle is a dynamic obstacle, the dynamic obstacle and speed values, acceleration values and position information of the dynamic obstacle in several frames before disappearing are stored in a preset occlusion array.

[0037] Optionally, the releasing the information stored in the preset occlusion array when a preset release condition is met comprises:

[0038] For any static obstacle corresponding to the information stored in the preset occlusion array, if the any static obstacle is re-identified, or if it is determined according to the state information of the any static obstacle before disappearing that the any static obstacle has exceeded the current detection area, the information related to the any static obstacle stored in the preset occlusion array is released.

[0039] For any dynamic obstacle corresponding to the information stored in the preset occlusion array, the current latest position of the any dynamic obstacle is predicted according to the state information of the any dynamic obstacle before disappearing, and if it is determined based on the current latest position that the ego vehicle has exceeded the any dynamic obstacle, or if the ego vehicle has completed a lane changing operation, or if the any dynamic obstacle is re-identified, or if there is another vehicle passing through the occlusion area, the information related to the any dynamic obstacle stored in the preset occlusion array is released.

[0040] In a second aspect, the present application discloses a device for solving visual occlusion, comprising:

[0041] An identification module is configured to identify obstacle information around the ego vehicle during driving of the ego vehicle.

[0042] A matching module is configured to determine an obstacle attention type according to a current driving path and a preset planning path, and match a corresponding target obstacle from the obstacle information based on the obstacle attention type.

[0043] An occlusion detection module is configured to continuously detect a state of the target obstacle, and determine whether the target obstacle is occluded based on corner point position information of a preceding vehicle if a preset disappearance event occurs to the target obstacle.

[0044] A storage module is configured to store the target obstacle and state information of the target obstacle before disappearing in a preset occlusion array if the target obstacle is occluded, so that the ego vehicle plans a path based on the preset occlusion array.

[0045] A release module is configured to release information stored in the preset occlusion array when a preset release condition is met.

[0046] In a third aspect, the present application discloses an electronic device, comprising:

[0047] a memory for storing the computer program;

[0048] a processor for executing the computer program to implement the steps of the method for solving visual occlusion disclosed above.

[0049] In a fourth aspect, the present application discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by a processor to implement the steps of the method for solving visual occlusion disclosed above.

[0050] It can be seen that in the driving process of the ego vehicle, the surrounding obstacle information is identified, the obstacle attention type is determined according to the current driving path and the preset planning path, the corresponding target obstacle is matched from the obstacle information based on the obstacle attention type, the state of the target obstacle is continuously detected, if the target obstacle occurs a preset disappearance event, whether the target obstacle is occluded is judged based on the corner point position information of the preceding vehicle, if the target obstacle is occluded, the target obstacle and the state information of the target obstacle before disappearance are stored to a preset occlusion array, so that the ego vehicle performs path planning based on the preset occlusion array, and when a preset release condition is met, the information stored in the preset occlusion array is released.

[0051] Beneficial effects: In the present application, the surrounding obstacle information is identified in real time in the driving process of the ego vehicle, and the corresponding target obstacle is matched from the obstacle information according to the obstacle attention type determined according to the current driving path and the preset planning path. It can be understood that in different road scenes, the types of obstacles that need to be paid attention to are different, and it is not necessary to pay attention to every obstacle around. Therefore, the present application needs to determine the obstacle attention type according to the current driving path and the preset planning path, and then further match the corresponding target obstacle from the obstacle information to filter out other obstacles that do not need to be paid attention to. After matching the target obstacle, the state of the target obstacle is continuously detected, if the target obstacle occurs a preset disappearance event, whether the target obstacle is occluded is judged based on the corner point position information of the preceding vehicle, if the target obstacle is determined to be occluded, the target obstacle and the state information of the target obstacle before disappearance are stored to a preset occlusion array, so that in the time period when the target obstacle disappears, the ego vehicle can perform path planning based on the information stored in the preset occlusion array, avoiding the situation that the ego vehicle makes an incorrect driving planning due to the occlusion of the obstacle, thereby improving the reliability and safety of path planning. That is, in the present application, only the occluded obstacle is stored to the preset occlusion array for subsequent path planning of the ego vehicle. In addition, when the preset release condition is met, the information stored in the preset occlusion array is released, which can avoid storing too much unnecessary information, achieve the purpose of releasing memory and computing resources, and make the system more efficient in processing new perception data and planning tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0053] Figure 1 This is a flow chart of a solution to visual occlusion disclosed in this application;

[0054] Figure 2 This is a flowchart of a specific solution to visual occlusion disclosed in this application;

[0055] Figure 3 A schematic diagram of constructing an occlusion space in a three-dimensional space disclosed in this application;

[0056] Figure 4 A schematic diagram of a height space disclosed in this application;

[0057] Figure 5 This is a schematic diagram of the structure of a device for solving visual occlusion disclosed in this application;

[0058] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0060] Current visual perception technology has obvious flaws. It can only record objects perceived at the current moment, and lacks the ability to continuously record objective objects that are always present in the environment but may be temporarily obscured. This situation is very likely to cause vehicles to commit serious violations of traffic regulations, such as running red lights. This not only poses a huge threat to traffic safety, but also affects the reliability and practicality of autonomous driving technology, becoming a key issue that needs to be urgently addressed in the development of autonomous driving in urban areas. To this end, the embodiments of the present application disclose a solution, device, equipment and medium for visual occlusion, which can improve the reliability and safety of path planning when visual occlusion occurs during vehicle driving.

[0061] See also Figure 1As shown, the embodiment of the present application discloses a visual occlusion solution method, which comprises the following steps:

[0062] Step S11: identifying surrounding obstacle information during self-vehicle driving.

[0063] In the embodiment, the self-vehicle identifies the surrounding obstacle information in real time during driving.

[0064] In the specific embodiment, the process of identifying the surrounding obstacle information further comprises: classifying the surrounding obstacle information; wherein the obstacle classes include vehicles, pedestrians, traffic signs and traffic lights; and comparing the speed of each surrounding obstacle with a preset speed threshold to determine the type of each surrounding obstacle; wherein the obstacle types include static obstacles and dynamic obstacles.

[0065] That is, the obstacle is also classified during the identification process. Specifically, a series of basic obstacle classes can be pre-set, including vehicles, pedestrians, traffic signs and traffic lights, and vehicles can be further divided into bicycles, small cars, large cars, etc. During self-vehicle driving, the surrounding environment is detected by a visual perception system, and when an obstacle is detected, it is matched and distinguished with the pre-set classes. Through analysis and identification of the visual features such as shape, color and texture of the obstacle, as well as possible motion patterns and other information, the basic class to which the obstacle belongs is preliminarily screened out.

[0066] In addition, it is further determined whether the obstacle is moving or stationary to determine the type of the obstacle, and the obstacle type includes static obstacles and dynamic obstacles. Specifically, a visual + front radar fusion method can be used to determine whether the obstacle is moving or stationary. Generally, by comparing the speed of each obstacle with a preset speed threshold, if the speed is less than the preset speed threshold, it is considered to be stationary, otherwise it is considered to be in motion.

[0067] Step S12: determining an obstacle attention type according to the current driving path and the preset planning path, and matching a corresponding target obstacle from the obstacle information based on the obstacle attention type.

[0068] In the embodiment, it can be understood that in different road scenes, the types of obstacles that need to be paid attention to are different, and not every obstacle around needs to be paid attention to. Therefore, the present application needs to first determine the obstacle attention type according to the current driving path and the preset planning path, and then further match the corresponding target obstacle from the obstacle information, so as to filter out other obstacles that do not need to be paid attention to.

[0069] In the specific embodiment, the above determining the obstacle attention type according to the current driving path and the preset planning path, and matching the corresponding target obstacle from the obstacle information based on the obstacle attention type, comprises: determining the current driving path, and obtaining the preset planning path from the navigation system; judging whether the ego vehicle needs to perform a lane changing operation according to the positional relationship between the current driving path and the preset planning path; if the ego vehicle does not need to perform a lane changing operation, judging whether the ego vehicle is located at an intersection position, so as to determine the corresponding first type of obstacle attention type based on the intersection position judgment result, and then match the corresponding target obstacle from the obstacle information based on the first type of obstacle attention type; if the ego vehicle needs to perform a lane changing operation, obtaining the second type of obstacle attention type based on the first type of obstacle attention type and the obstacle type on the target lane changing side, and then matching the corresponding target obstacle from the obstacle information based on the second type of obstacle attention type.

[0070] It can be understood that the present application needs to determine the current driving path, and obtain the preset planning path from the navigation system, which aims to judge whether the ego vehicle needs to perform a lane changing operation according to the positional relationship between the current driving path and the preset planning path. For example, if the ego vehicle lane is on the navigation planning lane, it means that there is no need to change lanes at this time, and if the ego vehicle is not on the navigation planning lane, it means that a lane changing operation may be needed at this time. In one specific embodiment, if the ego vehicle does not need to perform a lane changing operation, it is further judged whether the ego vehicle is located at an intersection position, so as to determine the corresponding first type of obstacle attention type based on the intersection position judgment result, and then match the corresponding target obstacle from the obstacle information based on the first type of obstacle attention type. In another specific embodiment, if the ego vehicle needs to perform a lane changing operation, the second type of obstacle attention type is obtained based on the first type of obstacle attention type and the obstacle type on the target lane changing side, and then the corresponding target obstacle is matched from the obstacle information based on the second type of obstacle attention type, wherein the obstacle type on the target lane changing side mainly refers to the vehicle on the lane changing side.

[0071] The determination of the first type of obstacle attention type based on the intersection position determination result comprises: if the ego vehicle is not located at the intersection position, a front vehicle on the ego lane is determined as the first type of obstacle attention type; if the ego vehicle is located at the intersection position, target dynamic obstacles and target static obstacles around the intersection position are determined as the first type of obstacle attention type; the target dynamic obstacles comprise pedestrians and non-motor vehicles on both sides of the intersection, and the target static obstacles comprise traffic lights and traffic signs. That is, when the ego vehicle is not at the intersection position, only the front vehicle needs to be focused on, that is, the front vehicle on the ego lane is determined as the first type of obstacle attention type, and when the ego vehicle is at the intersection position, the target dynamic obstacles and the target static obstacles around the intersection position are determined as the first type of obstacle attention type, the target dynamic obstacles specifically comprise pedestrians and non-motor vehicles on both sides of the intersection, and the target static obstacles specifically comprise traffic lights and traffic signs.

[0072] Step S13: The state of the target obstacle is continuously detected, and if a preset disappearance event of the target obstacle occurs, whether the target obstacle is blocked is determined based on the corner point position information of the front vehicle.

[0073] In this embodiment, after the target obstacle that needs to be focused on is determined, the state of the target obstacle is continuously detected, and if a preset disappearance event of the target obstacle occurs, whether the target obstacle is blocked is further determined based on the corner point position information of the front vehicle. For example, when the ego vehicle is not at the intersection position, if the ID of the front vehicle changes, it is considered that a preset disappearance event occurs, wherein the ID change may be that the front vehicle cut out or another vehicle cut in. The front vehicle cut out refers to that the vehicle driving in front suddenly drives away from the driving path of the ego vehicle and is no longer in the front field of view of the ego vehicle or no longer directly blocks or affects the driving of the ego vehicle, for example, the front vehicle may turn or change lanes to other lanes, so that the vehicle condition in front of the ego vehicle changes. The other vehicle cut in refers to that the other vehicle that is not originally on the driving path in front of the ego vehicle suddenly inserts between the ego vehicle and the front vehicle or directly enters the driving path in front of the ego vehicle and becomes a new factor affecting the driving of the ego vehicle. When the ego vehicle is at the intersection position, when the above target obstacle is first detected and the duration is higher than 1s, but the target obstacle disappears in the following period of time, it is considered that a preset disappearance event occurs.

[0074] Step S14: If the target obstacle is blocked, the target obstacle and the state information of the target obstacle before disappearing are stored in a preset blocking array, so that the ego vehicle performs path planning based on the preset blocking array.

[0075] In the embodiment, if it is determined that the target obstacle is blocked, the target obstacle and the state information of the target obstacle before disappearing are stored in the preset blocking array, so that the ego vehicle can perform path planning based on the stored information in the preset blocking array during the period when the target obstacle disappears, to avoid the situation that the ego vehicle makes wrong driving planning due to the blocking of the obstacle, thereby improving the reliability and safety of path planning. That is, in the present application, only the blocked obstacle is stored in the preset blocking array for subsequent path planning of the ego vehicle.

[0076] In the specific embodiment, the storing of the target obstacle and the state information of the target obstacle before disappearing into the preset blocking array includes: if the target obstacle is a traffic light, the traffic light and the display state and second information of the traffic light before disappearing are stored into the preset blocking array; if the target obstacle is a dynamic obstacle, the dynamic obstacle and the speed value, acceleration value and position information of the dynamic obstacle for several frames before disappearing are stored into the preset blocking array. In the embodiment, when storing information into the preset blocking array, taking the target obstacle as a static obstacle traffic light as an example, the identification information corresponding to the traffic light and the display state and second information of the traffic light before disappearing are stored into the preset blocking array. If the target obstacle is a dynamic obstacle, the identification information corresponding to the dynamic obstacle and the speed value, acceleration value and position information of the dynamic obstacle for several frames before disappearing are stored into the preset blocking array, which can be the average speed, acceleration of the last three frames and the position information of the last frame.

[0077] In the subsequent time step, if the target blocking object still exists, for a static obstacle, its range is released according to its latitude and longitude; for a dynamic obstacle, since the average speed, acceleration and position of the last frame before disappearing are read in the last three frames, the position can be updated based on the values at each frame. However, since this method may cause linear update to appear outside the blocking range, if the target obstacle still exists in the preset blocking array, the blocked field of view is used as a bottom value. Then when planning for the ego vehicle, an ST graph is drawn according to these parameters, i.e. the distance s traveled by the target obstacle in the future t time is drawn on the ST graph. From the driving trajectory, when there is a blocking obstacle on the right, the ego vehicle will not change lanes to the right. In the speed planning ST graph, since this method will cause the drawn ST to be lower than the speed of the ego vehicle, the situation that there is a front vehicle and the front vehicle accelerates to cause the ego vehicle to accelerate will not occur.

[0078] Step S15: releasing the information stored in the preset blocking array when the preset release condition is met.

[0079] In this embodiment, when the preset release condition is met, the information stored in the preset occlusion array is released, which can avoid the system storing too much unnecessary information, achieve the purpose of releasing memory and computing resources, and make the system more efficient in processing new perception data and planning tasks.

[0080] It can be seen that in this application, the surrounding obstacle information is identified in real time during the driving of the ego vehicle, and the corresponding target obstacle is matched from the obstacle information according to the obstacle attention type determined according to the current driving path and the preset planning path. It can be understood that in different road scenes, the types of obstacles that need to be paid attention to are different, and not every obstacle around needs to be paid attention to, so this application needs to determine the obstacle attention type according to the current driving path and the preset planning path, and then further match the corresponding target obstacle from the obstacle information to filter out other obstacles that do not need to be paid attention to. After matching the target obstacle, the state of the target obstacle is continuously detected, and if a preset disappearance event of the target obstacle is detected, it is determined whether the target obstacle is occluded based on the corner point position information of the preceding vehicle. If it is determined that the target obstacle is occluded, the target obstacle and the state information of the target obstacle before disappearance are stored in the preset occlusion array, so that during the period when the target obstacle disappears, the ego vehicle can perform path planning based on the information stored in the preset occlusion array, avoiding the situation that the ego vehicle makes wrong driving planning due to the occlusion of the obstacle, thereby improving the reliability and safety of path planning. That is, in this application, only the occluded obstacle is stored in the preset occlusion array for subsequent path planning of the ego vehicle. In addition, when the preset release condition is met, the information stored in the preset occlusion array is released, which can avoid the system storing too much unnecessary information, achieve the purpose of releasing memory and computing resources, and make the system more efficient in processing new perception data and planning tasks.

[0081] Referring to Figure 2 The embodiment of the application discloses a specific method for solving visual occlusion, which is further described and optimized compared with the previous embodiment. Specifically, it includes:

[0082] Step S21: identifying the surrounding obstacle information during the driving of the ego vehicle.

[0083] Step S22: determining the obstacle attention type according to the current driving path and the preset planning path, and matching the corresponding target obstacle from the obstacle information based on the obstacle attention type.

[0084] Step S23: continuously detecting the state of the target obstacle, if the target obstacle occurs a preset disappearance event, establishing a three-dimensional coordinate system with the camera of the ego vehicle as the origin, and determining the position information of each corner point of the front vehicle in the three-dimensional coordinate system according to the position information and length-width-height information of the front vehicle.

[0085] In this embodiment, if the target obstacle occurs a preset disappearance event, it is necessary to perform occlusion discrimination on the target obstacle. First, a three-dimensional coordinate system is established with the camera of the ego vehicle as the origin, and the position information of each corner point of the front vehicle in the three-dimensional coordinate system is determined according to the position information and length-width-height information of the front vehicle. Specifically, the plane corner points can be calculated first, and then the height information is added to obtain the position of the eight corner points of the front vehicle, as shown in FIG. 8. Figure 3

[0086] Step S24: connecting each corner point of the front vehicle with the origin based on the position information of each corner point to obtain a corresponding ray, and determining an occlusion space based on the region formed by each ray.

[0087] In this embodiment, in the three-dimensional space, based on the position information of each corner point, a ray is drawn from the origin to each of the eight corner points of the front vehicle, and then an occlusion space is determined based on the region formed by each ray. As shown in FIG. 9, the estimated position of the occlusion object red light is known, the ray is truncated at the position x+i (x is the longitudinal distance of the red light, and i is a parameter that can be calibrated), and the eight truncated points are connected in turn to obtain a polyhedron. In the figure, four lines are taken as an example, and the polyhedron is the occlusion space. Figure 3

[0088] Step S25: determining whether the target obstacle is located in the occlusion space according to the state information of the target obstacle before disappearance, if yes, determining that the target obstacle is occluded, and storing the target obstacle and the state information of the target obstacle before disappearance into a preset occlusion array, so that the ego vehicle performs path planning based on the preset occlusion array.

[0089] In this embodiment, it is determined whether the target obstacle is located in the occlusion space according to the state information of the target obstacle before disappearance. Taking a static obstacle as an example, since the position information of the last frame of the target obstacle before disappearance can be obtained, the position information can be converted into latitude and longitude coordinates, and then it can be determined whether the static obstacle is located in the occlusion space according to the latitude and longitude coordinates of the static obstacle. The judgment process of a dynamic obstacle such as a vehicle is also similar, but only when the eight corner points of the vehicle are located in the occlusion space, it is considered to be occluded.

[0090] ​​In a specific embodiment, determining whether the target obstacle is within the occlusion space based on the state information of the target obstacle before it disappears includes: projecting the occlusion space into a height space and a ground space, respectively; wherein the ground space is a two-dimensional space parallel to the vehicle's travel plane, and the height space is a two-dimensional space perpendicular to the ground; obtaining the spatial position coordinates of the target obstacle in the last frame before it disappears, and determining whether the height coordinates of the spatial position coordinates are within the height space; if so, determining whether the ground coordinates of the spatial position coordinates are within the ground space; if so, determining that the target obstacle is within the occlusion space. That is, the present application requires projecting the occlusion space into a height space (yz plane) and a ground space (xy plane), respectively. The ground space is a two-dimensional space parallel to the vehicle's travel plane, roughly corresponding to the actual ground, and is a space in which the positional relationship of objects in three-dimensional space is projected onto a horizontal plane for analysis and determination; the height space is a two-dimensional space perpendicular to the ground. For example, determining whether a static obstacle is above or below an occlusion ray formed by an object in front of the vehicle is used to determine whether it is within the occlusion range. Furthermore, the spatial position coordinates of the target obstacle in the last frame before it disappears are obtained. First, it is determined whether the height position coordinates in the spatial position coordinates are located in the height space, such as Figure 4 As shown, after the projection is completed, it is calculated whether the red light is within the ray area. When it is determined that it is located in the height space, the ground position coordinates in the spatial position coordinates are judged to be located in the ground space. When both are consistent, it is determined that the target obstacle is located in the occlusion space, that is, the target obstacle is blocked. By combining the judgment of the height space with the ground space, it is possible to comprehensively determine whether the object is in the effective occlusion area, providing a more accurate basis for the decision-making of the autonomous driving vehicle. Among them, when making judgments in the height space and the ground space, the angle sum method can be specifically used: calculate the sum of the angles of the point relative to all vertices of the polygon. If the sum is 360 degrees, it means that the point is inside the polygon; if it is less than 360 degrees, the point is outside the polygon; if it is equal to 0 degrees or greater than 360 degrees, the point is on the edge of the polygon.

[0091] It should be noted that before the above step of determining whether the target obstacle is located in the occlusion space according to the state information of the target obstacle before disappearing, further includes: if the target obstacle is a static obstacle, determining whether the target obstacle is in a preset stable state according to the state information of the target obstacle in several frames before disappearing; if not, prohibiting the step of determining whether the target obstacle is located in the occlusion space according to the state information of the target obstacle before disappearing. That is, if the target obstacle is a static obstacle, a stable state judgment needs to be performed on the static obstacle before occlusion judgment, that is, whether the target obstacle is in a preset stable state is determined according to the state information of the target obstacle in several frames before disappearing. Specifically, the position error of the target obstacle in the last three frames before disappearing needs to be not higher than vt+a, where v refers to the speed of the ego vehicle; a refers to a preset distance error a; can be calibrated according to the actual scene; t is the time difference between two frames; if this condition is not met, the target obstacle is directly excluded, that is, the step of determining whether the target obstacle is located in the occlusion space according to the state information of the target obstacle before disappearing is prohibited.

[0092] Step S26: releasing the information stored in the preset occlusion array when the preset release condition is met.

[0093] In the specific embodiment, the above releasing the information stored in the preset occlusion array when the preset release condition is met includes: for any static obstacle corresponding to the information stored in the preset occlusion array, if the any static obstacle is re-identified, or if it is determined according to the state information of the any static obstacle before disappearing that the any static obstacle has exceeded the current detection area, the information related to the any static obstacle stored in the preset occlusion array is released; for any dynamic obstacle corresponding to the information stored in the preset occlusion array, the current latest position of the any dynamic obstacle is predicted according to the state information of the any dynamic obstacle before disappearing, if it is determined based on the current latest position that the ego vehicle has passed the any dynamic obstacle, or if the ego vehicle has completed a lane changing operation, or if the any dynamic obstacle is re-identified, or if there is another vehicle passing through the occlusion area, the information related to the any dynamic obstacle stored in the preset occlusion array is released.

[0094] It can be understood that the information in the preset occlusion array is not stored for a long time. When the preset release condition is met, the information stored in the preset occlusion array is released to achieve the purpose of releasing memory and computing resources.

[0095] For static obstacles, considering that there are errors in sensor identification and self-vehicle motion estimation, when a static obstacle such as a traffic light or a traffic sign is identified again according to its latitude and longitude coordinates and the error with the stored latitude and longitude coordinates is within a 3m radius, the information related to the static obstacle stored in the preset occlusion array is released. Or when the static obstacle has exceeded the current detection area of the self-vehicle camera, that is, the self-vehicle camera detection area no longer covers the static obstacle, the static obstacle is also released from the preset occlusion array. For dynamic obstacles, when the self-vehicle exceeds the target vehicle, the self-vehicle has completed the lane changing operation, the dynamic obstacle is detected again, or other vehicles pass through the occlusion area, the information related to any dynamic obstacle stored in the preset occlusion array is released.

[0096] As a result, it is possible to avoid the system storing too much unnecessary information, release memory and computing resources, enable the system to more efficiently process new perception data and planning tasks, maintain smooth operation of the system, and improve processing speed and response capability.

[0097] The more specific processing procedures of steps S21 and S22 will be described below with reference to the corresponding contents disclosed in the foregoing embodiments.

[0098] It can be seen that, in the occlusion discrimination of the target obstacle, a three-dimensional coordinate system is established with the self-vehicle camera as the origin, the position information of each corner point of the front vehicle in the three-dimensional coordinate system is determined according to the position information and length-width-height information of the front vehicle, the rays are drawn from the origin to the eight corner points of the front vehicle based on the position information of each corner point, the occlusion space is determined based on the area formed by the rays, and then the occlusion discrimination is performed according to the position information between the state information of the target obstacle before disappearance and the occlusion space. In addition, the information in the preset occlusion array is not stored for a long time, and when the preset release condition is met, the information stored in the preset occlusion array is released to release memory and computing resources. The application sets corresponding release conditions for static obstacles and dynamic obstacles, and releases the information stored in the preset occlusion array when the conditions are met.

[0099] Referring to Figure 5 The embodiment of the application discloses a solution device for visual occlusion, which comprises:

[0100] The identification module 11 is configured to identify the obstacle information around the self-vehicle during driving of the self-vehicle.

[0101] The matching module 12 is configured to determine the obstacle attention type according to the current driving path and the preset planning path, and match the corresponding target obstacle from the obstacle information based on the obstacle attention type.

[0102] The shielding detection module 13 is configured to continuously detect the state of the target obstacle, and if the target obstacle has a preset disappearance event, determine whether the target obstacle is shielded based on the corner point position information of the preceding vehicle.

[0103] The storage module 14 is configured to, if the target obstacle is shielded, store the target obstacle and the state information of the target obstacle before disappearing into a preset shielding array, so that the ego vehicle plans a path based on the preset shielding array.

[0104] The release module 15 is configured to release the information stored in the preset shielding array when a preset release condition is met.

[0105] As can be seen, in the present application, the surrounding obstacle information is identified in real time during the driving of the ego vehicle, and the target obstacle corresponding to the obstacle information is matched from the obstacle information according to the determined obstacle attention type based on the current driving path and the preset planning path. It can be understood that in different road scenes, the types of obstacles that need to be paid attention to are different, and not every obstacle around needs to be paid attention to. Therefore, the present application needs to first determine the obstacle attention type based on the current driving path and the preset planning path, and then further match the corresponding target obstacle from the obstacle information to filter out other obstacles that do not need to be paid attention to. After matching the target obstacle, the state of the target obstacle is continuously detected, and if the target obstacle has a preset disappearance event, it is determined whether the target obstacle is shielded based on the corner point position information of the preceding vehicle. If it is determined that the target obstacle is shielded, the target obstacle and the state information of the target obstacle before disappearing are stored in a preset shielding array, so that during the period when the target obstacle disappears, the ego vehicle can plan a path based on the information stored in the preset shielding array, avoiding the situation that the ego vehicle makes an incorrect driving plan due to the shielding of the obstacle, thereby improving the reliability and safety of the path planning. That is, in the present application, only the shielded obstacle is stored in the preset shielding array for subsequent path planning by the ego vehicle. In addition, when the preset release condition is met, the information stored in the preset shielding array is released, which can avoid storing too much unnecessary information by the system, achieve the purpose of releasing memory and computing resources, and make the system more efficient in processing new perception data and planning tasks.

[0106] Since the embodiments of the device part correspond to the embodiments of the method part, the embodiments of the device part are described with reference to the embodiments of the method part, which are not described here. Moreover, the embodiments have the same beneficial effects as the above-mentioned method of solving visual shielding.

[0107] Figure 6A structural schematic diagram of an electronic device is provided in the embodiments of the present application. Specifically, it can include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is configured to store a computer program, which is loaded and executed by the processor 21 to implement the related steps in the method for solving visual occlusion performed by an electronic device disclosed in any of the preceding embodiments.

[0108] In the embodiments, the power supply 23 is configured to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 is capable of creating a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which is not limited specifically herein; the input / output interface 25 is configured to obtain external input data or output data to the outside world, and the specific interface type can be selected according to the specific application needs, which is not limited specifically herein.

[0109] The processor 21 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 21 can also include a main processor and a coprocessor. The main processor is a processor for processing data in a wake-up state, also known as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 can also include an AI (Artificial Intelligence) processor for processing machine learning-related computing operations.

[0110] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc. The resources stored thereon include an operating system 221, a computer program 222, and data 223, etc. The storage mode can be temporary storage or permanent storage.

[0111] The operating system 221 is used to manage and control each hardware device on the electronic device 20 and the computer program 222, so as to realize the operation and processing of the processor 21 on the mass data 223 in the memory 22, and can be Windows, Unix, Linux, etc. In addition to the computer program capable of completing the solving method of visual occlusion disclosed by the electronic device 20 in any of the foregoing embodiments, the computer program 222 can further include a computer program capable of completing other specific work. In addition to the data that can be received by the electronic device from the data transmitted by the external device, the data 223 can also include the data collected by the self input and output interface 25, etc.

[0112] Further, the embodiment of the present application further discloses a computer readable storage medium, the storage medium stores a computer program, and the computer program is loaded and executed by the processor to realize the solving method steps of visual occlusion disclosed by any of the foregoing embodiments.

[0113] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed by the embodiment, since it corresponds to the method disclosed by the embodiment, the description is relatively simple, and the related parts can be referred to the method part.

[0114] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized by electronic hardware, computer software or combination of the two. In order to clearly show the interchangeability of hardware and software, the components and steps of each example have been described in the above description. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0115] The steps of the method or algorithm described in combination with the embodiments disclosed in the present application can be directly implemented by hardware, software module executed by the processor, or combination of the two. The software module can be placed in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, compact disc read-only memory (CD-ROM), or any other form of storage medium known in the art.

[0116] Finally, it needs to be pointed out that, in this article, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the term "includes", "contains" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "includes a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0117] The above describes in detail the method, device, equipment and storage medium for solving visual occlusion provided by the present application. The principles and implementation manners of the present application are described by applying specific examples in this article. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A solution to visual occlusion, characterized in that: include: Identify surrounding obstacles while the vehicle is driving; Determining an obstacle attention type according to a current driving path and a preset planned path, and matching a corresponding target obstacle from the obstacle information based on the obstacle attention type; Continuously detecting the state of the target obstacle, and if a preset disappearance event occurs for the target obstacle, determining whether the target obstacle is blocked based on the corner point position information of the preceding vehicle; If blocked, the target obstacle and the state information of the target obstacle before disappearing are stored in a preset blocking array, so that the vehicle can perform path planning based on the preset blocking array; When a preset release condition is met, the information stored in the preset occlusion array is released.

2. The solution to visual occlusion according to claim 1, characterized in that: The process of identifying surrounding obstacle information also includes: Classify surrounding obstacle information into categories; obstacle categories include vehicles, pedestrians, traffic signs, and traffic lights; The speed of each surrounding obstacle is compared with a preset speed threshold to determine the type of each surrounding obstacle; wherein the types of obstacles include static obstacles and dynamic obstacles.

3. The solution to visual occlusion according to claim 1, characterized in that: The determining of the obstacle attention type according to the current driving path and the preset planned path, and matching the corresponding target obstacle from the obstacle information based on the obstacle attention type, includes: Determine the current driving path and obtain the preset planned path from the navigation system; Determining whether the vehicle needs to perform a lane change operation based on a positional relationship between the current driving path and the preset planned path; If the ego vehicle does not need to perform a lane change maneuver, determining whether the ego vehicle is located at an intersection, determining a corresponding first type of obstacle attention based on the intersection position determination result, and then matching a corresponding target obstacle from the obstacle information based on the first type of obstacle attention; If the ego vehicle needs to perform a lane change operation, a second obstacle attention type is obtained based on the first obstacle attention type and the obstacle type on the target lane change side, and then a corresponding target obstacle is matched from the obstacle information based on the second obstacle attention type.

4. The solution to visual occlusion according to claim 3, characterized in that: The determining of the corresponding first type of obstacle concern type based on the intersection position judgment result includes: If the vehicle is not at the intersection, the vehicle ahead in the lane is determined as a first type of obstacle of concern; If the vehicle is located at an intersection, the target dynamic obstacles and target static obstacles around the intersection are determined as the first type of obstacle attention type; the target dynamic obstacles include pedestrians and non-motor vehicles on both sides of the intersection, and the target static obstacles include traffic lights and traffic signs.

5. The solution to visual occlusion according to claim 1, characterized in that: The determining whether the target obstacle is blocked based on the corner point position information of the preceding vehicle includes: Establishing a three-dimensional coordinate system with the vehicle camera as the origin, and determining the position information of each corner point of the preceding vehicle in the three-dimensional coordinate system based on the position information and length, width and height information of the preceding vehicle; Connecting each corner point of the preceding vehicle with the origin based on the position information of each corner point to obtain corresponding rays, and determining the blocked space based on the area formed by each ray; It is determined whether the target obstacle is located in the obstruction space according to the state information before the target obstacle disappears, and if so, it is determined that the target obstacle is obstructed.

6. The solution to visual occlusion according to claim 5, characterized in that: The determining whether the target obstacle is located in the obstructed space according to the state information of the target obstacle before disappearing includes: Projecting the occluded space into a height space and a ground space respectively; wherein the ground space is a two-dimensional space parallel to the vehicle's travel plane, and the height space is a two-dimensional space perpendicular to the ground; Obtaining the spatial position coordinates of the last frame before the target obstacle disappears, and determining whether the height position coordinates in the spatial position coordinates are located in the height space; If so, it is determined whether the ground position coordinates in the spatial position coordinates are located in the ground space; if so, it is determined that the target obstacle is located in the blocking space.

7. The solution to visual occlusion according to claim 5, characterized in that: Before determining whether the target obstacle is located in the blocking space based on the state information of the target obstacle before disappearing, the method further includes: If the target obstacle is a static obstacle, determining whether the target obstacle is in a preset stable state based on state information of several frames before the target obstacle disappears; If not, the step of determining whether the target obstacle is located within the blocking space based on the state information of the target obstacle before it disappears is prohibited.

8. The solution to visual occlusion according to claim 1, characterized in that: The storing the target obstacle and the state information of the target obstacle before disappearing into a preset occlusion array includes: If the target obstacle is a traffic light, storing the traffic light and the display state and number of seconds before the traffic light disappears in a preset occlusion array; If the target obstacle is a dynamic obstacle, the dynamic obstacle and the velocity value, acceleration value and position information of the dynamic obstacle in several frames before the dynamic obstacle disappears are stored in a preset occlusion array.

9. The solution to visual obstruction according to any one of claims 1 to 8, characterized in that: When a preset release condition is met, releasing the information stored in the preset occlusion array includes: For any static obstacle corresponding to the information stored in the preset occlusion array, if the static obstacle is re-identified, or if it is determined based on the state information of the static obstacle before it disappears that the static obstacle has exceeded the current detection area, then releasing the information related to the static obstacle stored in the preset occlusion array; For any dynamic obstacle corresponding to the information stored in the preset occlusion array, a current latest position of the any dynamic obstacle is predicted based on the state information of the any dynamic obstacle before it disappears. If it is determined based on the current latest position that the ego vehicle has passed the any dynamic obstacle, or if the ego vehicle has completed a lane change operation, or if the any dynamic obstacle is re-identified, or if another vehicle passes through the occlusion area, then the information related to the any dynamic obstacle stored in the preset occlusion array is released.

10. A device for solving visual occlusion, characterized in that: include: The recognition module is used to identify surrounding obstacle information while the vehicle is driving; a matching module, configured to determine an obstacle attention type according to a current driving path and a preset planned path, and to match a corresponding target obstacle from the obstacle information based on the obstacle attention type; an occlusion detection module, configured to continuously detect the state of the target obstacle and, if a preset disappearance event occurs to the target obstacle, determine whether the target obstacle is occluded based on the corner position information of the preceding vehicle; a storage module, configured to store the target obstacle and state information of the target obstacle before it disappears into a preset occlusion array if the target obstacle is blocked, so that the ego vehicle can perform path planning based on the preset occlusion array; The release module is used to release the information stored in the preset occlusion array when a preset release condition is met.

11. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the steps of the visual occlusion solution according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that Used to store a computer program; wherein, when the computer program is executed by a processor, the steps of the solution to visual occlusion according to any one of claims 1 to 9 are implemented.

Images