Visual occlusion solving method and device, equipment and medium

Abstract

The invention discloses a visual occlusion solving method and device, equipment and a medium, and relates to the field of automatic driving, and the method comprises the steps: recognizing surrounding obstacle information in a vehicle driving process; matching a corresponding target obstacle from the obstacle information by using an obstacle attention type determined according to a current driving path and a preset planning path; the state of the target obstacle is continuously detected, and if a preset disappearance event happens to the target obstacle, whether the target obstacle is shielded or not is judged based on the angular point position information of the front vehicle; if the target obstacle is shielded, storing the target obstacle and the state information of the target obstacle before disappearance into a preset shielding array, so that the vehicle performs path planning based on the preset shielding array; and when a preset release condition is met, releasing the information stored in the preset shielding array. According to the method and the device, the reliability and the safety of path planning can be improved when visual occlusion occurs in the vehicle driving process.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and in particular to a solution, device, equipment and medium for visual occlusion. Background Art

[0002] In urban autonomous driving scenarios, current visual perception technology has obvious defects. It can only record objects perceived at the current moment, and lacks the ability to continuously record objective objects that always exist in the environment but may be temporarily obscured.

[0003] For example, at intersections, large vehicles often block traffic lights. Since the visual perception system of autonomous vehicles cannot retain the traffic light information before being blocked, the vehicle cannot make correct driving plans when it cannot see the traffic lights. This situation is very likely to cause vehicles to run red lights and other serious violations of traffic rules, which not only poses a huge threat to traffic safety, but also affects the reliability and practicality of autonomous driving technology, becoming a key issue that needs to be urgently resolved in the development of urban autonomous driving.

[0004] In summary, when visual occlusion occurs during vehicle driving, how to improve the reliability and safety of path planning is a problem that needs to be solved. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a solution, device, equipment and medium for visual occlusion, which can improve the reliability and safety of path planning when visual occlusion occurs during vehicle driving. The specific solution is as follows:

[0006] In a first aspect, the present application discloses a solution to visual occlusion, including:

[0007] During the driving process of the vehicle, identify the surrounding obstacle information;

[0008] Determine an obstacle attention type according to the current driving path and the preset planned path, and match a corresponding target obstacle from the obstacle information based on the obstacle attention type;

[0009] Continuously detecting the state of the target obstacle, and if a preset disappearance event occurs to the target obstacle, determining whether the target obstacle is blocked based on the corner point position information of the preceding vehicle;

[0010] If blocked, the target obstacle and the state information of the target obstacle before disappearing are stored in a preset blocking array, so that the vehicle can perform path planning based on the preset blocking array;

[0011] When a preset release condition is met, the information stored in the preset occlusion array is released.

[0012] Optionally, the process of identifying surrounding obstacle information further includes:

[0013] Classify surrounding obstacle information into categories; the categories of obstacles include vehicles, pedestrians, traffic signs and traffic lights;

[0014] The speed of each surrounding obstacle is compared with a preset speed threshold to determine the type of each surrounding obstacle; wherein the types of obstacles include static obstacles and dynamic obstacles.

[0015] Optionally, determining the obstacle attention type according to the current driving path and the preset planned path, and matching the corresponding target obstacle from the obstacle information based on the obstacle attention type, includes:

[0016] Determine the current driving path and obtain the preset planned path from the navigation system;

[0017] Determining whether the vehicle needs to perform a lane change operation according to the position relationship between the current driving path and the preset planned path;

[0018] If the ego vehicle does not need to perform a lane change operation, determining whether the ego vehicle is located at an intersection, determining a corresponding first type of obstacle attention type based on the intersection position determination result, and then matching a corresponding target obstacle from the obstacle information based on the first type of obstacle attention type;

[0019] If the ego vehicle needs to perform a lane change operation, a second obstacle attention type is obtained based on the first obstacle attention type and the obstacle type on the target lane change side, and then a corresponding target obstacle is matched from the obstacle information based on the second obstacle attention type.

[0020] Optionally, the determining of the corresponding first category obstacle attention type based on the intersection position judgment result includes:

[0021] If the vehicle is not located at the intersection, the front vehicle on the lane is determined as the first type of obstacle concern;

[0022] If the vehicle is located at an intersection, the target dynamic obstacles and target static obstacles around the intersection are determined as the first type of obstacle attention type; the target dynamic obstacles include pedestrians and non-motor vehicles on both sides of the intersection, and the target static obstacles include traffic lights and traffic signs.

[0023] Optionally, judging whether the target obstacle is blocked based on the corner point position information of the preceding vehicle includes:

[0024] A three-dimensional coordinate system is established with the camera of the vehicle as the origin, and the position information of each corner point of the front vehicle in the three-dimensional coordinate system is determined according to the position information and length, width and height information of the front vehicle;

[0025] Connecting each corner point of the preceding vehicle with the origin based on the position information of each corner point to obtain a corresponding ray, and determining the shielding space based on the area formed by each ray;

[0026] It is determined whether the target obstacle is located in the blocking space according to the state information of the target obstacle before it disappears, and if so, it is determined that the target obstacle is blocked.

[0027] Optionally, judging whether the target obstacle is located in the blocked space according to the state information of the target obstacle before disappearing includes:

[0028] The blocked space is projected to the height space and the ground space respectively; wherein the ground space is a two-dimensional space parallel to the vehicle travel plane, and the height space is a two-dimensional space perpendicular to the ground;

[0029] Obtaining the spatial position coordinates of the last frame before the target obstacle disappears, and determining whether the height position coordinates in the spatial position coordinates are located in the height space;

[0030] If so, it is determined whether the ground position coordinates in the spatial position coordinates are located in the ground space; if so, it is determined that the target obstacle is located in the blocking space.

[0031] Optionally, before judging whether the target obstacle is located in the blocking space according to the state information of the target obstacle before disappearing, the method further includes:

[0032] If the target obstacle is a static obstacle, judging whether the target obstacle is in a preset stable state according to state information of several frames before the target obstacle disappears;

[0033] If not, the step of determining whether the target obstacle is located within the blocked space based on the state information of the target obstacle before it disappears is prohibited.

[0034] Optionally, storing the target obstacle and the state information of the target obstacle before disappearing into a preset occlusion array includes:

[0035] If the target obstacle is a traffic light, the traffic light and the display state and seconds of the traffic light before disappearing are stored in a preset occlusion array;

[0036] If the target obstacle is a dynamic obstacle, the dynamic obstacle and the velocity value, acceleration value and position information of several frames before the dynamic obstacle disappears are stored in a preset occlusion array.

[0037] Optionally, when a preset release condition is met, releasing the information stored in the preset occlusion array includes:

[0038] For any static obstacle corresponding to the information stored in the preset occlusion array, if the any static obstacle is re-identified, or if it is determined that the any static obstacle has exceeded the current detection area according to the state information of the any static obstacle before disappearance, then the information related to the any static obstacle stored in the preset occlusion array is released;

[0039] For any dynamic obstacle corresponding to the information stored in the preset occlusion array, the current latest position of any dynamic obstacle is predicted according to the state information of any dynamic obstacle before it disappears; if it is determined based on the current latest position that the ego vehicle has exceeded any dynamic obstacle, or if the ego vehicle has completed a lane change operation, or if any dynamic obstacle is re-identified, or if other vehicles pass through the occlusion area, the information related to the any dynamic obstacle stored in the preset occlusion array is released.

[0040] In a second aspect, the present application discloses a device for solving visual occlusion, comprising:

[0041] The recognition module is used to identify surrounding obstacle information while the vehicle is driving;

[0042] A matching module, used to determine the obstacle attention type according to the current driving path and the preset planned path, so as to match the corresponding target obstacle from the obstacle information based on the obstacle attention type;

[0043] An occlusion detection module is used to continuously detect the state of the target obstacle, and if the target obstacle disappears in a preset manner, determine whether the target obstacle is occluded based on the corner point position information of the preceding vehicle;

[0044] A storage module, for storing the target obstacle and the state information of the target obstacle before disappearing into a preset occlusion array if blocked, so that the ego vehicle can perform path planning based on the preset occlusion array;

[0045] The release module is used to release the information stored in the preset occlusion array when a preset release condition is met.

[0046] In a third aspect, the present application discloses an electronic device, comprising:

[0047] Memory, used to store computer programs;

[0048] A processor is used to execute the computer program to implement the steps of the aforementioned disclosed solution to visual occlusion.

[0049] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the aforementioned solution to visual occlusion are implemented.

[0050] It can be seen that the present application identifies surrounding obstacle information during the driving process of the self-vehicle; determines the obstacle attention type according to the current driving path and the preset planned path, so as to match the corresponding target obstacle from the obstacle information based on the obstacle attention type; continuously detects the state of the target obstacle, and if the target obstacle has a preset disappearance event, determines whether the target obstacle is blocked based on the corner point position information of the preceding vehicle; if blocked, stores the target obstacle and the state information of the target obstacle before disappearance in a preset blocking array, so that the self-vehicle can perform path planning based on the preset blocking array; and releases the information stored in the preset blocking array when the preset release condition is met.

[0051] Beneficial effect: In this application, the surrounding obstacle information will be identified in real time during the driving process of the vehicle, and the corresponding target obstacle will be matched from the obstacle information using the obstacle attention type determined according to the current driving path and the preset planned path. It can be understood that in different road scenes, the obstacle types that need to be paid attention to are different, and it is not necessary to pay attention to every obstacle around. Therefore, this application needs to first determine the obstacle attention type according to the current driving path and the preset planned path, and then further match the corresponding target obstacle from the obstacle information to filter out other obstacles that do not need to be paid attention to. After matching the required target obstacle, the state of the target obstacle is continuously detected. If the preset disappearance event of the target obstacle is detected, it is determined whether the target obstacle is blocked based on the corner position information of the front vehicle. If it is determined that the target obstacle is blocked, the target obstacle and the state information of the target obstacle before disappearance are stored in the preset blocking array, so that during the time period when the target obstacle disappears, the vehicle can plan the path based on the information stored in the preset blocking array, avoiding the situation where the vehicle makes an erroneous driving plan due to the occlusion of the obstacle, thereby improving the reliability and safety of the path planning. That is, in the present application, only obstacles that are determined to be blocked will be stored in the preset blocking array, so that the vehicle can perform path planning later. In addition, when the preset release condition is met, the information stored in the preset blocking array is released, which can avoid the system from storing too much unnecessary information, achieve the purpose of releasing memory and computing resources, and enable the system to process new perception data and planning tasks more efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0053] Figure 1 A flow chart of a solution to visual occlusion disclosed in this application;

[0054] Figure 2 A flowchart of a specific visual occlusion solution disclosed in this application;

[0055] Figure 3 A schematic diagram of constructing an occlusion space in a three-dimensional space disclosed in the present application;

[0056] Figure 4 A schematic diagram of a height space disclosed in the present application;

[0057] Figure 5 This is a schematic diagram of the structure of a visual occlusion solution device disclosed in this application;

[0058] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0060] The current visual perception technology has obvious defects. It can only record the objects perceived at the current moment, and lacks the ability to continuously record those objective objects that have always existed in the environment but may be temporarily obscured. This situation is very likely to cause vehicles to violate traffic regulations, such as running red lights. It not only poses a huge threat to traffic safety, but also affects the reliability and practicality of autonomous driving technology, becoming a key issue that needs to be urgently resolved in the development of urban autonomous driving. To this end, the embodiments of the present application disclose a solution, device, equipment and medium for visual occlusion, which can improve the reliability and safety of path planning when visual occlusion occurs during vehicle driving.

[0061] See also Figure 1As shown, the embodiment of the present application discloses a solution to visual occlusion, the method comprising:

[0062] Step S11: Identify surrounding obstacle information while the vehicle is traveling.

[0063] In this embodiment, the vehicle can identify surrounding obstacle information in real time while driving.

[0064] In a specific implementation, the above process of identifying surrounding obstacle information also includes: classifying the surrounding obstacle information; wherein the categories of obstacles include vehicles, pedestrians, traffic signs and traffic lights; comparing the speed of each surrounding obstacle with a preset speed threshold to determine the type of each surrounding obstacle; wherein the types of obstacles include static obstacles and dynamic obstacles.

[0065] That is, obstacles will also be classified during the process of obstacle recognition. Specifically, a series of basic categories that may become obstacles can be pre-set, including vehicles, pedestrians, traffic signs and traffic lights. Vehicles can also be subdivided into two-wheeled vehicles, small cars, large cars, etc. During the driving process of the vehicle, the visual perception system is used to detect the surrounding environment. When an obstacle is detected, it is matched and distinguished with these pre-set categories. By analyzing and identifying the visual features of the obstacle such as shape, color, texture, and possible movement patterns, the basic category to which it belongs is preliminarily screened.

[0066] In addition, it will further determine whether the obstacle is moving or stationary to determine the type of obstacle, which includes static obstacles and dynamic obstacles. Specifically, the vision + front radar fusion method can be used to determine whether the obstacle is moving or stationary. Generally, the speed of each obstacle is compared with the preset speed threshold. If the speed is less than the preset speed threshold, it is considered to be stationary, otherwise it is considered to be in motion.

[0067] Step S12: determining an obstacle attention type according to the current driving path and the preset planned path, so as to match a corresponding target obstacle from the obstacle information based on the obstacle attention type.

[0068] In this embodiment, it can be understood that in different road scenarios, the types of obstacles that need to be paid attention to are different, and it is not necessary to pay attention to every obstacle around. Therefore, this application needs to first determine the obstacle attention type based on the current driving path and the preset planned path, and then further match the corresponding target obstacle from the obstacle information to filter out other obstacles that do not need to be paid attention to.

[0069] In a specific implementation, the above-mentioned determining the obstacle attention type according to the current driving path and the preset planned path, so as to match the corresponding target obstacle from the obstacle information based on the obstacle attention type, includes: determining the current driving path, and obtaining the preset planned path from the navigation system; judging whether the self-vehicle needs to perform a lane change operation according to the position relationship between the current driving path and the preset planned path; if the self-vehicle does not need to perform a lane change operation, judging whether the self-vehicle is located at an intersection, so as to determine the corresponding first type of obstacle attention type based on the intersection position judgment result, and then matching the corresponding target obstacle from the obstacle information based on the first type of obstacle attention type; if the self-vehicle needs to perform a lane change operation, obtaining a second type of obstacle attention type based on the first type of obstacle attention type and the obstacle type on the target lane change side, and then matching the corresponding target obstacle from the obstacle information based on the second type of obstacle attention type.

[0070] It is understandable that the present application needs to determine the current driving path and obtain the preset planning path from the navigation system, the purpose of which is to determine whether the vehicle needs to perform a lane change operation according to the positional relationship between the current driving path and the preset planning path. For example, if the vehicle's lane is on the lane planned by the navigation, it means that there is no need to change lanes at this time. If the vehicle's lane is not on the lane planned by the navigation, it means that it may be necessary to perform a lane change operation at this time. In a specific implementation, if the vehicle does not need to perform a lane change operation, it is further determined whether the vehicle is located at the intersection position, so as to determine the corresponding first type of obstacle attention type based on the intersection position judgment result, and then match the corresponding target obstacle from the obstacle information based on the first type of obstacle attention type. In another specific implementation, if the vehicle needs to perform a lane change operation, then on the basis of the first type of obstacle attention type, the obstacle type on the target lane change side is added to obtain the second type of obstacle attention type, and then the corresponding target obstacle is matched from the obstacle information based on the second type of obstacle attention type, wherein the obstacle type on the target lane change side mainly refers to the vehicle on the lane change side.

[0071] Among them, the determination of the corresponding first-category obstacle attention type based on the intersection position judgment result includes: if the self-vehicle is not located at the intersection, the front vehicle on the self-lane is determined as the first-category obstacle attention type; if the self-vehicle is located at the intersection, the target dynamic obstacles and target static obstacles around the intersection are determined as the first-category obstacle attention type; the target dynamic obstacles include pedestrians and non-motor vehicles on both sides of the intersection, and the target static obstacles include traffic lights and traffic signs. That is, when the self-vehicle is not at the path position, only the front vehicle needs to be paid attention to, that is, the front vehicle on the self-lane is determined as the first-category obstacle attention type, and when the self-vehicle is at the intersection, the target dynamic obstacles and target static obstacles around the intersection are determined as the first-category obstacle attention type, the target dynamic obstacles specifically include pedestrians and non-motor vehicles on both sides of the intersection, and the target static obstacles specifically include traffic lights and traffic signs.

[0072] Step S13: continuously detecting the state of the target obstacle, and if the target obstacle has a preset disappearance event, determining whether the target obstacle is blocked based on the corner point position information of the preceding vehicle.

[0073] In this embodiment, after determining the target obstacle that needs attention, the state of the target obstacle will be continuously detected. If the target obstacle has a preset disappearance event, it is necessary to further determine whether the target obstacle is blocked based on the corner position information of the front vehicle. For example, when the self-vehicle is not at the intersection, if the ID of the front vehicle changes, it is considered that the preset disappearance event has occurred, wherein the ID change may be the front vehicle cut out or the other vehicle cut in. The front vehicle cut out refers to the vehicle in front suddenly leaving the driving path of the self-vehicle, no longer in the front field of view of the self-vehicle or no longer directly blocking or affecting the driving of the self-vehicle. For example, the front vehicle may have turned, changed lanes to other lanes, etc., so that the vehicle situation in front of the self-vehicle changes. The other vehicle cut in refers to other vehicles that were not originally on the driving path in front of the self-vehicle suddenly inserted between the self-vehicle and the front vehicle or directly entered the front of the driving path of the self-vehicle, becoming a new vehicle factor affecting the driving of the self-vehicle. When the self-vehicle is at the intersection, when the above target obstacle is detected for the first time and the duration is longer than 1s, but the target obstacle disappears within the next period of time, it is considered that the preset disappearance event has occurred.

[0074] Step S14: If blocked, the target obstacle and the state information of the target obstacle before disappearing are stored in a preset blocking array, so that the ego vehicle can perform path planning based on the preset blocking array.

[0075] In this embodiment, if it is determined that the target obstacle is blocked, the target obstacle and the state information of the target obstacle before it disappears need to be stored in the preset blocking array, so that during the time period when the target obstacle disappears, the vehicle can perform path planning based on the information stored in the preset blocking array, avoiding the situation where the vehicle makes an erroneous driving plan due to the obstacle being blocked, thereby improving the reliability and safety of path planning. That is, in this application, only obstacles that are determined to be blocked will be stored in the preset blocking array so that the vehicle can perform path planning later.

[0076] In a specific implementation, the above-mentioned storing the target obstacle and the state information of the target obstacle before disappearing into the preset occlusion array includes: if the target obstacle is a traffic light, the display state and the number of seconds of the traffic light before disappearing are stored into the preset occlusion array; if the target obstacle is a dynamic obstacle, the speed value, acceleration value and position information of the dynamic obstacle and the dynamic obstacle in several frames before disappearing are stored into the preset occlusion array. In this embodiment, when storing information into the preset occlusion array, taking the target obstacle as a static obstacle traffic light as an example, it is necessary to store the identification information corresponding to the traffic light and the display state and the number of seconds of the traffic light before disappearing into the preset occlusion array. If the target obstacle is a dynamic obstacle, the identification information corresponding to the dynamic obstacle and the speed value, acceleration value and position information of the dynamic obstacle in several frames before disappearing are stored into the preset occlusion array, which may specifically be the average speed and acceleration of the last three frames, and the position information of the last frame.

[0077] In the subsequent time step, if the target occluder still exists, for static obstacles, its range is released according to its longitude and latitude; for dynamic obstacles, since the average speed, acceleration, and position of the last frame will be read in the last three frames before disappearing, its position can be updated in each frame based on this value. However, since this approach may cause linear updates to appear outside the occlusion range, if the target obstacle is still within the preset occlusion number, the occlusion field of view is used as a bottom value. Then when planning the ego vehicle, an ST diagram will be drawn based on these parameters, that is, the distance s that the target obstacle will travel in the future t time will be drawn on the ST diagram. From the driving trajectory, when there is an occluding obstacle on the right, the ego vehicle will not change lanes to the right. On the speed planning ST diagram, because this method will cause the drawn ST to press down the ego vehicle speed, there will be no situation where the front vehicle accelerates and the front vehicle accelerates.

[0078] Step S15: when a preset release condition is met, releasing the information stored in the preset shielding array.

[0079] In this embodiment, when the preset release conditions are met, the information stored in the preset occlusion array is released, which can avoid the system from storing too much unnecessary information, thereby releasing memory and computing resources, and enabling the system to process new perception data and planning tasks more efficiently.

[0080] It can be seen that in the present application, the obstacle information around the vehicle will be identified in real time during the driving process, and the obstacle attention type determined according to the current driving path and the preset planned path is used to match the corresponding target obstacle from the obstacle information. It can be understood that in different road scenes, the obstacle types that need to be paid attention to are different, and it is not necessary to pay attention to every obstacle around. Therefore, the present application needs to first determine the obstacle attention type according to the current driving path and the preset planned path, and then further match the corresponding target obstacle from the obstacle information to filter out other obstacles that do not need to be paid attention to. After matching the required target obstacle, the state of the target obstacle is continuously detected. If the preset disappearance event of the target obstacle is detected, it is determined whether the target obstacle is blocked based on the corner position information of the front vehicle. If it is determined that the target obstacle is blocked, the target obstacle and the state information of the target obstacle before disappearance are stored in the preset blocking array, so that in the time period when the target obstacle disappears, the vehicle can plan the path based on the information stored in the preset blocking array to avoid the situation where the vehicle makes an incorrect driving plan due to the blocking of the obstacle, thereby improving the reliability and safety of the path planning. That is, in this application, only obstacles that are determined to be blocked will be stored in the preset blocking array so that the vehicle can perform subsequent path planning. In addition, when the preset release conditions are met, the information stored in the preset blocking array is released, which can avoid the system from storing too much unnecessary information, achieve the purpose of freeing up memory and computing resources, and enable the system to process new perception data and planning tasks more efficiently.

[0081] See also Figure 2 As shown, the embodiment of the present application discloses a specific solution to visual occlusion. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically, it includes:

[0082] Step S21: During the driving process of the vehicle, identify surrounding obstacle information.

[0083] Step S22: determining an obstacle attention type according to the current driving path and the preset planned path, so as to match a corresponding target obstacle from the obstacle information based on the obstacle attention type.

[0084] Step S23: Continuously detect the state of the target obstacle. If the target obstacle disappears as preset, establish a three-dimensional coordinate system with the vehicle camera as the origin, and determine the position information of each corner point of the front vehicle in the three-dimensional coordinate system according to the position information and length, width and height information of the front vehicle.

[0085] In this embodiment, if the target obstacle disappears in a preset event, it is necessary to identify the target obstacle for occlusion. First, a three-dimensional coordinate system is established with the camera of the vehicle as the origin, and the position information of each corner point of the front vehicle in the three-dimensional coordinate system is determined according to the position information and length, width and height information of the front vehicle. Specifically, the plane corner points can be calculated first, and then the height information can be added to obtain the eight corner point positions of the front vehicle. Figure 3 shown.

[0086] Step S24: connecting each corner point of the preceding vehicle with the origin based on the position information of each corner point to obtain corresponding rays, so as to determine the blocked space based on the area formed by each ray.

[0087] In this embodiment, in the three-dimensional space, based on the position information of each corner point, rays are drawn from the origin to the eight corner points of the front vehicle, and then the occlusion space is determined based on the area formed by each ray. Figure 3 As shown in the figure, the estimated position of the red light of the obstruction is known, the ray is cut off at the red light position x+i (x is the longitudinal distance of the red light, i is a calibrable parameter), and the 8 cut points are connected in sequence to obtain a polyhedron. The figure takes four lines as an example, and the polyhedron is the obstruction space.

[0088] Step S25: Determine whether the target obstacle is located in the occlusion space based on the state information of the target obstacle before it disappears. If so, determine that the target obstacle is occluded, and store the target obstacle and the state information of the target obstacle before it disappears into a preset occlusion array, so that the vehicle can perform path planning based on the preset occlusion array.

[0089] In this embodiment, whether the target obstacle is located in the occlusion space is determined based on the state information of the target obstacle before it disappears. Taking a static obstacle as an example, since the position information of the target obstacle in the last frame before it disappears can be obtained, the position information can be converted into longitude and latitude coordinates, and then the longitude and latitude coordinates of the static obstacle can be used to determine whether it is located in the occlusion space. The judgment process for dynamic obstacles such as vehicles is similar, but only when all eight corner points of the vehicle are located in the occlusion space is it considered to be occluded.

[0090] In a specific implementation, judging whether the target obstacle is located in the occlusion space according to the state information of the target obstacle before disappearance includes: projecting the occlusion space into the height space and the ground space respectively; wherein the ground space is a two-dimensional space parallel to the vehicle driving plane, and the height space is a two-dimensional space perpendicular to the ground; obtaining the spatial position coordinates of the last frame of the target obstacle before disappearance, and judging whether the height position coordinates in the spatial position coordinates are located in the height space; if they are, judging whether the ground position coordinates in the spatial position coordinates are located in the ground space, and if they are, judging that the target obstacle is located in the occlusion space. That is, the present application needs to project the occlusion space into the height space (yz plane) and the ground space (xy plane) respectively, the ground space is a two-dimensional space parallel to the vehicle driving plane, which roughly corresponds to the actual ground, and it is a space that projects the position relationship of objects in the three-dimensional space onto the horizontal plane for analysis and judgment; the height space is a two-dimensional space perpendicular to the ground, for example, judging whether the static obstacle is above or below the occlusion ray formed by the object in front of the vehicle, so as to determine whether it is within the occlusion range. Furthermore, the spatial position coordinates of the target obstacle in the last frame before it disappears are obtained. First, it is determined whether the height position coordinates in the spatial position coordinates are located in the height space, such as Figure 4 As shown, after the projection is completed, it is calculated whether the red light is in the ray area. When it is determined to be in the height space, it is judged whether the ground position coordinates in the spatial position coordinates are in the ground space. When both are consistent, it is determined that the target obstacle is in the occlusion space, that is, the target obstacle is blocked. By combining the judgment of the height space with the ground space, it is possible to comprehensively determine whether the object is in the effective occlusion area, providing a more accurate basis for the decision-making of the autonomous driving vehicle. Among them, when making judgments in the height space and the ground space, the angle sum method can be specifically used: calculate the sum of the angles of the point relative to all vertices of the polygon. If the sum is 360 degrees, it means that the point is inside the polygon; if it is less than 360 degrees, the point is outside the polygon; if it is equal to 0 degrees or greater than 360 degrees, the point is on the edge of the polygon.

[0091] It should be pointed out that before judging whether the target obstacle is located in the occlusion space according to the state information of the target obstacle before disappearance, the above-mentioned step further includes: if the target obstacle is a static obstacle, judging whether the target obstacle is in a preset stable state according to the state information of several frames before disappearance of the target obstacle; if not, the step of judging whether the target obstacle is located in the occlusion space according to the state information of the target obstacle before disappearance is prohibited. That is, if the target obstacle is a static obstacle, a steady-state judgment needs to be made on the static obstacle before occlusion judgment, that is, judging whether the target obstacle is in a preset stable state according to the state information of several frames before disappearance of the target obstacle. Specifically, the position error between the two positions of the target obstacle in the last three frames before disappearance is required to be no higher than vt+a, where v refers to the speed of the vehicle; a refers to the preset distance error, a; can be calibrated according to the actual scene; t is the time difference between two frames; if this condition is not met, the target obstacle is directly eliminated, that is, the step of judging whether the target obstacle is located in the occlusion space according to the state information of the target obstacle before disappearance is prohibited.

[0092] Step S26: when a preset release condition is met, releasing the information stored in the preset shielding array.

[0093] In a specific embodiment, when a preset release condition is met, releasing the information stored in the preset occlusion array includes: for any static obstacle corresponding to the information stored in the preset occlusion array, if the any static obstacle is re-identified, or, if it is determined that the any static obstacle has exceeded the current detection area according to the state information of the any static obstacle before disappearing, then releasing the information related to the any static obstacle stored in the preset occlusion array; for any dynamic obstacle corresponding to the information stored in the preset occlusion array, predicting the current latest position of the any dynamic obstacle according to the state information of the any dynamic obstacle before disappearing, if it is determined that the ego vehicle has exceeded the any dynamic obstacle based on the current latest position, or, if the ego vehicle has completed the lane change operation, or, if the any dynamic obstacle is re-identified, or, if there are other vehicles passing through the occlusion area, then releasing the information related to the any dynamic obstacle stored in the preset occlusion array.

[0094] It is understandable that the information in the preset occlusion array is not stored permanently. When the preset release condition is met, the information stored in the preset occlusion array will be released to achieve the purpose of releasing memory and computing resources.

[0095] For static obstacles, considering the errors in sensor recognition and the errors in the estimation of the ego vehicle's motion, when a static obstacle such as a traffic light or a traffic sign is re-identified based on its longitude and latitude coordinates and the error with the stored longitude and latitude coordinates is within a radius of 3m, the information related to the static obstacle stored in the preset occlusion array is released. Alternatively, when the static obstacle has exceeded the current detection area of ​​the ego vehicle's camera, that is, the detection area of ​​the ego vehicle's camera no longer covers the static obstacle, it is also released from the preset occlusion array. For dynamic obstacles, when the ego vehicle exceeds the target vehicle, the ego vehicle has completed the lane change operation, the dynamic obstacle is detected again, or other vehicles pass through the occlusion area, the information related to any dynamic obstacle stored in the preset occlusion array is released.

[0096] In this way, the system can avoid storing too much unnecessary information, free up memory and computing resources, enable the system to process new perception data and planning tasks more efficiently, maintain the smooth operation of the system, and improve processing speed and responsiveness.

[0097] For more specific processing procedures of the above steps S21 and S22, reference may be made to the corresponding contents disclosed in the above embodiments, which will not be described in detail here.

[0098] It can be seen that when the present application performs occlusion judgment on the target obstacle, it is necessary to establish a three-dimensional coordinate system with the vehicle camera as the origin, and determine the position information of each corner point of the front vehicle in the three-dimensional coordinate system based on the position information and length, width and height information of the front vehicle, and then based on the position information of each corner point, make rays from the origin to the eight corner points of the front vehicle respectively, so as to determine the occlusion space based on the area formed by each ray, and then perform occlusion judgment based on the state information of the target obstacle before disappearance and the position information between the occlusion space. In addition, the information in the preset occlusion array is not stored permanently. When the preset release condition is met, the information stored in the preset occlusion array will be released to achieve the purpose of releasing memory and computing resources. The present application sets corresponding release conditions for static obstacles and dynamic obstacles respectively, and releases the information stored in the preset occlusion array when the condition is met.

[0099] See also Figure 5 As shown, the embodiment of the present application discloses a device for solving visual occlusion, the device comprising:

[0100] The recognition module 11 is used to recognize surrounding obstacle information during the driving process of the vehicle;

[0101] A matching module 12, configured to determine an obstacle attention type according to a current driving path and a preset planned path, so as to match a corresponding target obstacle from the obstacle information based on the obstacle attention type;

[0102] The occlusion detection module 13 is used to continuously detect the state of the target obstacle, and if the target obstacle has a preset disappearance event, determine whether the target obstacle is occluded based on the corner point position information of the preceding vehicle;

[0103] The storage module 14 is used to store the target obstacle and the state information of the target obstacle before disappearing into a preset occlusion array if it is blocked, so that the vehicle can perform path planning based on the preset occlusion array;

[0104] The release module 15 is used to release the information stored in the preset shielding array when a preset release condition is met.

[0105] It can be seen that in the present application, the obstacle information around the vehicle will be identified in real time during the driving process, and the obstacle attention type determined according to the current driving path and the preset planned path is used to match the corresponding target obstacle from the obstacle information. It can be understood that in different road scenes, the obstacle types that need to be paid attention to are different, and it is not necessary to pay attention to every obstacle around. Therefore, the present application needs to first determine the obstacle attention type according to the current driving path and the preset planned path, and then further match the corresponding target obstacle from the obstacle information to filter out other obstacles that do not need to be paid attention to. After matching the required target obstacle, the state of the target obstacle is continuously detected. If the preset disappearance event of the target obstacle is detected, it is determined whether the target obstacle is blocked based on the corner position information of the front vehicle. If it is determined that the target obstacle is blocked, the target obstacle and the state information of the target obstacle before disappearance are stored in the preset blocking array, so that in the time period when the target obstacle disappears, the vehicle can plan the path based on the information stored in the preset blocking array to avoid the situation where the vehicle makes an incorrect driving plan due to the blocking of the obstacle, thereby improving the reliability and safety of the path planning. That is, in this application, only obstacles that are determined to be blocked will be stored in the preset blocking array so that the vehicle can perform subsequent path planning. In addition, when the preset release conditions are met, the information stored in the preset blocking array is released, which can avoid the system from storing too much unnecessary information, achieve the purpose of freeing up memory and computing resources, and enable the system to process new perception data and planning tasks more efficiently.

[0106] Since the embodiments of the device part correspond to the embodiments of the method part, the embodiments of the device part refer to the description of the embodiments of the method part, and will not be repeated here. In addition, the invention has the same beneficial effects as the above-mentioned solution to visual occlusion.

[0107] Figure 6A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the solution to visual occlusion performed by the electronic device disclosed in any of the aforementioned embodiments.

[0108] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0109] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0110] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 221, a computer program 222 and data 223, etc. The storage method can be temporary storage or permanent storage.

[0111] Among them, the operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, so as to realize the operation and processing of the massive data 223 in the memory 22 by the processor 21, which can be Windows, Unix, Linux, etc. In addition to including computer programs that can be used to complete the visual occlusion solution performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs that can be used to complete other specific tasks. In addition to data transmitted from an external device received by the electronic device, the data 223 can also include data collected by its own input and output interface 25, etc.

[0112] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the steps of the solution to visual occlusion disclosed in any of the aforementioned embodiments are implemented.

[0113] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0114] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0115] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a compact disc read-only memory (CD-ROM), or any other form of storage medium known in the art.

[0116] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0117] The above is a detailed introduction to a visual occlusion solution, device, equipment and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A solution to visual occlusion, characterized in that: include: During the driving process of the vehicle, identify the surrounding obstacle information; Determine an obstacle attention type according to the current driving path and the preset planned path, and match a corresponding target obstacle from the obstacle information based on the obstacle attention type; Continuously detecting the state of the target obstacle, and if a preset disappearance event occurs to the target obstacle, determining whether the target obstacle is blocked based on the corner point position information of the preceding vehicle; If blocked, the target obstacle and the state information of the target obstacle before disappearing are stored in a preset blocking array, so that the vehicle can perform path planning based on the preset blocking array; When a preset release condition is met, the information stored in the preset occlusion array is released.

2. The solution to visual occlusion according to claim 1, characterized in that: The process of identifying surrounding obstacle information also includes: Classify surrounding obstacle information into categories; the categories of obstacles include vehicles, pedestrians, traffic signs and traffic lights; The speed of each surrounding obstacle is compared with a preset speed threshold to determine the type of each surrounding obstacle; wherein the types of obstacles include static obstacles and dynamic obstacles.

3. The solution to visual occlusion according to claim 1, characterized in that: The determining of the obstacle attention type according to the current driving path and the preset planned path, so as to match the corresponding target obstacle from the obstacle information based on the obstacle attention type, includes: Determine the current driving path and obtain the preset planned path from the navigation system; Determining whether the vehicle needs to perform a lane change operation according to the position relationship between the current driving path and the preset planned path; If the ego vehicle does not need to perform a lane change operation, determining whether the ego vehicle is located at an intersection, determining a corresponding first type of obstacle attention type based on the intersection position determination result, and then matching a corresponding target obstacle from the obstacle information based on the first type of obstacle attention type; If the ego vehicle needs to perform a lane change operation, a second obstacle attention type is obtained based on the first obstacle attention type and the obstacle type on the target lane change side, and then a corresponding target obstacle is matched from the obstacle information based on the second obstacle attention type.

4. The visual occlusion solution according to claim 3, characterized in that: The determining of the corresponding first type of obstacle attention type based on the intersection position judgment result includes: If the vehicle is not located at the intersection, the front vehicle on the lane is determined as the first type of obstacle concern; If the vehicle is located at an intersection, the target dynamic obstacles and target static obstacles around the intersection are determined as the first type of obstacle attention type; the target dynamic obstacles include pedestrians and non-motor vehicles on both sides of the intersection, and the target static obstacles include traffic lights and traffic signs.

5. The visual occlusion solution according to claim 1, characterized in that: The determining whether the target obstacle is blocked based on the corner point position information of the preceding vehicle includes: A three-dimensional coordinate system is established with the camera of the vehicle as the origin, and the position information of each corner point of the front vehicle in the three-dimensional coordinate system is determined according to the position information and length, width and height information of the front vehicle; Connecting each corner point of the preceding vehicle with the origin based on the position information of each corner point to obtain a corresponding ray, and determining the shielding space based on the area formed by each ray; It is determined whether the target obstacle is located in the blocking space according to the state information before the target obstacle disappears, and if so, it is determined that the target obstacle is blocked.

6. The visual occlusion solution according to claim 5, characterized in that: The determining whether the target obstacle is located in the blocking space according to the state information of the target obstacle before disappearing includes: The blocked space is projected to the height space and the ground space respectively; wherein the ground space is a two-dimensional space parallel to the vehicle travel plane, and the height space is a two-dimensional space perpendicular to the ground; Obtaining the spatial position coordinates of the last frame before the target obstacle disappears, and determining whether the height position coordinates in the spatial position coordinates are located in the height space; If so, it is determined whether the ground position coordinates in the spatial position coordinates are located in the ground space; if so, it is determined that the target obstacle is located in the blocking space.

7. The solution to visual occlusion according to claim 5, characterized in that: Before judging whether the target obstacle is located in the blocking space according to the state information of the target obstacle before disappearing, the method further includes: If the target obstacle is a static obstacle, judging whether the target obstacle is in a preset stable state according to state information of several frames before the target obstacle disappears; If not, the step of determining whether the target obstacle is located within the blocked space based on the state information of the target obstacle before it disappears is prohibited.

8. The visual occlusion solution according to claim 1, characterized in that: The storing the target obstacle and the state information of the target obstacle before disappearing into a preset occlusion array includes: If the target obstacle is a traffic light, the traffic light and the display state and seconds of the traffic light before disappearing are stored in a preset occlusion array; If the target obstacle is a dynamic obstacle, the dynamic obstacle and the velocity value, acceleration value and position information of several frames before the dynamic obstacle disappears are stored in a preset occlusion array.

9. The visual occlusion solution according to any one of claims 1 to 8, characterized in that: When a preset release condition is met, releasing the information stored in the preset occlusion array includes: For any static obstacle corresponding to the information stored in the preset occlusion array, if the any static obstacle is re-identified, or if it is determined that the any static obstacle has exceeded the current detection area according to the state information of the any static obstacle before disappearance, then the information related to the any static obstacle stored in the preset occlusion array is released; For any dynamic obstacle corresponding to the information stored in the preset occlusion array, the current latest position of any dynamic obstacle is predicted according to the state information of any dynamic obstacle before it disappears; if it is determined based on the current latest position that the ego vehicle has exceeded any dynamic obstacle, or if the ego vehicle has completed a lane change operation, or if any dynamic obstacle is re-identified, or if other vehicles pass through the occlusion area, the information related to the any dynamic obstacle stored in the preset occlusion array is released.

10. A device for solving visual occlusion, characterized in that: include: The recognition module is used to identify surrounding obstacle information while the vehicle is driving; A matching module, used to determine the obstacle attention type according to the current driving path and the preset planned path, so as to match the corresponding target obstacle from the obstacle information based on the obstacle attention type; An occlusion detection module is used to continuously detect the state of the target obstacle, and if the target obstacle disappears in a preset manner, determine whether the target obstacle is occluded based on the corner point position information of the preceding vehicle; A storage module, for storing the target obstacle and the state information of the target obstacle before disappearing into a preset occlusion array if blocked, so that the ego vehicle can perform path planning based on the preset occlusion array; The release module is used to release the information stored in the preset occlusion array when a preset release condition is met.

11. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor is used to execute the computer program to implement the steps of the visual occlusion solution as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: Used to store a computer program; wherein, when the computer program is executed by a processor, the steps of the solution to visual occlusion as described in any one of claims 1 to 9 are implemented.

Images