A shielding screening method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a kind of screening method, device, electronic equipment and storage medium of shielding, applied in vehicle simulation scene;Method includes: obtaining the position information of each to-be-screened object in the visual range of vehicle-mounted sensor;According to the position information, determine target object in order from near to far, respectively execute shielding screening processing for each target object, determine unshielded object;Shielding screening processing includes: if target object is the first order in sequence, determine target object as unshielded object;If target object is other order in sequence, traverse all unshielded objects, according to the shielding state between target object and unshielded object, determine target object as shielded object or unshielded object.The above process is based on the data of to-be-screened object itself to determine the shielding relationship, can directly output the data of object, without generating object data based on rendering effect, the efficiency of data output is high, meet the demand of vehicle simulation scene.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method, apparatus, electronic device, and storage medium for occlusion screening. Background Technology

[0002] Occlusion culling (OCC) refers to the process of not rendering an object when it is occluded by other objects and is not within the visible range. Common occlusion culling techniques include the occlusion modules built into rendering tools such as UE4 and Unity; painter algorithms, BSP trees, and Z-buffers suitable for image rendering; and raycasting. However, the above-mentioned occlusion culling techniques are based on rendering effects and are often used for rendering game scenes, but they cannot meet the needs of vehicle simulation scenes. Summary of the Invention

[0003] The purpose of this application is to provide an occlusion screening method, device, electronic device, and storage medium to achieve the technical effect required to meet the needs of vehicle simulation scenarios.

[0004] The first aspect of this application provides an occlusion filtering method applied to a vehicle simulation scenario; the method includes:

[0005] Acquire the position information of each object to be screened within the field of view of the vehicle-mounted sensor;

[0006] Based on the location information, the target object is determined from all the objects to be filtered in order from near to far. Occlusion filtering is performed on each target object to determine the unoccluded object from all the objects to be filtered.

[0007] The occlusion filtering process includes:

[0008] If the target object is the first in the sequence, the target object is determined to be an unobstructed object;

[0009] If the target object is in any other position in the sequence, traverse all unoccluded objects and determine whether the target object is an occluded object or an unoccluded object based on the occlusion state between the target object and the unoccluded objects.

[0010] The above implementation process is based on data filtering, that is, determining the occlusion relationships between objects based on the data of the objects to be filtered themselves. Compared with occlusion filtering techniques in related technologies, the above method can directly output object data, such as the set of unoccluded objects and the occlusion relationships between each object. It eliminates the need to generate object data based on the rendering effect, resulting in higher data output efficiency. Furthermore, since the objects to be filtered are generated from simulation data, the above method achieves the technical effect of judging the occlusion situation between objects based on simulation data, thus meeting the needs of vehicle simulation scenarios.

[0011] Furthermore, acquiring the position information of each object to be screened within the field of view of the vehicle-mounted sensor includes:

[0012] Based on the sensing data measured by the sensor, all objects to be screened within the field of view are detected;

[0013] For each of the objects to be filtered, generate the bounding box of the object to be filtered;

[0014] The position information of the reference point in the bounding box is determined as the position information of the object to be filtered.

[0015] In the above implementation process, by using the position information of reference points in the bounding box as the position information of the objects to be filtered, the process of determining the position information of the objects to be filtered is simplified and the data output efficiency is improved.

[0016] Further, the occlusion state includes occlusion ratio and occlusion position; determining whether the target object is an occluded object or an unoccluded object based on the occlusion state between the target object and the unoccluded object includes:

[0017] If the occlusion ratio is not greater than the first threshold, the target object is determined to be an unoccluded object;

[0018] If the occlusion is unilateral, and the occlusion ratio is less than a second threshold, the target object is determined to be an unoccluded object; if the occlusion ratio is not less than the second threshold, the target object is determined to be an occluded object.

[0019] If the occlusion position is full occlusion or central occlusion, the target object is determined to be an occluded object;

[0020] Wherein, the second threshold is greater than the first threshold.

[0021] In the above implementation process, the occlusion state between the target object and the unoccluded object is determined based on the occlusion ratio and / or occlusion position, accurately identifying whether each object to be screened is occluded. Furthermore, this process is data-driven occlusion screening, eliminating the need to generate object data based on the rendering results, resulting in high data output efficiency and meeting the requirements of vehicle simulation scenarios.

[0022] Furthermore, the occlusion ratio is calculated through the following steps:

[0023] The contour curvature information of the target object and the unoccluded object are obtained respectively; the contour curvature information includes the curvature of the multiple edges of the contour relative to the direction of the sensor;

[0024] The occlusion ratio between the target object and the unoccluded object is determined based on the difference in curvature of the same-side edges of the target object and the contour curvature information of the target object.

[0025] In the above implementation process, the occlusion ratio is calculated based on the curvature of the multiple edges of the target object and the unoccluded object relative to the sensor direction, providing a new method for calculating the occlusion ratio. Moreover, it only requires obtaining the curvature information of the edges, and the calculation process is simple, fast, and efficient.

[0026] Furthermore, the contour curvature information is determined based on the vertex position information of the bounding box of the object; the vertex position information is polar coordinate position information, including the direction information and distance information of the vertex of the bounding box relative to the sensor.

[0027] In the above implementation process, by using the edges in the bounding box as the edges of the objects to be filtered, the process of determining the object edges and the contour curvature information is simplified, thereby improving the data output efficiency.

[0028] Furthermore, the method also includes:

[0029] If the difference in curvature of the multiple edges of the target object is greater than the third threshold, the curvature of one edge of the multiple edges is updated according to the difference between the circumferential angle and the difference in curvature, so that the difference in curvature of the multiple edges after the update is less than the third threshold.

[0030] The occlusion ratio is determined using the updated radians.

[0031] In the above implementation process, the curvature of the edge of the target object located on both sides of the polar axis is updated relative to the sensor direction, so that the target object located on both sides of the polar axis can also correctly calculate its occlusion state.

[0032] Furthermore, the occlusion filtering process also includes:

[0033] If the target object is in the first priority position, the arc occupied by the target object within the field of view is determined as the occlusion range;

[0034] If the target object is in any of the other order, before traversing all unoccluded objects, if it is determined that the target object is within the occlusion range, the target object is determined to be an occluded object.

[0035] In the above implementation process, by recording the occlusion range, if it is determined that the target object is within the occlusion range, it can be directly identified as an occluded object without having to traverse all unoccluded objects, thereby saving computational resources.

[0036] Furthermore, the occlusion filtering process also includes:

[0037] If the occlusion state between the target object and the unoccluded object is partial occlusion, the occlusion range is updated according to the curvature of at least one edge of the target object's contour relative to the sensor direction.

[0038] In the above implementation process, the occlusion range is updated according to the occlusion state between the target object and the unoccluded object, which simplifies the subsequent occlusion screening process for the objects to be screened and saves computing resources.

[0039] Furthermore, the method also includes:

[0040] If all the objects to be filtered have completed the occlusion filtering process, and / or if the occlusion range is consistent with the field of view, output all the unoccluded objects.

[0041] In the above implementation, when all objects to be filtered have completed the occlusion filtering process, and / or if the occlusion range is consistent with the field of view, the occlusion filtering loop is exited, and all unoccluded objects are output. By setting a loop exit condition, computational resources can be saved.

[0042] Furthermore, the method also includes:

[0043] A vehicle simulation scene model is established based on the object information of all unoccluded objects; the object information includes one or more of the following: type information, shape information, and occlusion information.

[0044] In the above implementation process, a vehicle simulation scene model is built using the object information of all unoccluded objects to render the vehicle simulation scene. This achieves a process of first filtering data and then rendering, thus meeting the requirements of the vehicle simulation scene.

[0045] A second aspect of this application provides an occlusion screening device applied in a vehicle simulation scenario; the device includes:

[0046] The acquisition module is used to acquire the position information of each object to be screened within the field of view of the vehicle-mounted sensor;

[0047] The occlusion filtering module is used to determine the target object from all the objects to be filtered according to the location information in order from near to far, perform occlusion filtering processing for each target object, and determine the unoccluded object from all the objects to be filtered.

[0048] The occlusion filtering module includes:

[0049] The first priority submodule is used to determine that the target object is an unoccluded object if the target object is the first priority in the order.

[0050] Other order submodules are used to traverse all unoccluded objects if the target object is in another order, and determine whether the target object is an occluded object or an unoccluded object based on the occlusion state between the target object and the unoccluded objects.

[0051] A third aspect of this application provides an electronic device, the electronic device comprising:

[0052] processor;

[0053] Memory used to store processor-executable instructions;

[0054] Wherein, when the processor invokes the executable instructions, it implements the operation of any of the methods described in the first aspect.

[0055] A fourth aspect of this application provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of any of the methods described in the first aspect. Attached Figure Description

[0056] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart illustrating an occlusion filtering method provided in an embodiment of this application;

[0058] Figure 2 A flowchart illustrating another occlusion filtering method provided in an embodiment of this application;

[0059] Figure 3 A flowchart illustrating another occlusion filtering method provided in an embodiment of this application;

[0060] Figure 4 A flowchart illustrating another occlusion filtering method provided in an embodiment of this application;

[0061] Figure 5 A flowchart illustrating another occlusion filtering method provided in an embodiment of this application;

[0062] Figure 6 This is a schematic diagram illustrating the calculation of the occlusion ratio provided in an embodiment of this application;

[0063] Figure 7 This is a schematic diagram illustrating the calculation of the occlusion ratio in another embodiment of this application;

[0064] Figure 8 This is a schematic diagram illustrating the calculation of the occlusion range provided in an embodiment of this application;

[0065] Figure 9 A flowchart illustrating another occlusion filtering method provided in an embodiment of this application;

[0066] Figure 10 A structural block diagram of a shading screening device provided in an embodiment of this application;

[0067] Figure 11 This is a hardware schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0068] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0069] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0070] Occlusion culling (OCC) refers to the process of not rendering an object when it is occluded by other objects and is not within the visible range. Common occlusion culling techniques include the occlusion modules built into rendering tools such as UE4 and Unity; painter algorithms, BSP trees, and Z-buffers suitable for image rendering; and raycasting. These occlusion culling techniques are frequently used in game scene rendering.

[0071] Vehicle simulation platforms are commonly used for testing autonomous driving technologies. To ensure the safety and reliability of newly developed autonomous driving technologies, conducting autonomous driving tests on vehicle simulation platforms has become an indispensable process. During testing, it's not only necessary to render the vehicle simulation scene, but also to acquire data on various objects within the scene, such as occlusion data, for subsequent testing and verification. However, the aforementioned occlusion culling techniques are based on filtering rendered data, generating object data on top of the rendered results. They cannot directly output object data, resulting in low efficiency and failing to meet the needs of vehicle simulation scenes.

[0072] Therefore, this application proposes an occlusion filtering method for use in vehicle simulation scenarios, including, for example... Figure 1 The steps shown are as follows:

[0073] Step S1: Obtain the position information of each object to be screened within the field of view of the vehicle-mounted sensor;

[0074] Step S2: Based on the location information, determine the target object from all the objects to be filtered in order from near to far, perform occlusion filtering processing on each target object, and determine the unoccluded object from all the objects to be filtered.

[0075] In vehicle simulation scenarios, vehicles are equipped with sensors. These onboard sensors can include, but are not limited to, vision sensors, LiDAR, millimeter-wave radar, or ultrasonic sensors. Based on the different sensors, the sensor data collected by the vehicle also varies. For example, vision sensors collect image data, LiDAR sensors collect point cloud data, millimeter-wave radar sensors collect millimeter-wave data, and ultrasonic sensors collect ultrasonic data.

[0076] The visual sensors mounted on a vehicle can include, but are not limited to, monocular vision systems and / or binocular vision systems. Monocular vision systems typically use a single camera to capture multiple images from different locations, determining the object's depth information by observing the changes in the same object across these images. Binocular vision systems, on the other hand, use two cameras to form a binocular system. Based on the principle of parallax, they acquire two images of the object from different locations using an imaging device. By calculating the positional deviation between corresponding points in the images, they obtain the object's three-dimensional geometric information; that is, two cameras forming a binocular system can be used to perceive depth information of objects in a specific direction.

[0077] Using sensor data, all objects to be screened within the sensor's field of view can be detected. In the vehicle simulation scenario, the objects to be screened are generated based on simulation data. The sensor data is also measured based on this simulation data.

[0078] For example, deep learning techniques can be used to identify objects to be screened from sensor data. For instance, a pre-trained object recognition model can be used, with the measured sensor data input into the object recognition model, and the identified objects to be screened output.

[0079] For example, if the sensing data is image data, image recognition technology can be used to identify all objects to be screened from the image.

[0080] For example, if the sensing data is one of point cloud data, millimeter wave data, or ultrasonic data, an image can be generated based on the sensing data first, and then image recognition technology can be used to identify all the objects to be screened from the image.

[0081] After identifying the object to be filtered, its position information can be determined based on its pixel position in the image and / or depth information in the sensor data. Optionally, this position information can be three-dimensional position information. For example, it could be three-dimensional position information in a world coordinate system. Optionally, this position information can be polar coordinate position information. For example, it could be polar coordinate information with the sensor as the origin, including direction information relative to the origin and distance information. The direction information in the polar coordinates of a point can be represented by angles (°) or radians, representing the angle formed by the line connecting that point to the origin and the polar axis.

[0082] Based on the location information of the objects to be filtered, the distance of each object can be determined. All objects are then sorted from closest to furthest to obtain the order in which they are to be filtered. The distance refers to how close the object is to the sensor.

[0083] Subsequently, target objects can be identified sequentially from nearest to farthest, starting with the first one in the aforementioned order. Occlusion filtering is performed on each target object. After occlusion filtering, it can be determined whether the target object is occluded or unoccluded. An occluded object is one that is occluded by other objects close to the sensor and will not be displayed in the vehicle simulation scene. An unoccluded object is one that is not occluded by other objects and will be displayed in the vehicle simulation scene. After completing the occlusion filtering for the current target object, the next object to be filtered is identified as the target object in the same order, and the occlusion filtering process is returned for execution. The occlusion filtering process is executed cyclically according to the above process until the loop exit condition is met.

[0084] Among them, the occlusion filtering process includes, for example Figure 2 The steps shown are as follows:

[0085] Step S21: If the target object is the first in the sequence, determine that the target object is an unobstructed object;

[0086] Step S22: If the target object is in any other position in the sequence, traverse all unoccluded objects and determine whether the target object is an occluded object or an unoccluded object based on the occlusion state between the target object and the unoccluded objects.

[0087] If the target object is the first in the sequence, it means that the target object is closest to the sensor (origin). Since all other objects to be screened are farther from the sensor than the target object, the target object must not be occluded. Thus, the first-ranked target object can be directly determined as an unoccluded object.

[0088] If the target object is in a different order, then objects in the preceding order are closer to the sensor than the target object, and may therefore occlude it. In this case, it is necessary to iterate through the previously identified unoccluded objects and analyze the occlusion state between the target object and each unoccluded object. By comprehensively considering the occlusion states between the target object and each unoccluded object, it is determined whether the target object is occluded or not.

[0089] It's easy to understand that the previously identified unoccluded objects must be the objects preceding the current target object in the sequence. In other words, the objects preceding the current target object include unoccluded objects, or both unoccluded and occluded objects.

[0090] In the above process, only the pre-determined unoccluded objects are traversed, not all objects in the preceding order. This is because, for the target object, regardless of whether an occluded object occludes it, the target object may be occluded by an unoccluded object. Therefore, to save computational resources, only all unoccluded objects can be traversed.

[0091] The following is a specific example to illustrate this:

[0092] If the objects to be filtered include vehicles A, B, and C, and are arranged in order of distance from the vehicle's sensor from closest to furthest, then the order is: A—B—C. Based on the occlusion filtering process described above, vehicle A is closest to the sensor and is therefore determined to be an unoccluded object. Subsequently, based on the occlusion state between vehicle B and vehicle A (the unoccluded object), vehicle B is determined to be an occluded object. Therefore, when vehicle C is selected as the target object, only the occlusion state between vehicle C and the unoccluded vehicle A is calculated; the occlusion state between vehicle C and the occluded vehicle B is not calculated.

[0093] This application provides an occlusion filtering method. First, based on the position information of each object to be filtered, all objects are sorted from near to far. Then, in sequence, it is determined whether each object to be filtered is occluded by a preceding unoccluded object. This process is data-based filtering, that is, determining the occlusion relationship between objects based on the data of the objects themselves. Compared with occlusion filtering techniques in related technologies, the above method can directly output object data, such as the set of unoccluded objects and the occlusion relationship between objects. There is no need to generate object data based on the rendering effect, resulting in higher data output efficiency. Furthermore, since the objects to be filtered are generated from simulation data, the above method achieves the technical effect of judging the occlusion situation between objects based on simulation data, thus meeting the needs of vehicle simulation scenarios.

[0094] In some embodiments, step S1 includes as follows Figure 3 The steps shown are as follows:

[0095] Step S11: Based on the sensing data measured by the sensor, detect all objects to be screened within the field of view;

[0096] Step S12: For each of the objects to be filtered, generate a bounding box for the object to be filtered;

[0097] Step S13: Determine the position information of the reference point in the bounding box as the position information of the object to be filtered.

[0098] The specific implementation of step S11 can be found above.

[0099] After detecting all objects to be filtered, a corresponding bounding box can be generated for each object. The method for generating bounding boxes can be found in related technologies, and will not be elaborated upon here.

[0100] Subsequently, the position information of the reference point within the bounding box is determined as the position information of the object to be filtered. The reference point of the bounding box can be a feature point within the bounding box. Optionally, the reference point can be the center point of the bounding box. Optionally, if the bounding box is a polyhedron, the reference point can be any vertex of the polyhedron. It can be seen that the position information of the reference point within the bounding box is used to determine the distance of the object to be filtered.

[0101] In this embodiment, by using the position information of reference points in the bounding box as the position information of the object to be filtered, the process of determining the position information of the object to be filtered is simplified and the data output efficiency is improved.

[0102] Regarding occlusion filtering, in some embodiments, the occlusion state between the target object and the unoccluded object includes the occlusion ratio and the occlusion position. Thus, the target object can be determined as an occluded object or an unoccluded object based on the occlusion ratio and / or occlusion position between the target object and the unoccluded object. The occlusion position refers to the location where the target object is occluded, including unilateral occlusion, central occlusion, and complete occlusion. Unilateral occlusion includes one or more of left-side occlusion, right-side occlusion, top-side occlusion, and bottom-side occlusion.

[0103] For example, in step S22 above, determining whether the target object is an occluded object or an unoccluded object based on the occlusion state between the target object and the unoccluded object includes, for example: Figure 4 The steps shown are as follows:

[0104] Step S221: If the occlusion ratio is less than the first threshold, determine that the target object is an unoccluded object;

[0105] Step S222: If the occlusion position is unilateral occlusion, and the occlusion ratio is less than the second threshold, the target object is determined to be an unoccluded object; if the occlusion ratio is greater than the second threshold, the target object is determined to be an occluded object.

[0106] Step S223: If the occlusion position is full occlusion or central occlusion, determine that the target object is an occluded object;

[0107] Wherein, the second threshold is greater than the first threshold.

[0108] First, the target object can be determined to be unoccluded based on the occlusion ratio between the target object and the unoccluded object. If the occlusion ratio is not greater than a first threshold, the target object is determined to be an occluded object. For example, the first threshold can be 0%. If the occlusion ratio is not greater than 0%, it means that the target object is not occluded by any other object. Of course, those skilled in the art can also determine the value of the first threshold according to actual needs, such as 1% or a value not less than 0.

[0109] Second, the occluded object can be determined directly based on the occlusion position between the target object and the unoccluded object. If the occlusion is complete or central occlusion, the target object is determined to be occluded. Complete occlusion refers to 100% occlusion. Central occlusion means the central area of ​​the target object is occluded, while the edges are not.

[0110] Third, the occlusion ratio and occlusion location can be combined to determine whether the target object is occluded or unoccluded. In the case of unilateral occlusion, if the occlusion ratio is less than a second threshold, the target object is determined to be unoccluded; if the occlusion ratio is not less than the second threshold, the target object is determined to be occluded. The second threshold is greater than the first threshold. For example, the second threshold can be 90%. That is, the target object is considered occluded only when more than 90% of one side is occluded. If the occlusion ratio is less than 90%, the target object is considered unoccluded.

[0111] As can be seen from the above logic, an unobstructed object does not mean that it is completely unobstructed, but includes both cases where it is completely unobstructed and cases where it is only partially obstructed.

[0112] Regarding the calculation method for the occlusion ratio, in some embodiments, the occlusion ratio is calculated according to, for example... Figure 5 Perform the calculations as shown:

[0113] Step S2201: Obtain the contour curvature information of the target object and the unoccluded object respectively;

[0114] The contour curvature information includes the curvature of the multiple edges of the contour relative to the direction of the sensor;

[0115] Step S2203: Determine the occlusion ratio between the target object and the unoccluded object based on the difference in curvature of the same-side edges of the target object and the contour curvature information of the target object.

[0116] The object's outline includes multiple edges, such as, but not limited to, left edge, right edge, top edge, and bottom edge. The radian of the edge relative to the sensor direction, that is, in polar coordinates, the radian of the angle formed by the line connecting the edge to the sensor (origin) and the polar axis.

[0117] The same-side edge of the target object and the unobstructed object, such as one or more of the left edge, right edge, top edge, and bottom edge of the target object and the unobstructed object.

[0118] Thus, based on the difference in curvature of the edges on the same side and the curvature information of the target object's outline, the occlusion ratio between the target object and the unoccluded object can be determined.

[0119] In some embodiments, the outline of the object can be the actual outline of the object, and the multi-sided edges can be the actual edges of the object.

[0120] In other embodiments, to simplify the process of determining object edges and contour curvature information, the object's contour can be an approximate contour, such as the contour of the object's bounding box. Then, the side edges are the edges of the bounding box, which are approximate edges of the object. Thus, the contour curvature information can be determined based on the vertex position information of the object's bounding box. This vertex position information is polar coordinate position information, including the direction and distance information of the bounding box's vertices relative to the sensor (origin).

[0121] The following is Figure 6 The calculation process for the occlusion ratio will be explained in detail as an example:

[0122] Figure 6 The calculation process for the occlusion ratio between the target object 601 and the unoccluded object 602 is shown. Figure 6 As shown in the left figure, the bounding box of an object can be a cuboid. For the target object 601, the multiple edges of its contour can be any edges of the cuboid. Taking the left edge AD and the right edge BC as examples, the radian of the left edge AD relative to the sensor direction can be determined based on the polar coordinates (ρ1, θ1) of vertex A (or vertex D). The radian of the right edge BC relative to the sensor direction can be determined based on the polar coordinates (ρ2, θ2) of vertex B (or vertex C). Here, ρ represents distance information, and θ represents direction information, expressed in angles or radians.

[0123] like Figure 6 As shown in the right figure, the radian of the left edge AD relative to the sensor direction, which is the angle between the line connecting vertex A and origin O (i.e., line segment OA) and the polar axis, is θ1. The radian of the right edge BC relative to the sensor direction, which is the angle between the line connecting vertex B and origin O (i.e., line segment OB) and the polar axis, is θ2.

[0124] Thus, the contour curvature information of the target object 601 includes at least θ1 and θ2. Similarly, the contour curvature information of the unoccluded object 602 includes at least θ3 and θ4.

[0125] Further, θ1 and θ3 represent the curvature of the left edges of the target object 601 and the unobstructed object 602. θ2 and θ4 represent the curvature of the right edges of the target object 601 and the unobstructed object 602. Therefore, the difference in curvature between the same-side edges of the target object and the unobstructed object can include the difference in curvature between the left and right edges of the target object and the unobstructed object. That is, it includes the difference between θ1 and θ3, and the difference between θ2 and θ4. Specifically, the difference in curvature of the left edges can be the curvature of the left edge of the target object relative to the sensor direction minus the curvature of the left edge of the unobstructed object relative to the sensor direction, i.e., θ1 minus θ3. The difference in curvature of the right edges can be the curvature of the right edge of the unobstructed object relative to the sensor direction minus the curvature of the right edge of the target object relative to the sensor direction, i.e., θ4 minus θ2.

[0126] Where the difference in radians is less than 0, then the difference in radians is taken as 0. In the example above, since polar coordinates take the counter-clockwise direction as positive, it can be known that θ1 minus θ3 is greater than zero, while θ4 minus θ2 is less than zero. Therefore, in... Figure 6 In the example, the difference in curvature of the right edge is 0.

[0127] Subsequently, based on the difference in curvature between the edges of the target object and the unoccluded object on the same side, as well as the contour curvature information of the target object, the occlusion ratio between the target object and the unoccluded object can be determined.

[0128] For example, the ratio of the sum of the differences in curvature of all edges on the same side to the differences in curvature of the two edges of the target object can be determined as the unoccluded ratio. Based on the unoccluded ratio, the occlusion ratio can be calculated.

[0129] In the example above, the sum of the differences in curvature of all edges on the same side includes: the difference in curvature of the left edge, i.e., θ1 minus θ3, and the difference in curvature of the right edge, i.e., 0. The difference in curvature between the two edges of the target object is θ1 minus θ2, which is the curvature of ∠AOB. Therefore, the formula for calculating the occlusion scale (OS) of the target object 601 is as follows:

[0130]

[0131] Where max is the function for finding the maximum value; |·| is the function for finding the absolute value.

[0132] Furthermore, in some embodiments, the calculation of the occlusion ratio also includes the step of:

[0133] S2202: If the difference in curvature of the multiple edges of the target object is greater than the third threshold, update the curvature of one edge of the multiple edges according to the difference between the circumference angle and the curvature difference, so that the difference in curvature of the multiple edges after the update is less than the third threshold.

[0134] In step S2203, the occlusion ratio is determined using the updated radian.

[0135] For example, the third threshold could be 180° or π. The difference in curvature between the multiple edges is greater than 180°, meaning the included angle formed by the multiple edges is an obtuse angle. For example... Figure 7 As shown, the target object 701 is located on both sides of the polar axis. Since the polar coordinates take the counterclockwise direction as the positive direction, the difference in radians of the multiple edges of the target object 701, that is, the difference between θ5 and θ6, is actually the radian of the obtuse angle ∠EOF, not the radian of the acute angle ∠EOF.

[0136] Therefore, when the difference in curvature between the multiple edges exceeds the third threshold, the curvature of one edge needs to be updated based on the difference between the circumferential angle (360° or 2π) and this difference in curvature. This ensures that the updated difference in curvature between the multiple edges is less than the third threshold.

[0137] Taking θ6 = 10° and θ5 = 350° as an example, the difference between θ5 and θ6 is 340°. Therefore, the difference between the inscribed angle 360° and this radian difference is 20°. Thus, θ5 can be updated to -10°, or θ6 can be updated to 370°. It can be seen that after the update, the difference between θ5 and θ6 is 20°, which is less than the third threshold.

[0138] Subsequently, the updated radians can be used to calculate the occlusion ratio.

[0139] Optionally, the polar axis points to the rear of the vehicle. Therefore, when the difference in curvature of the edges of the target object is greater than the third threshold, it indicates that the target object is located behind the vehicle.

[0140] In this embodiment, the curvature of the edge of the target object located on both sides of the polar axis is updated relative to the sensor direction, so that the target object located on both sides of the polar axis can also correctly calculate its occlusion state.

[0141] Regarding the occlusion filtering process, in some embodiments, step S21 above further includes:

[0142] If the target object is the first priority, the arc ratio occupied by the target object within the field of view is determined as the occlusion range;

[0143] The above step S22 also includes:

[0144] If the target object is in any of the other order, before traversing all unoccluded objects, if it is determined that the target object is within the occlusion range, the target object is determined to be an occluded object.

[0145] The proportion of the arc occupied by the target object within the field of view is the difference in arc of the multiple edges of the target object relative to the sensor direction.

[0146] like Figure 6 As shown, if the unoccluded object 602 is the first priority, then when performing occlusion filtering processing on it, the radian of ∠A'OB' is determined as the occlusion range.

[0147] The occlusion range refers to the area within which an object is completely occluded. Therefore, when performing occlusion filtering on other target objects, we can first determine if the target object is within the occlusion range. If it is, the target object can be directly identified as an occluded object. If not, we then iterate through all unoccluded objects and determine whether the target object is occluded or not based on the occlusion state between it and the unoccluded objects. This saves computational resources.

[0148] Furthermore, in some embodiments, the occlusion range can be updated based on the occlusion status during the occlusion filtering process. Thus, the occlusion filtering process further includes:

[0149] Step S23: If the occlusion state between the target object and the unoccluded object is partial occlusion, update the occlusion range according to the curvature of at least one edge of the target object's contour relative to the sensor direction.

[0150] Partial occlusion includes central occlusion and unilateral occlusion.

[0151] If the occlusion state between the target object and the unoccluded object is central occlusion, the occlusion range is updated based on the curvature of the multiple edges of the target object's contour relative to the sensor direction.

[0152] As in Figure 8 In the scenario, the occlusion state between the target object 801 and the unoccluded object 802 is central occlusion. If the current occlusion range is the radius of ∠GOH, then the occlusion range can be updated to ∠IOJ based on the radius of the multiple edges of the target object 801's contour relative to the sensor direction, i.e., the angle between line segment IO and the polar axis (not shown in the figure) and the angle between line segment JO and the polar axis.

[0153] If the occlusion between the target object and the unoccluded object is unilateral occlusion, the occlusion range is updated based on the curvature of the unoccluded edge of the target object relative to the sensor direction.

[0154] As in Figure 6In the scenario, the occlusion state between the target object 601 and the unoccluded object 602 is unilateral occlusion. If the current occlusion range is the radius of ∠A'OB', then the occlusion range can be updated to ∠AOB' based on the radius of the left edge (unoccluded) of the target object 601 relative to the sensor direction, i.e., θ1.

[0155] Thus, in some embodiments, the occlusion screening method proposed in this application further includes the step of:

[0156] S3: If all the objects to be filtered have completed the occlusion filtering process, and / or if the occlusion range is consistent with the field of view, output all the unoccluded objects.

[0157] The loop exit conditions for occlusion filtering include two points: first, all objects to be filtered have completed the occlusion filtering process; second, the occlusion range is consistent with the sensor's field of view. The loop of occlusion filtering can exit when either of these conditions is met.

[0158] The occlusion range is defined as the same as the sensor's field of view, meaning they overlap. This implies that even if there are still objects to be filtered that haven't undergone occlusion filtering, these remaining objects will inevitably fall within the occlusion range and be completely occluded. Therefore, there's no need to continue the occlusion filtering process, saving computational resources.

[0159] After exiting the occlusion filtering loop, all unoccluded objects can be output.

[0160] For example, an output list can be set up so that when performing occlusion filtering for each target object, if the target object is determined to be an unoccluded object, its identifier is recorded in the output list. All objects recorded in the list are output objects.

[0161] For example, when sorting objects to be filtered from nearest to farthest, an output list can be created according to the order. When performing occlusion filtering for each target object, if the target object is determined to be an occluded object, then the target object is removed from the output list.

[0162] In some embodiments, the occlusion filtering method proposed in this application further includes the following steps:

[0163] S4: Based on the object information of all unobstructed objects, establish a vehicle simulation scene model.

[0164] The object information includes one or more of the following: type information, shape information, and occlusion information.

[0165] For example, all unobstructed objects are recorded in the output list above.

[0166] In this embodiment, a vehicle simulation scene model is established based on the object information of unobstructed objects, realizing the generation of a simulation model based on object data and scene rendering, which is suitable for vehicle simulation scenarios.

[0167] This application provides an occlusion filtering method. First, based on the position information of each object to be filtered, all objects are sorted from near to far. Then, in sequence, it is determined whether each object is occluded by a preceding unoccluded object. This process is data-based filtering, that is, determining the occlusion relationship between objects based on the data of the objects themselves. Compared with occlusion filtering techniques in related technologies, the above method can directly output object data, such as the set of unoccluded objects and the occlusion relationships between objects. It eliminates the need to generate object data based on the rendering effect, resulting in higher data output efficiency. Furthermore, since the objects to be filtered are generated from simulation data, the above method achieves the technical effect of determining the occlusion situation between objects based on simulation data, thus meeting the needs of vehicle simulation scenarios.

[0168] Furthermore, this application also provides an occlusion filtering method applied to vehicle simulation scenarios, including, for example... Figure 9 The steps shown are as follows:

[0169] Step 910: For each object to be filtered within the sensor's field of view, obtain the position information of the reference point and vertex position information in the bounding box of the object to be filtered;

[0170] Step 920: Obtain the contour curvature information of the object to be filtered based on the vertex position information;

[0171] The contour curvature information includes the curvature of the multiple edges of the bounding box contour relative to the direction of the sensor.

[0172] Step 930: Based on the location information of the reference point, sort all the objects to be filtered from near to far to obtain a sequential list;

[0173] Step 940: Identify the target objects in order from near to far;

[0174] Step 950: Perform occlusion filtering on the target object to determine whether the target object is an occluded object or an unoccluded object;

[0175] Step 960: Is the exit condition met?

[0176] The exit conditions include all objects to be screened having completed the occlusion screening process, and / or the occlusion range being consistent with the field of view.

[0177] If yes, proceed to step 970; otherwise, return to step 940.

[0178] Step 970: Output the output list.

[0179] The occlusion filtering process in step 950 includes:

[0180] Step 951: Obtain the contour curvature information of the target object;

[0181] Step 952: Determine whether the target object is the first item in the sequential list and whether the target object is located on both sides of the polar axis;

[0182] If the polar axis points to the rear of the vehicle, then the target object is not located on either side of the polar axis, meaning the target object is not behind the vehicle.

[0183] If yes, proceed to steps 958a and 958b; otherwise, proceed to step 953.

[0184] Step 953: Is the target object located on both sides of the polar axis?

[0185] If yes, proceed to step 954; otherwise, proceed to step 955.

[0186] Step 954: Update the outline curvature information of the target object;

[0187] Specifically, the curvature of one edge among the multiple edges can be updated based on the difference between the circumferential angle and the curvature difference of the multiple edges of the target object.

[0188] After executing step 954, execute:

[0189] Step 955: Traverse the output list and calculate the occlusion ratio and occlusion position between the target object and the unoccluded object;

[0190] If the occlusion is complete, proceed to step 957a; if the occlusion is central, proceed to steps 957a and 957b; if the occlusion is unilateral, proceed to step 956; if there is no occlusion, proceed to step 958b.

[0191] Step 956: Determine if the occlusion ratio is greater than 90%;

[0192] If yes, proceed to steps 957a and 957b; otherwise, proceed to step 958b.

[0193] Step 957a: Determine that the target object is an occluded object.

[0194] Step 957b: Update the occlusion area;

[0195] Specifically, the occlusion range can be updated based on the curvature of at least one edge of the target object's contour relative to the sensor direction.

[0196] Step 958a: Record the occlusion range based on the contour curvature information of the target object.

[0197] Step 958b: Determine that the target object is an unoccluded object and insert it into the output list.

[0198] For details on the implementation process of each of the above steps, please refer to the corresponding steps in the above method; they will not be repeated here.

[0199] Based on the occlusion filtering method provided in any of the above embodiments, this application also provides an occlusion filtering device applied in a vehicle simulation scenario. Figure 10 As shown, the occlusion screening device 1000 includes:

[0200] The acquisition module 1010 is used to acquire the position information of each object to be screened within the field of view of the vehicle-mounted sensor;

[0201] The occlusion filtering module 1020 is used to determine the target object from all the objects to be filtered according to the location information in the order from near to far, perform occlusion filtering processing for each target object, and determine the unoccluded object from all the objects to be filtered.

[0202] The occlusion filtering module 1020 includes:

[0203] The first priority submodule 1021 is used to determine that the target object is an unobstructed object if the target object is the first priority in the order.

[0204] Other order submodule 1022 is used to traverse all unoccluded objects if the target object is in another order in the sequence, and determine whether the target object is an occluded object or an unoccluded object according to the occlusion state between the target object and the unoccluded objects.

[0205] In some embodiments, the acquisition module 1010 includes:

[0206] The detection submodule is used to detect all objects to be screened within the field of view based on the sensing data measured by the sensor.

[0207] A generation submodule is used to generate the bounding box of each object to be filtered.

[0208] The determination submodule is used to determine the position information of the reference point in the bounding box as the position information of the object to be filtered.

[0209] In some embodiments, the occlusion state includes an occlusion ratio and an occlusion position; other sequential submodules 1022 are specifically used for:

[0210] If the occlusion ratio is not greater than the first threshold, the target object is determined to be an unoccluded object;

[0211] If the occlusion is unilateral, and the occlusion ratio is less than a second threshold, the target object is determined to be an unoccluded object; if the occlusion ratio is not less than the second threshold, the target object is determined to be an occluded object.

[0212] If the occlusion position is full occlusion or central occlusion, the target object is determined to be an occluded object;

[0213] Wherein, the second threshold is greater than the first threshold.

[0214] In some embodiments, the other sequence submodule 1022 is further configured to:

[0215] The contour curvature information of the target object and the unoccluded object are obtained respectively; the contour curvature information includes the curvature of the multiple edges of the contour relative to the direction of the sensor;

[0216] The occlusion ratio between the target object and the unoccluded object is determined based on the difference in curvature of the same-side edges of the target object and the contour curvature information of the target object.

[0217] In some embodiments, the contour curvature information is determined based on the vertex position information of the bounding box of the object; the vertex position information is polar coordinate position information, including the orientation and distance information of the vertices of the bounding box relative to the sensor.

[0218] In some embodiments, the occlusion filtering module 1020 is further configured to:

[0219] If the difference in curvature of the multiple edges of the target object is greater than the third threshold, the curvature of one edge of the multiple edges is updated according to the difference between the circumferential angle and the difference in curvature, so that the difference in curvature of the multiple edges after the update is less than the third threshold.

[0220] The occlusion ratio is determined using the updated radians.

[0221] In some embodiments, the first sequential submodule 1021 is further configured to:

[0222] If the target object is in the first priority position, the arc occupied by the target object within the field of view is determined as the occlusion range;

[0223] Other priority submodule 1022 is also used to determine the target object as an occluded object if it is determined that the target object is within the occlusion range before traversing all unoccluded objects when the target object is in the other priority position.

[0224] In some embodiments, the other sequence submodule 1022 is further configured to:

[0225] If the occlusion state between the target object and the unoccluded object is partial occlusion, the occlusion range is updated according to the curvature of at least one edge of the target object's contour relative to the sensor direction.

[0226] In some embodiments, the occlusion screening device 1000 further includes:

[0227] The output module is used to output all unoccluded objects if all the objects to be filtered have completed the occlusion filtering process, and / or if the occlusion range is consistent with the field of view.

[0228] In some embodiments, the occlusion screening device 1000 further includes:

[0229] A module is established to build a vehicle simulation scene model based on the object information of all unoccluded objects; the object information includes one or more of type information, shape information, and occlusion information.

[0230] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0231] Based on the occlusion filtering method described in any of the above embodiments, this application also provides, as well as... Figure 10 The diagram shows the structure of an electronic device. Figure 11 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the occlusion screening method described in any of the above embodiments.

[0232] This application also provides a computer storage medium storing a computer program, which, when executed by a processor, can be used to perform an occlusion screening method as described in any of the above embodiments.

[0233] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0234] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0235] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0236] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0237] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0238] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for screening by occlusion, characterized in that, The method is applied in vehicle simulation scenarios; the method includes: Acquire the position information of each object to be screened within the field of view of the vehicle-mounted sensor; Based on the location information, the target object is determined from all the objects to be filtered in order from near to far. Occlusion filtering is performed on each target object to determine the unoccluded object from all the objects to be filtered. The occlusion filtering process includes: If the target object is the first in the sequence, the target object is determined to be an unobstructed object; If the target object is in any other position in the sequence, traverse all unoccluded objects and determine whether the target object is an occluded object or an unoccluded object based on the occlusion state between the target object and the unoccluded objects.

2. The method according to claim 1, characterized in that, The acquisition of the position information of each object to be screened within the field of view of the vehicle-mounted sensor includes: Based on the sensing data measured by the sensor, all objects to be screened within the field of view are detected; For each of the objects to be filtered, generate the bounding box of the object to be filtered; The position information of the reference point in the bounding box is determined as the position information of the object to be filtered.

3. The method according to claim 1, characterized in that, The occlusion status includes the occlusion ratio and the occlusion position; The step of determining whether the target object is an occluded object or an unoccluded object based on the occlusion state between the target object and the unoccluded object includes: If the occlusion ratio is not greater than the first threshold, the target object is determined to be an unoccluded object; If the occlusion is unilateral, and the occlusion ratio is less than a second threshold, the target object is determined to be an unoccluded object; if the occlusion ratio is not less than the second threshold, the target object is determined to be an occluded object. If the occlusion position is full occlusion or central occlusion, the target object is determined to be an occluded object; Wherein, the second threshold is greater than the first threshold.

4. The method according to claim 3, characterized in that, The occlusion ratio is calculated through the following steps: The contour curvature information of the target object and the unoccluded object are obtained respectively; the contour curvature information includes the curvature of the multiple edges of the contour relative to the direction of the sensor; The occlusion ratio between the target object and the unoccluded object is determined based on the difference in curvature of the same-side edges of the target object and the contour curvature information of the target object.

5. The method according to claim 4, characterized in that, The contour curvature information is determined based on the vertex position information of the bounding box of the object; the vertex position information is polar coordinate position information, including the direction information and distance information of the vertices of the bounding box relative to the sensor.

6. The method according to claim 4, characterized in that, The method further includes: If the difference in curvature of the multiple edges of the target object is greater than the third threshold, the curvature of one edge of the multiple edges is updated according to the difference between the circumferential angle and the difference in curvature, so that the difference in curvature of the multiple edges after the update is less than the third threshold. The occlusion ratio is determined using the updated radians.

7. The method according to claim 1, characterized in that, The occlusion filtering process also includes: If the target object is in the first priority position, the arc occupied by the target object within the field of view is determined as the occlusion range; If the target object is in any of the other order, before traversing all unoccluded objects, if it is determined that the target object is within the occlusion range, the target object is determined to be an occluded object.

8. The method according to claim 7, characterized in that, The occlusion filtering process also includes: If the occlusion state between the target object and the unoccluded object is partial occlusion, the occlusion range is updated according to the curvature of at least one edge of the target object's contour relative to the sensor direction.

9. The method according to claim 8, characterized in that, The method further includes: If all the objects to be filtered have completed the occlusion filtering process, and / or if the occlusion range is consistent with the field of view, output all the unoccluded objects.

10. The method according to claim 1, characterized in that, The method further includes: A vehicle simulation scene model is established based on the object information of all unoccluded objects; the object information includes one or more of the following: type information, shape information, and occlusion information.

11. A screening device with obstruction, characterized in that, The device is used in vehicle simulation scenarios; it includes: The acquisition module is used to acquire the position information of each object to be screened within the field of view of the vehicle-mounted sensor; The occlusion filtering module is used to determine the target object from all the objects to be filtered according to the location information in order from near to far, perform occlusion filtering processing for each target object, and determine the unoccluded object from all the objects to be filtered. The occlusion filtering module includes: The first priority submodule is used to determine that the target object is an unoccluded object if the target object is the first priority in the order. Other order submodules are used to traverse all unoccluded objects if the target object is in another order, and determine whether the target object is an occluded object or an unoccluded object based on the occlusion state between the target object and the unoccluded objects.

12. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store processor-executable instructions; Wherein, when the processor invokes the executable instructions, it implements the operation of any one of the methods described in claims 1-10.

13. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-10.

Images