Devices, methods, electronic devices and readable storage media for determining the driving area

Abstract

A device, method, electronic device, and readable storage medium for determining a driving area are disclosed. The device for determining the driving area includes one or more processors configured to: acquire an initial sensing area at a first moment, a first fused sensing area and a second fused sensing area at a second moment; wherein the second moment is earlier than the first moment; determine a third fused sensing area and a fourth fused sensing area at the first moment based on the initial sensing area, the first fused sensing area, and the second fused sensing area; determine a target sensing area at the first moment based on the third fused sensing area and the fourth fused sensing area; and determine the driving area at the first moment based on the target sensing area. This disclosure can effectively suppress high-frequency jitter at the edges of the driving area and achieve a smooth transition between the road surface area and the non-road surface area.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a device, method, electronic device, and readable storage medium for determining a driving area. Background Technology

[0002] In intelligent driving technology, driving area detection models are typically based on multimodal sensor data collected by multiple sources of sensors (such as cameras, LiDAR, millimeter-wave radar, etc.). Deep neural networks are used to perform semantic segmentation of the environment surrounding the agent, outputting the driving area. The pixel value of each pixel within the driving area represents the confidence level that the pixel belongs to the driving area.

[0003] In practical applications, unstable phenomena such as high-frequency jitter, boundary jumps, and jagged transitions often occur in the driving area output by the driving area detection model, particularly at the edges of the driving area and at the boundaries between the road surface area and non-road surface area (such as road shoulders, green belts, sidewalks, and construction areas). Therefore, there is an urgent need for a method that can effectively suppress high-frequency jitter at the edges of the driving area and achieve a smooth transition between the road surface area and the non-road surface area. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides a device, method, electronic device, and readable storage medium for determining a driving area, thereby suppressing high-frequency jitter at the edge of the driving area and achieving a smooth transition between the road surface area and the non-road surface area.

[0005] A first aspect of this disclosure provides a driving area determination apparatus, including one or more processors, said one or more processors being configured to: The initial perception region at a first moment, the first fused perception region at a second moment, and the second fused perception region at a second moment are obtained; wherein the second moment is earlier than the first moment. Based on the initial sensing region, the first fused sensing region, and the second fused sensing region, the third fused sensing region and the fourth fused sensing region at the first moment are determined. Based on the third fusion sensing region and the fourth fusion sensing region, the target sensing region at the first moment is determined; Based on the target perception area, the driving area at the first moment is determined.

[0006] A second aspect of this disclosure provides a method for determining a driving area, including: The initial perception region at a first moment, the first fused perception region at a second moment, and the second fused perception region at a second moment are obtained; wherein the second moment is earlier than the first moment. Based on the initial sensing region, the first fused sensing region, and the second fused sensing region, the third fused sensing region and the fourth fused sensing region at the first moment are determined. Based on the third fusion sensing region and the fourth fusion sensing region, the target sensing region at the first moment is determined; Based on the target perception area, the driving area at the first moment is determined.

[0007] A third aspect of this disclosure provides a computer-readable storage medium storing a computer program that is executed by a processor to perform the method for determining a driving area provided in the second aspect of the present invention.

[0008] A fourth aspect of this disclosure provides an electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the method for determining a driving area provided in the second aspect of the disclosure.

[0009] A fifth aspect of this disclosure provides a computer program product that, when instructions in the computer program product are executed by a processor, performs the method for determining a driving area provided in a second aspect of this disclosure.

[0010] This disclosure provides an apparatus, method, electronic device, and readable storage medium for determining a driving area. The apparatus includes one or more processors configured to: acquire an initial sensing area at a first moment, and a first fused sensing area and a second fused sensing area at a second moment, wherein the second moment is earlier than the first moment; determine a third fused sensing area and a fourth fused sensing area at the first moment based on the initial sensing area, the first fused sensing area, and the second fused sensing area; determine a target sensing area at the first moment based on the third fused sensing area and the fourth fused sensing area; and determine the driving area at the first moment based on the target sensing area. The processors fuse the fused sensing areas from historical moments with the initial sensing area at the current moment, which can cumulatively enhance the confidence of road surface pixels and cumulatively suppress the confidence of non-road surface pixels, thereby maintaining a high confidence level for road surface pixels and a low confidence level for non-road surface pixels. In this way, the confidence level of a pixel will not change abruptly with the fluctuation of the confidence level of pixels in a single frame of data. This effectively filters out instantaneous confidence level fluctuations of pixels, making the confidence level of pixels more stable over time and avoiding abrupt transitions between road and non-road surfaces caused by fluctuations in the confidence level of pixels in a single frame of data. Moreover, by fusing the fused perception area from historical moments with the initial perception area at the current moment, the processor can achieve a smooth transition between road and non-road surfaces and filter out misidentified road surface pixels in the non-road surface area. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the structure of a device for determining a driving area provided in an exemplary embodiment of this disclosure.

[0012] Figure 2 This is a schematic diagram of a road surface area and a non-road surface area provided in an exemplary embodiment of this disclosure.

[0013] Figure 3 This is a schematic diagram of a sensing region mask provided in another exemplary embodiment of this disclosure.

[0014] Figure 4 This is a flowchart illustrating a method for determining a driving area provided in an exemplary embodiment of this disclosure.

[0015] Figure 5 This is a flowchart illustrating a method for determining a driving area provided in another exemplary embodiment of this disclosure.

[0016] Figure 6 This is a flowchart illustrating a method for determining a driving area provided in another exemplary embodiment of this disclosure.

[0017] Figure 7This is a flowchart illustrating a method for determining a driving area provided in another exemplary embodiment of this disclosure.

[0018] Figure 8 This is a flowchart illustrating a method for determining a driving area provided in another exemplary embodiment of this disclosure.

[0019] Figure 9 This is a flowchart illustrating a method for determining a driving area provided in another exemplary embodiment of this disclosure.

[0020] Figure 10 This is a flowchart illustrating a method for determining a driving area provided in another exemplary embodiment of this disclosure.

[0021] Figure 11 This is a flowchart illustrating a method for determining a driving area provided in another exemplary embodiment of this disclosure.

[0022] Figure 12 This is a flowchart illustrating a method for determining a driving area provided in another exemplary embodiment of this disclosure.

[0023] Figure 13 This is a schematic diagram of the structure of a device for determining a driving area provided in an exemplary embodiment of this disclosure.

[0024] Figure 14 This is a structural diagram of an electronic device provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0025] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0026] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0027] Application Overview The driving area determination device provided in this disclosure can be applied to, for example, intelligent driving scenarios and any other feasible scenarios.

[0028] The intelligent agent is equipped with at least one visual sensor. This visual sensor is used to collect visual perception data of the surrounding environment of the intelligent agent. The visual perception data may include two-dimensional image data. This disclosure does not limit the number or location of the visual sensors on the intelligent agent. In practical scenarios, the visual sensor can be a front-view wide-angle / narrow-angle camera, a side-view / surround-view / panoramic-view camera, an RGB-D (Red Green Blue-Depth) camera, etc., and this disclosure does not impose any limitations.

[0029] The intelligent driving described in this disclosure can encompass multiple fields, including autonomous driving, driver assistance systems, and robotic systems. Autonomous driving technology aims to enable intelligent agents to drive completely autonomously in various complex road conditions without human intervention; it is also known as driverless driving and represents an advanced form of intelligent driving. Driver assistance systems, through a series of sensors and algorithms, provide drivers with real-time road condition information, warnings, and support for some driving operations, such as automatic parking and adaptive cruise control, aiming to improve driving safety and convenience. Robotic systems further extend intelligent driving technology to service robots, industrial robots, and other fields, enabling robots to autonomously navigate, avoid obstacles, and complete specific tasks in complex environments, such as logistics delivery and warehouse management, demonstrating the broad application potential of intelligent driving technology in different scenarios.

[0030] In intelligent driving technology, driving area detection models are usually based on multi-modal sensor data collected by multi-source sensors (such as cameras, lidar, millimeter-wave radar, etc.), and use deep neural networks to perform semantic segmentation of the environment around the intelligent agent to output the driving area.

[0031] In practical applications, unstable phenomena such as high-frequency jitter, boundary jumps, and sawtooth transitions often occur in the driving area output by the driving area detection model at the edge area of ​​the driving area and at the junction of the road area and non-road area (such as road shoulders, green belts, sidewalks, construction areas, etc.).

[0032] The intelligent agent in this disclosure can be an intelligent driving vehicle, a robot, etc., and this disclosure does not limit it. This disclosure uses an intelligent driving vehicle as an example for description, and other situations are similar, and this disclosure will not repeat them.

[0033] The device for determining the driving area in this embodiment includes one or more processors, which are configured to: acquire an initial sensing area at a first moment, a first fused sensing area and a second fused sensing area at a second moment; wherein the second moment is earlier than the first moment; determine a third fused sensing area and a fourth fused sensing area at the first moment based on the initial sensing area, the first fused sensing area and the second fused sensing area; determine a target sensing area at the first moment based on the third fused sensing area and the fourth fused sensing area; and determine the driving area at the first moment based on the target sensing area. The processors fuse the fused sensing areas from historical moments with the initial sensing area at the current moment. This prevents abrupt changes in the road surface area and non-road surface area due to instantaneous fluctuations in a single frame of data, making the road surface area and non-road surface area more stable in the time dimension and avoiding jumps between the road surface area and non-road surface area caused by instantaneous fluctuations in a single frame of data. Furthermore, the processor's fusion of the fused sensing areas from historical moments with the initial sensing area at the current moment enables a smooth transition between the road surface area and the non-road surface area.

[0034] Exemplary System Figure 1 This is a schematic diagram of the structure of a device for determining a driving area provided in an exemplary embodiment of this disclosure. Figure 1 As shown, the device 100 for determining the driving area includes one or more processors 110. The one or more processors 110 are configured to: acquire an initial sensing area at a first moment, a first fused sensing area and a second fused sensing area at a second moment; wherein the second moment is earlier than the first moment; determine a third fused sensing area and a fourth fused sensing area at the first moment based on the initial sensing area, the first fused sensing area, and the second fused sensing area; determine a target sensing area at the first moment based on the third fused sensing area and the fourth fused sensing area; and determine the driving area at the first moment based on the target sensing area.

[0035] For example, a vision sensor installed on an intelligent driving vehicle can periodically acquire environmental images and transmit the acquired environmental images to a processor 110 in a device for determining the driving area. After receiving the environmental images, the processor 110 can perform image segmentation processing on the environmental images to obtain an initial perception area. The processor 110 can use a pre-trained neural network model to perform image segmentation processing on the environmental images to obtain the initial perception area, which will not be described in detail in this embodiment.

[0036] In one implementation, the perception region can be represented by a confidence map or a cost map. In the confidence map, the pixel value of each pixel represents the confidence that the region corresponding to that pixel is a road surface region; a larger pixel value indicates a higher probability that the region corresponding to that pixel is a road surface region. For example, the range of pixel values ​​is typically [0, 9]. In the cost map, the pixel value of each pixel represents the cost that the region corresponding to that pixel is a road surface region; a smaller pixel value indicates a higher probability that the region corresponding to that pixel is a road surface region.

[0037] It should be noted that the perception area described in this embodiment is the entire perception area determined based on sensor-collected perception data within a certain range centered on the intelligent driving vehicle. Alternatively, it can be a portion of the perception area of ​​interest further determined within the entire perception area. This perception area can include road surface areas and non-road surface areas. The road surface area encompasses the entire road space, including the road itself and its surrounding areas. This road surface area may include dynamic obstacles in motion or at rest, static obstacles, or none at all. Dynamic obstacles are those capable of autonomous movement or whose position can continuously shift under external forces; these may include motor vehicles, non-motor vehicles, and pedestrians. Static obstacles are those without autonomous movement, whose spatial position remains fixed for a long period, and which do not actively shift their position; these may include road guardrails, traffic cones, traffic signs, road cones, and road debris.

[0038] After obtaining the initial sensing region at the first moment, the processor 110 can determine the third and fourth fusion sensing regions at the second moment based on the first and second fusion sensing regions at the second moment. The process by which the processor 110 determines the third and fourth fusion sensing regions at the first moment based on the initial sensing region, the first fusion sensing region, and the second fusion sensing region will be described in detail later and will not be repeated here. Since the first moment is the first moment in the series, there is no second moment earlier than the first moment. Therefore, the processor 110 directly uses the initial sensing region at the first moment as the third and fourth fusion sensing regions at the first moment, respectively.

[0039] After obtaining the third and fourth fused sensing regions, the processor 110 can determine the target sensing region at the first moment based on these regions. The process by which the processor 110 determines the target sensing region at the first moment based on the third and fourth fused sensing regions will be described in detail later and will not be repeated here. After obtaining the target sensing region, the processor 110 can determine the road surface area within the target sensing region as the driving area at the first moment. This driving area can be used for path planning, obstacle avoidance, and environmental perception visualization. In one embodiment, when the driving area within the target sensing region is used for path planning and obstacle avoidance, the target sensing region can be the entire sensing region determined based on sensor-collected sensing data. In another embodiment, when the driving area within the target sensing region is used for environmental perception visualization, the target sensing region can be a portion of the sensing region of interest further determined within the entire sensing region.

[0040] In this embodiment, the processor 110 fuses the fused sensing region from historical moments with the initial sensing region at the current moment. This prevents abrupt changes in the road surface area and the non-road surface area due to instantaneous fluctuations in a single frame of data, making the road surface area and the non-road surface area more stable over time and avoiding jumps between them caused by instantaneous fluctuations in a single frame of data. Furthermore, the fusion of the fused sensing region from historical moments with the initial sensing region at the current moment by the processor 110 enables a smooth transition between the road surface area and the non-road surface area.

[0041] In one embodiment, when the processor 110 executes the process of determining the third and fourth fusion sensing regions at a first moment based on the initial sensing region, the first fusion sensing region, and the second fusion sensing region, it is specifically configured to: determine the fifth and sixth fusion sensing regions at the first moment based on the vehicle motion state data, the first fusion sensing region, and the second fusion sensing region; determine the third fusion sensing region at the first moment based on the initial sensing region and the fifth fusion sensing region; and determine the fourth fusion sensing region at the first moment based on the initial sensing region, the third fusion sensing region, and the sixth fusion sensing region.

[0042] For example, the position and orientation of an intelligent driving vehicle may change during driving. Therefore, the first and second fused perception regions at the second moment will be spatially misaligned with the initial perception region at the first moment due to the vehicle's motion. After obtaining the first and second fused perception regions at the second moment, the processor 110 can first perform rigid body transformation processing on the vehicle's motion state data to obtain the rotation and translation matrix between the second and first moments. The vehicle's motion state data may include the changes in yaw rate, lateral displacement, and longitudinal displacement between the second and first moments. After obtaining the rotation and translation matrix between the second and first moments, the processor 110 can rotate and translate the first and second fused perception regions at the second moment to the vehicle coordinate system of the intelligent driving vehicle at the first moment, respectively, to obtain the fifth and sixth fused perception regions at the first moment. The above processing procedure of the processor 110 can be expressed as the following formulas: fusion5 = RT × fusion1; fusion6 = RT × fusion2. Wherein, fusion5 represents the fifth fusion sensing region at the first time step, fusion1 represents the first fusion sensing region at the second time step, fusion6 represents the sixth fusion sensing region at the first time step, fusion2 represents the second fusion sensing region at the second time step, and RT represents the rotation and translation matrix between the second time step and the first time step.

[0043] After obtaining the fifth and sixth fused sensing regions, the processor 110 can determine the third fused sensing region at the first moment based on the initial sensing region and the fifth fused sensing region. That is, the processor 110 can fuse the initial sensing region and the fifth fused sensing region to obtain the third fused sensing region at the first moment. The process by which the processor 110 determines the third fused sensing region at the first moment based on the initial sensing region and the fifth fused sensing region will be described in detail later and will not be repeated here.

[0044] After obtaining the third fused sensing region, the processor 110 can further determine the fourth fused sensing region at the first moment based on the initial sensing region, the third fused sensing region, and the sixth fused sensing region. That is, the processor 110 can fuse the initial sensing region, the third fused sensing region, and the sixth fused sensing region to obtain the fourth fused sensing region at the first moment. The process by which the processor 110 determines the fourth fused sensing region at the first moment based on the initial sensing region, the third fused sensing region, and the sixth fused sensing region will be described in detail later and will not be repeated here.

[0045] In this embodiment, the processor 110 fuses the fused sensing region from historical moments with the initial sensing region at the current moment. This prevents abrupt changes in the road surface area and the non-road surface area due to instantaneous fluctuations in a single frame of data, making the road surface area and the non-road surface area more stable over time and avoiding jumps between them caused by instantaneous fluctuations in a single frame of data. Furthermore, the fusion of the fused sensing region from historical moments with the initial sensing region at the current moment by the processor 110 enables a smooth transition between the road surface and the non-road surface.

[0046] In one embodiment, when the processor 110 executes the process of determining the third fusion sensing region at a first moment based on the initial sensing region and the fifth fusion sensing region, it is specifically configured to: generate the road surface region and the non-road surface region at the first moment based on the initial sensing region; and determine the third fusion sensing region at the first moment based on the road surface region, the non-road surface region, and the fifth fusion sensing region.

[0047] For example, after acquiring the initial perception region at the first moment, the processor 110 can generate the road surface region and the non-road surface region at the first moment based on the initial perception region. The road surface region is used to cumulatively enhance the road surface region in the fifth fused perception region. The non-road surface region is used to cumulatively suppress the non-road surface region in the fifth fused perception region. The process by which the processor 110 generates the road surface region and the non-road surface region at the first moment based on the initial perception region will be described in detail later and will not be repeated here. After obtaining the road surface region and the non-road surface region at the first moment, the processor 110 can further determine the third fused perception region at the first moment based on the road surface region, the non-road surface region, and the fifth fused perception region. That is, the processor 110 can fuse the fifth fused perception region with the road surface region and the non-road surface region to obtain the third fused perception region. The process by which the processor 110 determines the third fused perception region at the first moment based on the road surface region, the non-road surface region, and the fifth fused perception region will be described in detail later and will not be repeated here.

[0048] In this embodiment, the processor 110 fuses the road surface region and the non-road surface region generated by the fused sensing region at a historical moment with the initial sensing region at the current moment. This allows for cumulative enhancement of the road surface region and cumulative suppression of the non-road surface region. In this way, the road surface region and the non-road surface region do not undergo abrupt changes due to instantaneous fluctuations in a single frame of data, making the road surface region and the non-road surface region more stable in the time dimension and avoiding jumps between the road surface region and the non-road surface region caused by instantaneous fluctuations in a single frame of data.

[0049] In one embodiment, when the processor 110 generates the road surface region and non-road surface region at a first moment based on the initial sensing region, it is specifically configured to: determine a first pixel point with a confidence level less than a first confidence level threshold and a second pixel point with a confidence level greater than or equal to the first confidence level threshold in the initial sensing region confidence map corresponding to the initial sensing region; set the confidence level of the first pixel point in the initial sensing region confidence map to the first confidence level and retain the confidence level of the second pixel point to generate the road surface region confidence map corresponding to the road surface region at the first moment; subtract the second confidence level from the confidence level of the first pixel point in the initial sensing region confidence map and set the confidence level of the second pixel point to the third confidence level to generate the non-road surface region confidence map corresponding to the non-road surface region at the first moment.

[0050] For example, after the processor 110 obtains the initial perception region confidence map corresponding to the initial perception region, it can first filter, based on a preset first confidence threshold, first pixels with confidence levels less than the first confidence threshold and second pixels with confidence levels greater than or equal to the first confidence threshold in the initial perception region confidence map. The first confidence threshold is used to determine whether any pixel in the initial perception region confidence map is a road surface pixel or a non-road surface pixel. If the confidence level of a pixel is less than the first confidence threshold, then the pixel is a non-road surface pixel. If the confidence level of a pixel is greater than or equal to the first confidence threshold, then the pixel is a road surface pixel. Therefore, the first pixel is a non-road surface pixel, and the second pixel is a road surface pixel. This first confidence threshold can be reasonably set by a technician based on the confidence range of the pixels in the initial perception region confidence map. For example, if the confidence range of the pixels in the initial perception region confidence map is [0, 9], the first confidence threshold can be set to 4. Further, refer to... Figure 2 In the initial perception region confidence map, the first pixel without shadow filling is a non-road pixel, and the second pixel filled with right diagonal shadow filling is a road pixel.

[0051] After the processor 110 determines the first pixel (i.e., the non-road pixel) and the second pixel (i.e., the road pixel) in the initial sensing region, it can set the confidence level of the first pixel in the initial sensing region confidence map to the first confidence level and retain the confidence level of the second pixel to generate a road region confidence map corresponding to the road region at the first time step. Since the road region confidence map is used to cumulatively enhance the confidence level of the road pixels, the confidence level of the road pixels (i.e., the second pixel) in the road region confidence map needs to retain its original confidence level value in the initial sensing region confidence map. To avoid the road region confidence map affecting the confidence level of non-road pixels, the confidence level of the non-road pixels (i.e., the first pixel) in the road region confidence map is set to the first confidence level. The smaller the first confidence level, the smaller the impact of the road region confidence map on the confidence level of non-road pixels. When the first confidence level is set to 0, the impact of the road region confidence map on the confidence level of non-road pixels is minimal. For example, refer to... Figure 2 The confidence scores of non-road pixels (i.e., the first pixel) in the road area confidence map are set to 0, while the confidence scores of road pixels (i.e., the second pixel) retain their original confidence scores.

[0052] After the processor 110 determines the first pixel (i.e., the non-road pixel) and the second pixel (i.e., the road pixel) in the initial sensing region, it can subtract the second confidence level from the confidence level of the first pixel in the initial sensing region confidence map and set the confidence level of the second pixel to the third confidence level, generating a non-road region confidence map corresponding to the non-road region at the first time step. The non-road region confidence map is used to cumulatively suppress the confidence levels of the non-road pixels. Therefore, the confidence levels of the non-road pixels in the non-road region confidence map must be negative. Based on this, the confidence level of the first pixel (i.e., the non-road pixel) in the initial sensing region confidence map can be subtracted from the second confidence level to obtain the confidence level of the non-road pixels in the non-road region confidence map. Since the confidence level of the first pixel is less than the first confidence threshold, the second confidence level must be greater than or equal to the first confidence threshold to ensure that the confidence level of the non-road pixels in the non-road region confidence map is negative. Meanwhile, the higher the second confidence level, the greater the cumulative suppression effect of the non-road area confidence map on the confidence levels of non-road pixels. The second confidence level can be set to the first confidence threshold or other confidence values; this embodiment is not limited thereto. To avoid the non-road area confidence map affecting the confidence levels of road pixels, the confidence level of the road pixels (i.e., the second pixels) in the non-road area confidence map is set to the third confidence level. The lower the third confidence level, the smaller the impact of the non-road area confidence map on the confidence levels of road pixels. When the third confidence level is set to 0, the impact of the non-road area confidence map on the confidence levels of road pixels is minimal.

[0053] In this embodiment, the processor 110 fuses a road surface area confidence map and a non-road surface area confidence map generated by combining the fused perception area confidence map from historical moments with the initial perception area confidence map from the current moment. This fusion enhances the confidence of road surface pixels and suppresses the cumulative confidence of non-road surface pixels, thus maintaining a higher confidence level for road surface pixels and a lower confidence level for non-road surface pixels. This prevents sudden changes in pixel confidence levels due to fluctuations in pixel confidence levels within a single frame of data, effectively filtering out instantaneous confidence level fluctuations and making pixel confidence levels more stable over time. This avoids abrupt transitions between road and non-road surfaces caused by fluctuations in pixel confidence levels within a single frame of data.

[0054] In one embodiment, when the processor 110 executes the process of determining the third fusion sensing region at a first moment based on the road surface region, the non-road surface region, and the fifth fusion sensing region, it is specifically configured to: determine the first fusion coefficient and the second fusion coefficient based on the vehicle's yaw rate; and determine the third fusion sensing region at the first moment based on the road surface region, the first fusion coefficient, the non-road surface region, the second fusion coefficient, and the fifth fusion sensing region.

[0055] For example, when the autonomous vehicle makes a rapid turn (large yaw rate) at the first moment, the current perception area of ​​the autonomous vehicle changes drastically. At this time, the processor 110 increases the first fusion coefficient corresponding to the road surface area generated from the initial perception area and the second fusion coefficient corresponding to the non-road surface area, thereby ensuring that the third fused perception area can fully match the current perception area changes of the autonomous vehicle. When the autonomous vehicle travels straight (yaw rate remains constant) or turns slowly (large yaw rate with small changes) at the first moment, the current perception area changes of the autonomous vehicle are relatively stable. At this time, the processor 110 decreases the first fusion coefficient corresponding to the road surface area generated from the initial perception area and the second fusion coefficient corresponding to the non-road surface area, thereby ensuring that the third fused perception area can fully utilize the historical fused perception area (i.e., the fifth fused perception area). Therefore, the processor 110 can determine the first fusion coefficient corresponding to the road surface area and the second fusion coefficient corresponding to the non-road surface area based on the vehicle's yaw rate. The larger the vehicle's yaw rate, the larger the first and second fusion coefficients. For example, the vehicle's yaw rate and the first and second fusion coefficients can satisfy the following formula:

[0056]

[0057] Where coefficient1 represents the first fusion coefficient, coefficient2 represents the second fusion coefficient, and yawrate represents the yaw rate of the vehicle.

[0058] After obtaining the first fusion coefficient corresponding to the road surface area and the second fusion coefficient corresponding to the non-road surface area, the processor 110 can further perform weighted fusion of the road surface area, the non-road surface area and the fifth fusion sensing area based on the first fusion coefficient and the second fusion coefficient to obtain the third fusion sensing area at the first moment.

[0059] In one implementation, taking the road surface area, non-road surface area, fifth fusion sensing area, and third fusion sensing area as represented by confidence maps, for each pixel, the processor 110 can calculate the first product of the pixel's confidence score in the road surface area confidence map and the first fusion coefficient, and calculate the second product of the pixel's confidence score in the non-road surface area confidence map and the second fusion coefficient. Then, the processor 110 can sum the first product, the second product, and the pixel's confidence score in the fifth fusion sensing area confidence map to obtain the pixel's confidence score in the third fusion sensing area confidence map. The above processing procedure of the processor 110 can be expressed as the following formula: fusion3 i，j =fusion5 i，j +coefficient1×Road i，j +coefficient2×Non_Road i，j Among them, fusion3 i，j This represents the confidence score of the pixel at coordinates (i, j) in the confidence map of the third fusion sensing region. i，j This represents the confidence score of the pixel at coordinates (i, j) in the confidence map of the fifth fusion sensing region. i，j Non_Road represents the confidence score of the pixel with coordinates (i, j) in the road area confidence map. i，jThis represents the confidence score of the pixel at coordinates (i, j) in the non-road area confidence map. `coefficient1` represents the first fusion coefficient, and `coefficient2` represents the second fusion coefficient. Specifically, in the third fused perception area confidence map, the confidence score of a road surface pixel enhanced by the road surface area confidence map is higher than its confidence score in the fifth fused perception area confidence map, while the confidence score of a non-road surface pixel suppressed by the non-road surface area confidence map is lower than its confidence score in the fifth fused perception area confidence map. For example, a road surface pixel has a confidence score of 8 in the fifth fused perception area confidence map and a confidence score of 9.6 in the third fused perception area confidence map. Similarly, a non-road surface pixel has a confidence score of 2 in the fifth fused perception area confidence map and a confidence score of 1.8 in the third fused perception area confidence map; another non-road surface pixel has a confidence score of 0 in the fifth fused perception area confidence map and a confidence score of -0.4 in the third fused perception area confidence map.

[0060] After processor 110 continuously accumulates and enhances the confidence of road surface pixels across multiple frames, the confidence of these pixels increases. Conversely, after processor 110 continuously accumulates and suppresses the confidence of non-road surface pixels across multiple frames, the confidence of these pixels decreases. Furthermore, the difference between the confidence of road surface pixels and non-road surface pixels also increases. Thus, assuming a road surface pixel is misidentified as a non-road surface pixel in the initial perception region confidence map, its confidence in the non-road surface region confidence map will be low. However, due to the cumulative enhancement across multiple frames, its confidence in the fifth fused perception region confidence map is high. Therefore, the confidence of this road surface pixel in the third fused perception region confidence map is only slightly lower than its confidence in the fifth fused perception region confidence map, preventing it from being directly misidentified as a non-road surface pixel. This avoids the road surface / non-road surface jump caused by fluctuations in the confidence of pixels within a single frame of data. Similarly, suppose that in the initial perception region confidence map, a non-road pixel is misidentified as a road pixel. This non-road pixel has a high confidence level in the road region confidence map. However, due to the cumulative suppression over multiple frames, its confidence level in the fifth fused perception region confidence map is low. Therefore, the confidence level of this non-road pixel in the third fused perception region confidence map is only slightly higher than its confidence level in the fifth fused perception region confidence map, preventing it from being directly misidentified as a road pixel. This avoids the road-to-non-road transition caused by fluctuations in pixel confidence levels within a single frame.

[0061] It should be noted that, compared to the confidence range of pixels in the initial perception region confidence map, the confidence range of pixels in the third fused perception region confidence map can be set to a larger range. This ensures that the confidence of road surface pixels can be continuously accumulated and enhanced. For example, the confidence range of pixels in the initial perception region confidence map is [0, 9], while the confidence range of pixels in the third fused perception region confidence map can be set to [0, 12]. In the third fused perception region confidence map, when the confidence of a road surface pixel after road surface enhancement exceeds the upper limit of the confidence range, the processor 110 sets the confidence of that road surface pixel to the upper limit of the confidence range. For example, if the confidence of a road surface pixel is 14, then the processor 110 sets the confidence of that road surface pixel to 12. Similarly, in the confidence map of the third fused sensing region, when the confidence of a non-road pixel after non-road region suppression is less than the lower limit of the confidence range, the processor 110 sets the confidence of the non-road pixel to the lower limit of the confidence range. For example, if the confidence of a non-road pixel is -0.4, the processor 110 sets the confidence of the non-road pixel to 0.

[0062] In this embodiment, the processor 110 determines a first fusion coefficient and a second fusion coefficient based on the vehicle's yaw rate. Then, based on the first and second fusion coefficients, the processor 110 performs weighted fusion of the road surface area, the non-road surface area, and the fifth fused perception area to obtain a third fused perception area at a first moment. This ensures that when the intelligent driving vehicle is making rapid turns (with a large yaw rate), the third fused perception area fully reflects the current changes in the intelligent driving vehicle's perception area. Conversely, when the intelligent driving vehicle is traveling straight (with a constant yaw rate) or turning slowly (with a large but small change in yaw rate), the third fused perception area can fully utilize historical fused perception areas.

[0063] In one embodiment, when the processor 110 executes the process of determining the fourth fusion sensing region at a first moment based on the initial sensing region, the third fusion sensing region, and the sixth fusion sensing region, it is specifically configured to: determine the sensing region mask at the first moment based on the third fusion sensing region; determine the seventh fusion sensing region at the first moment based on the initial sensing region and the sixth fusion sensing region; and determine the fourth fusion sensing region at the first moment based on the sensing region mask and the seventh fusion sensing region.

[0064] For example, in the third fused sensing region, the road surface area is cumulatively enhanced while the non-road surface area is cumulatively suppressed. That is, the road surface area and the non-road surface area are clearly distinguished in the third fused sensing region. Therefore, after obtaining the third fused sensing region, the processor 110 can generate a sensing region mask for the first moment based on the third fused sensing region. To maintain the complete road surface area and achieve a natural and smooth transition between the road surface area and the non-road surface area, the road surface area and its adjacent non-road surface area are marked as preserved areas in the sensing region mask, while other non-road surface areas are marked as occluded areas. The process by which the processor 110 determines the sensing region mask for the first moment based on the third fused sensing region will be described in detail later and will not be repeated here. After obtaining the initial sensing region for the first moment, the processor 110 can further determine the seventh fused sensing region for the first moment based on the initial sensing region and the sixth fused sensing region. That is, the processor 110 can fuse the initial sensing region and the sixth fused sensing region to obtain the seventh fused sensing region for the first moment. Thus, in the case of a jagged transition between the road surface area and the non-road surface area in the initial sensing region, a smooth transition is achieved between the road surface area and the non-road surface area in the sixth fused sensing region, since the transition between these areas has already been smoothed. The process by which the processor 110 determines the seventh fused sensing region at the first moment based on the initial sensing region and the sixth fused sensing region will be described in detail later and will not be repeated here. After obtaining the seventh fused sensing region, the processor 110 can perform masking processing on the seventh fused sensing region based on the sensing region mask to obtain the fourth fused sensing region at the first moment.

[0065] In one implementation, taking the fourth and seventh fusion sensing regions as represented by confidence maps as an example, for each pixel in the confidence map of the fourth fusion sensing region, if the sensing region mask of the pixel is a reserved mask (represented by 1), then the confidence of the pixel in the confidence map of the fourth fusion sensing region is the same as its confidence in the confidence map of the seventh fusion sensing region. If the sensing region mask of the pixel is a masking mask (represented by 0), then the confidence of the pixel in the confidence map of the fourth fusion sensing region is 0. Thus, even if some non-road surface pixels in the non-road surface region are identified as road surface pixels, by masking the confidence map of the seventh fusion sensing region using the sensing region mask, it can be ensured that some misidentified road surface pixels in the non-road surface region are filtered out in the confidence map of the fourth fusion sensing region. The above processing procedure of the processor 110 can be expressed as the following formula:

[0066] Among them, fusion4 i，jThis represents the confidence level of the pixel with coordinates (i, j) in the fourth fusion sensing region, fusion7. i，j Mask represents the confidence level of the pixel with coordinates (i, j) in the seventh fusion sensing region. i，j The receptive region mask representing the pixel with pixel coordinates (i, j).

[0067] In this embodiment, the processor 110 determines a sensing region mask at a first moment based on a third fused sensing region. Then, the processor 110 determines a seventh fused sensing region at the first moment based on the initial sensing region and a sixth fused sensing region. Afterward, the processor 110 determines a fourth fused sensing region at the first moment based on the sensing region mask and the seventh fused sensing region. This achieves a smooth transition between the road surface region and the non-road surface region in the fourth fused sensing region, while filtering out misidentified road surface regions in the non-road surface region.

[0068] In one embodiment, when the processor 110 executes the process of determining the sensing region mask at a first moment based on the third fused sensing region, it is specifically configured to: determine a third pixel point with a confidence level greater than a preset second confidence threshold and a fourth pixel point with a confidence level less than or equal to the second confidence threshold in the fused sensing region confidence map corresponding to the third fused sensing region; determine a fifth pixel point among the fourth pixel points that satisfies a preset proximity condition with the third pixel point; set the sensing region masks of the third pixel point and the fifth pixel point as reserved masks, and set the sensing region masks of other pixels besides the third pixel point and the fifth pixel point as occlusion masks, thereby obtaining the sensing region mask at the first moment.

[0069] For example, after the processor 110 obtains the confidence map of the fusion sensing region corresponding to the third fusion sensing region (i.e., the third fusion sensing region confidence map), it can filter third pixels with a confidence level greater than a preset second confidence threshold and fourth pixels with a confidence level less than or equal to the second confidence threshold. The second confidence threshold is used to determine whether any pixel in the third fusion sensing region confidence map is a road surface pixel or a non-road surface pixel. If the confidence level of a pixel is less than or equal to the second confidence threshold, then the pixel is a non-road surface pixel. If the confidence level of a pixel is greater than the second confidence threshold, then the pixel is a road surface pixel. Since the confidence level of road surface pixels in the third fusion sensing region confidence map is cumulatively enhanced to a higher confidence value, while the confidence level of non-road surface pixels is cumulatively suppressed to a lower confidence value, the second confidence threshold can be set to a lower confidence value compared to the first confidence threshold. For example, the second confidence threshold can be set to 0 or 1, or to other confidence values; this disclosure does not limit the specific settings. For example, refer to... Figure 3In the confidence map of the third fusion perception region, the third pixel filled with the left diagonal shadow is the road surface pixel, and the fourth pixel without shadow filling or with horizontal shadow filling is the non-road surface pixel.

[0070] After obtaining the third pixel (i.e., the road surface pixel), the processor 110 can determine a fifth pixel among the fourth pixels (i.e., non-road surface pixels) that satisfies a preset proximity condition with the third pixel. In one embodiment, for each third pixel, the processor 110 can calculate the distance between the third pixel and each of the fourth pixels based on the pixel coordinates of the third pixel and the pixel coordinates of each fourth pixel. Then, the processor 110 can further determine the fourth pixels whose distance is less than or equal to a preset distance threshold as the fifth pixel that satisfies the preset proximity condition with the third pixel. For example, refer to... Figure 3 In the confidence map of the third fusion perception region, the pixel filled with the horizontal shaded line is the fifth pixel.

[0071] After obtaining the third and fifth pixels, the processor 110 can set the receptive region masks of the third and fifth pixels to reserved masks (represented by 1), and set the receptive region masks of all pixels except the third and fifth pixels to occlusion masks (represented by 0), thus obtaining the receptive region mask map corresponding to the receptive region mask at the first moment. For example, refer to... Figure 3 In the confidence map of the third fused perception region, the third and fifth pixels are preserved by the perception region mask (represented by 1) in the perception region mask map, while the other pixels are occluded by the perception region mask (represented by 0).

[0072] In this embodiment of the present disclosure, in the perception region mask determined by the processor 110, the perception region masks of the road surface pixels and their adjacent non-road surface pixels are reserved masks, while the perception region masks of other pixels are occlusion masks. In this way, the perception region mask includes part of the road surface edge transition region, thereby ensuring that the complete road surface region is maintained in the final generated target perception region confidence map, while achieving a natural smoothness between the road surface region and the non-road surface region.

[0073] In one embodiment, when the processor 110 executes the process of determining the seventh fusion perception region at a first moment based on the initial perception region and the sixth fusion perception region, it is specifically configured to: determine the third fusion coefficient based on the vehicle's yaw rate; and determine the seventh fusion perception region at the first moment based on the initial perception region, the third fusion coefficient, and the sixth fusion perception region.

[0074] For example, when the autonomous vehicle makes a rapid turn (large yaw rate) at the first moment, the current perception area of ​​the autonomous vehicle changes drastically. At this time, the processor 110 increases the third fusion coefficient corresponding to the initial perception area, thereby ensuring that the seventh fused perception area can fully match the current perception area change of the autonomous vehicle. However, when the autonomous vehicle travels straight (yaw rate remains constant) or turns slowly (large yaw rate with small change) at the first moment, the current perception area change of the autonomous vehicle is relatively stable. At this time, the processor 110 decreases the third fusion coefficient corresponding to the initial perception area, thereby ensuring that the seventh fused perception area can fully utilize the historical fused perception area (i.e., the sixth fused perception area). Therefore, the processor 110 can determine the third fusion coefficient corresponding to the initial perception area based on the vehicle's yaw rate. The larger the vehicle's yaw rate, the larger the third fusion coefficient. For example, the relationship between the vehicle's yaw rate and the third fusion coefficient can satisfy the following formula:

[0075] Here, coefficient3 represents the third fusion coefficient, and yawrate represents the yaw rate of the vehicle.

[0076] After obtaining the third fusion coefficient corresponding to the initial sensing region, the processor 110 can further perform weighted fusion of the initial sensing region and the sixth fused sensing region based on the third fusion coefficient to obtain the seventh fused sensing region at the first moment.

[0077] In one implementation, taking the initial sensing region, the sixth fusion sensing region, and the seventh fusion sensing region as represented by a confidence map as an example, for each pixel, the processor 110 can calculate the first product of the confidence score of the pixel in the confidence map of the sixth fusion sensing region and the complement of the third fusion coefficient, and calculate the second product of the confidence score of the pixel in the confidence map of the initial sensing region and the third fusion coefficient. Then, the processor 110 can sum the first and second products to obtain the confidence score of the pixel in the confidence map of the seventh fusion sensing region. The above processing procedure of the processor 110 can be expressed as the following formula: fusion7 i，j =(1-coefficient3)×fusion6 i，j +coefficient3×roadseg i，j Among them, fusion7 i，j This represents the confidence score of the pixel at coordinates (i, j) in the confidence map of the seventh fusion sensing region, fusion6. i，j This represents the confidence score of the pixel with coordinates (i, j) in the confidence map of the sixth fused sensing region, roadseg i，jrepresents the confidence score of the pixel with pixel coordinates (i, j) in the initial receptive region confidence map, and coefficient3 represents the third fusion coefficient.

[0078] It should be noted that the confidence range of pixels in the confidence map of the seventh fused perception region can be consistent with the confidence range of pixels in the confidence map of the initial perception region. For example, if the confidence range of pixels in the initial perception region confidence map is [0, 9], then the confidence range of pixels in the confidence map of the seventh fused perception region can be set to [0, 9]. In the confidence map of the seventh fused perception region, when the confidence of a pixel is greater than the upper limit of the confidence range, the processor 110 sets the confidence of that pixel to the upper limit of the confidence range. Similarly, in the confidence map of the seventh fused perception region, when the confidence of a pixel is less than the lower limit of the confidence range, the processor 110 sets the confidence of that pixel to the lower limit of the confidence range.

[0079] In this embodiment, the processor 110 determines a third fusion coefficient based on the vehicle's yaw rate. Then, based on the third fusion coefficient, the processor 110 performs a weighted fusion of the initial perception region and the sixth fused perception region to obtain a seventh fused perception region at the first moment. This ensures that when the autonomous vehicle is making rapid turns (with a large yaw rate), the seventh fused perception region fully reflects the current changes in the autonomous vehicle's perception region. Conversely, when the autonomous vehicle is traveling straight (with a constant yaw rate) or turning slowly (with a large but small change in yaw rate), the seventh fused perception region can fully utilize the historical fused perception regions.

[0080] In one embodiment, when the processor 110 executes the process of determining the target sensing region at a first moment based on the third fusion sensing region and the fourth fusion sensing region, it is specifically configured to: determine the backbone region at the first moment based on the third fusion sensing region; determine the eighth fusion sensing region at the first moment based on the backbone region and the fourth fusion sensing region; and determine the target sensing region at the first moment based on the eighth fusion sensing region.

[0081] For example, because the road surface area is cumulatively enhanced while the non-road surface area is cumulatively suppressed in the third fused sensing region, that is, the road surface area and the non-road surface area are clearly distinguished in the third fused sensing region. Therefore, after obtaining the third fused sensing region, the processor 110 can determine the backbone region at the first moment based on the third fused sensing region. That is, the processor 110 can filter the backbone region in the third fused sensing region.

[0082] In one implementation, taking the third fused sensing region and the backbone region as represented by a confidence map as an example, for each pixel in the confidence map of the third fused sensing region, if the confidence level of the pixel in the confidence map of the third fused sensing region is greater than or equal to the third confidence threshold, then the confidence level of the pixel in the confidence map of the backbone region is the same as its confidence level in the confidence map of the third fused sensing region. If the confidence level of the pixel in the confidence map of the third fused sensing region is less than the third confidence threshold, then the confidence level of the pixel in the confidence map of the backbone region is 0. The third confidence threshold can be set to a high confidence threshold. For example, the third confidence threshold is 8 or 9, or it can be other confidence values; this embodiment of the disclosure is not limited to any particular value. The above processing procedure of the processor 110 can be expressed as the following formula:

[0083] Among them, main_road i，j This represents the confidence score of the pixel with coordinates (i, j) in the backbone region confidence map, fusion3 i，j represents the confidence level of the pixel with pixel coordinates (i, j) in the confidence map of the third fusion sensing region, and threshold represents the third confidence threshold.

[0084] After obtaining the backbone region, the processor 110 can fuse the backbone region and the fourth fusion sensing region to obtain the eighth fusion sensing region at the first moment.

[0085] In one implementation, taking the backbone region, the fourth fusion sensing region, and the eighth fusion sensing region as examples, and for each pixel in the backbone region confidence map, the processor 110 can sum the confidence score of that pixel in the backbone region confidence map with its confidence score in the fourth fusion sensing region confidence map. Then, the processor 110 can use the sum as the confidence score of that pixel in the eighth fusion sensing region confidence map. The above processing procedure of the processor 110 can be expressed as the following formula: fusion8 i，j =main_road i，j +fusion4 i，j Among them, fusion8 i，j This represents the confidence score of the pixel at coordinates (i, j) in the confidence map of the eighth fused sensing region, main_road i，j This represents the confidence score of the pixel with coordinates (i, j) in the backbone region confidence map, fusion4 i，jThis represents the confidence level of the pixel with coordinates (i, j) in the fourth fused perception region confidence map. The confidence level range of pixels in the eighth fused perception region confidence map can be consistent with the confidence level range of pixels in the initial perception region confidence map. For example, if the confidence level range of pixels in the initial perception region confidence map is [0, 9], then the confidence level range of pixels in the eighth fused perception region confidence map can be set to [0, 9]. In the eighth fused perception region confidence map, when the confidence level of a pixel is greater than the upper limit of the confidence level range, the processor 110 sets the confidence level of that pixel to the upper limit of the confidence level range. Similarly, in the eighth fused perception region confidence map, when the confidence level of a pixel is less than the lower limit of the confidence level range, the processor 110 sets the confidence level of that pixel to the lower limit of the confidence level range.

[0086] To make the confidence level of the pixels smoother, after obtaining the eighth fusion sensing region, the processor 110 can also perform mean filtering on the eighth fusion sensing region to obtain the target sensing region at the first moment.

[0087] In this embodiment, the processor 110 determines the backbone region at a first moment based on the third fused sensing region. Then, the processor 110 determines the eighth fused sensing region at the first moment based on the backbone region and the fourth fused sensing region. Afterwards, the processor 110 determines the target sensing region at the first moment based on the eighth fused sensing region. This ensures the stability of the backbone region within the target sensing region. Simultaneously, the processor 110 uses mean filtering to make the transition between the road surface area and the non-road surface area in the final output target sensing region more natural and smooth.

[0088] Exemplary methods Figure 4 This is a flowchart illustrating a method for determining a driving area provided in an exemplary embodiment of this disclosure. Figure 4 As shown, it includes the following steps: Step 401: Obtain the initial perception region at the first moment, and the first fused perception region and the second fused perception region at the second moment. The second moment is earlier than the first moment.

[0089] Step 402: Based on the initial sensing region, the first fusion sensing region, and the second fusion sensing region, determine the third fusion sensing region and the fourth fusion sensing region at the first moment.

[0090] Step 403: Determine the target perception area at the first moment based on the third and fourth fusion perception areas.

[0091] Step 404: Determine the driving area at the first moment based on the target perception area.

[0092] In one embodiment, such as Figure 5 As shown above, in the above Figure 4 Based on the illustrated embodiment, step 402 may include the following steps: Step 501: Based on the vehicle motion state data, the first fusion sensing region, and the second fusion sensing region, determine the fifth and sixth fusion sensing regions at the first moment.

[0093] Step 502: Based on the initial sensing region and the fifth fusion sensing region, determine the third fusion sensing region at the first moment.

[0094] Step 503: Based on the initial sensing region, the third fusion sensing region, and the sixth fusion sensing region, determine the fourth fusion sensing region at the first moment.

[0095] In one embodiment, such as Figure 6 As shown above, in the above Figure 5 Based on the illustrated embodiment, step 502 may include the following steps: Step 601: Based on the initial sensing area, generate the road surface area and non-road surface area at the first moment.

[0096] Step 602: Based on the road surface area, the non-road surface area, and the fifth fusion sensing area, determine the third fusion sensing area at the first moment.

[0097] In one embodiment, such as Figure 7 As shown above, in the above Figure 6 Based on the illustrated embodiment, step 601 may include the following steps: Step 701: Determine the first pixel point with a confidence level less than the first confidence threshold and the second pixel point with a confidence level greater than or equal to the first confidence threshold in the confidence map of the initial perception region corresponding to the initial perception region.

[0098] Step 702: Set the confidence level of the first pixel in the initial perception region confidence map to the first confidence level, and retain the confidence level of the second pixel to generate the road surface region confidence map corresponding to the road surface region at the first moment.

[0099] Step 703: Subtract the second confidence level from the confidence level of the first pixel in the initial perception area confidence map, and set the confidence level of the second pixel to the third confidence level to generate the non-road area confidence map corresponding to the non-road area at the first time.

[0100] In one embodiment, such as Figure 8 As shown above, in the above Figure 6 Based on the illustrated embodiment, step 602 may include the following steps: Step 801: Determine the first fusion coefficient and the second fusion coefficient based on the yaw rate of the vehicle.

[0101] Step 802: Based on the road surface area, the first fusion coefficient, the non-road surface area, the second fusion coefficient, and the fifth fusion sensing area, determine the third fusion sensing area at the first moment.

[0102] In one embodiment, such as Figure 9 As shown above, in the above Figure 5 Based on the illustrated embodiment, step 503 may include the following steps: Step 901: Determine the sensing region mask at the first moment based on the third fused sensing region.

[0103] Step 902: Based on the initial sensing region and the sixth fusion sensing region, determine the seventh fusion sensing region at the first moment.

[0104] Step 903: Based on the sensing region mask and the seventh fused sensing region, determine the fourth fused sensing region at the first moment.

[0105] In one embodiment, such as Figure 10 As shown above, in the above Figure 9 Based on the illustrated embodiment, step 901 may include the following steps: Step 1001: Determine the third pixel point with a confidence level greater than the preset second confidence threshold and the fourth pixel point with a confidence level less than or equal to the second confidence threshold in the confidence map of the third fusion sensing region.

[0106] Step 1002: Determine the fifth pixel in the fourth pixel that satisfies the preset proximity condition with the third pixel.

[0107] Step 1003: Set the receptive region masks of the third and fifth pixels to reserved masks, and set the receptive region masks of the other pixels except the third and fifth pixels to occlusion masks, to obtain the receptive region masks at the first moment.

[0108] In one embodiment, such as Figure 11 As shown above, in the above Figure 9 Based on the illustrated embodiment, step 902 may include the following steps: Step 1101: Determine the third fusion coefficient based on the yaw rate of the vehicle.

[0109] Step 1102: Based on the initial sensing region, the third fusion coefficient, and the sixth fusion sensing region, determine the seventh fusion sensing region at the first moment.

[0110] In one embodiment, such as Figure 12 As shown above, in the above Figure 4 Based on the illustrated embodiment, step 403 may include the following steps: Step 1201: Based on the third fusion sensing region, determine the backbone region at the first moment.

[0111] Step 1202: Based on the main region and the fourth fusion sensing region, determine the eighth fusion sensing region at the first moment.

[0112] Step 1203: Based on the eighth fusion perception region, determine the target perception region at the first moment.

[0113] Regarding the method for determining a driving area in the above embodiments, the specific manner of each step and the corresponding beneficial effects have been described in detail in the corresponding embodiment section of the aforementioned driving area determination device section. Please refer to the corresponding execution method and beneficial technical effects of the above exemplary system section, which will not be repeated here.

[0114] Exemplary device Figure 13 This is a schematic diagram of the structure of a device for determining a driving area provided in an exemplary embodiment of this disclosure. Figure 13 As shown, the device for determining the driving area includes a sensing area acquisition module 1310, a first sensing area determination module 1320, a second sensing area determination module 1330, and a driving area determination module 1340.

[0115] The sensing region acquisition module 1310 is used to acquire the initial sensing region at a first moment, the first fused sensing region at a second moment, and the second fused sensing region at a second moment; wherein the second moment is earlier than the first moment. The first sensing region determination module 1320 is used to determine the third and fourth fusion sensing regions at the first moment based on the initial sensing region, the first fusion sensing region, and the second fusion sensing region. The second sensing region determination module 1330 is used to determine the target sensing region at the first moment based on the third fused sensing region and the fourth fused sensing region. The driving area determination module 1340 is used to determine the driving area at the first moment based on the target perception area.

[0116] In one embodiment, the first sensing region determination module 1320 includes: The first sensing region determination unit is used to determine the fifth and sixth fusion sensing regions at the first moment based on the vehicle motion state data, the first fusion sensing region, and the second fusion sensing region. The second sensing region determination unit is used to determine the third fusion sensing region at the first moment based on the initial sensing region and the fifth fusion sensing region. The third sensing region determination unit is used to determine the fourth fusion sensing region at the first moment based on the initial sensing region, the third fusion sensing region, and the sixth fusion sensing region.

[0117] In one embodiment, the second sensing region determining unit is specifically used for: Based on the initial sensing area, the road surface area and non-road surface area at the first moment are generated; Based on the road surface area, the non-road surface area, and the fifth fusion sensing area, the third fusion sensing area at the first moment is determined.

[0118] In one embodiment, the second sensing region determining unit is specifically used for: Identify the first pixel point with a confidence level less than the first confidence threshold and the second pixel point with a confidence level greater than or equal to the first confidence threshold in the confidence map of the initial perception region corresponding to the initial perception region. Set the confidence level of the first pixel in the initial perception region confidence map to the first confidence level, and retain the confidence level of the second pixel to generate a road surface region confidence map corresponding to the road surface region at the first time point. Subtract the second confidence level from the confidence level of the first pixel in the initial perception region confidence map, and set the confidence level of the second pixel to the third confidence level to generate the non-road region confidence map corresponding to the non-road region at the first time.

[0119] In one embodiment, the second sensing region determining unit is specifically used for: Based on the yaw rate of the vehicle, the first fusion coefficient and the second fusion coefficient are determined; Based on the road surface area, the first fusion coefficient, the non-road surface area, the second fusion coefficient, and the fifth fusion sensing area, the third fusion sensing area at the first moment is determined.

[0120] In one embodiment, the third sensing region determining unit is specifically used for: Based on the third fused sensing region, the sensing region mask at the first moment is determined; Based on the initial sensing region and the sixth fused sensing region, the seventh fused sensing region at the first moment is determined; Based on the sensing region mask and the seventh fused sensing region, the fourth fused sensing region at the first moment is determined.

[0121] In one embodiment, the third sensing region determining unit is specifically used for: Identify the third pixel point with a confidence level greater than a preset second confidence threshold and the fourth pixel point with a confidence level less than or equal to the second confidence threshold in the confidence map of the third fusion sensing region; Determine the fifth pixel among the fourth pixel points that satisfies the preset proximity condition with the third pixel point; The perceptual region masks of the third and fifth pixels are set as reserved masks, and the perceptual region masks of all pixels other than the third and fifth pixels are set as occlusion masks to obtain the perceptual region mask at the first moment.

[0122] In one embodiment, the third sensing region determining unit is specifically used for: The third fusion coefficient is determined based on the vehicle's yaw rate; Based on the initial sensing region, the third fusion coefficient, and the sixth fusion sensing region, the seventh fusion sensing region at the first moment is determined.

[0123] In one embodiment, the second sensing region determination module 1330 includes: The backbone region determination unit is used to determine the backbone region at the first moment based on the third fused sensing region; The fourth sensing region determination unit is used to determine the eighth fusion sensing region at the first moment based on the main region and the fourth fusion sensing region; The fifth sensing region determination unit is used to determine the target sensing region at the first moment based on the eighth fused sensing region.

[0124] The beneficial technical effects corresponding to the exemplary embodiments of this device can be found in the corresponding beneficial technical effects of the exemplary method section above, and will not be repeated here.

[0125] Exemplary electronic devices Figure 14 A structural diagram of an electronic device provided in an embodiment of this disclosure includes at least one processor 11 and a memory 12.

[0126] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0127] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute one or more computer program instructions to implement the methods for determining the driving area and / or other desired functions of the various embodiments of this disclosure described above.

[0128] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0129] The input device 13 may also include, for example, a keyboard, a mouse, etc.

[0130] The output device 14 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0131] Of course, for the sake of simplicity, Figure 14 Only some of the components of the electronic device 10 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0132] Exemplary computer program products and computer-readable storage media In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, cause the processor to perform the steps in the methods for determining the driving area of ​​the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0133] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0134] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the methods for determining the driving area of ​​the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0135] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0136] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0137] Various modifications and variations can be made to this disclosure without departing from its spirit and scope. Therefore, this disclosure is also intended to include such modifications and variations if they fall within the scope of the claims of this disclosure and their equivalents.

Claims

1. A means for determining a driving area, comprising one or more processors, said one or more processors being configured to: Obtain the initial perception region at the first moment, and the first fused perception region and the second fused perception region at the second moment; wherein, The second moment is earlier than the first moment; Based on the initial sensing region, the first fused sensing region, and the second fused sensing region, the third fused sensing region and the fourth fused sensing region at the first moment are determined. Based on the third fusion sensing region and the fourth fusion sensing region, the target sensing region at the first moment is determined; Based on the target perception area, the driving area at the first moment is determined.

2. The apparatus according to claim 1, wherein, The step of determining the third and fourth fusion sensing regions at the first moment based on the initial sensing region, the first fusion sensing region, and the second fusion sensing region includes: Based on the vehicle's motion state data, the first fused perception region, and the second fused perception region, the fifth fused perception region and the sixth fused perception region at the first moment are determined. Based on the initial sensing region and the fifth fused sensing region, the third fused sensing region at the first moment is determined; Based on the initial sensing region, the third fusion sensing region, and the sixth fusion sensing region, the fourth fusion sensing region at the first moment is determined.

3. The apparatus according to claim 2, wherein, Determining the third fusion sensing region at the first moment based on the initial sensing region and the fifth fusion sensing region includes: Based on the initial sensing area, the road surface area and non-road surface area at the first moment are generated; Based on the road surface area, the non-road surface area, and the fifth fusion sensing area, the third fusion sensing area at the first moment is determined.

4. The apparatus according to claim 3, wherein, The step of generating the road surface area and non-road surface area at the first moment based on the initial sensing area includes: Identify the first pixel point with a confidence level less than the first confidence threshold and the second pixel point with a confidence level greater than or equal to the first confidence threshold in the confidence map of the initial perception region corresponding to the initial perception region. Set the confidence level of the first pixel in the initial perception region confidence map to the first confidence level, and retain the confidence level of the second pixel to generate a road surface region confidence map corresponding to the road surface region at the first time point. Subtract the second confidence level from the confidence level of the first pixel in the initial perception region confidence map, and set the confidence level of the second pixel to the third confidence level to generate the non-road region confidence map corresponding to the non-road region at the first time.

5. The apparatus according to claim 3 or 4, wherein, The step of determining the third fusion sensing region at the first moment based on the road surface region, the non-road surface region, and the fifth fusion sensing region includes: Based on the yaw rate of the vehicle, the first fusion coefficient and the second fusion coefficient are determined; Based on the road surface area, the first fusion coefficient, the non-road surface area, the second fusion coefficient, and the fifth fusion sensing area, the third fusion sensing area at the first moment is determined.

6. The apparatus according to claim 2, wherein, Determining the fourth fusion sensing region at the first moment based on the initial sensing region, the third fusion sensing region, and the sixth fusion sensing region includes: Based on the third fused sensing region, the sensing region mask at the first moment is determined; Based on the initial sensing region and the sixth fused sensing region, the seventh fused sensing region at the first moment is determined; Based on the sensing region mask and the seventh fused sensing region, the fourth fused sensing region at the first moment is determined.

7. The apparatus according to claim 6, wherein, Determining the sensing region mask at the first moment based on the third fused sensing region includes: Identify the third pixel point with a confidence level greater than a preset second confidence threshold and the fourth pixel point with a confidence level less than or equal to the second confidence threshold in the confidence map of the third fusion sensing region; Determine the fifth pixel among the fourth pixel points that satisfies the preset proximity condition with the third pixel point; The perceptual region masks of the third and fifth pixels are set as reserved masks, and the perceptual region masks of all pixels other than the third and fifth pixels are set as occlusion masks to obtain the perceptual region mask at the first moment.

8. The apparatus according to claim 6 or 7, wherein, Determining the seventh fusion sensing region at the first moment based on the initial sensing region and the sixth fusion sensing region includes: The third fusion coefficient is determined based on the vehicle's yaw rate; Based on the initial sensing region, the third fusion coefficient, and the sixth fusion sensing region, the seventh fusion sensing region at the first moment is determined.

9. The apparatus according to claim 1, wherein, Determining the target sensing region at the first moment based on the third fused sensing region and the fourth fused sensing region includes: Based on the third fused sensing region, the main region at the first moment is determined; Based on the main region and the fourth fusion sensing region, the eighth fusion sensing region at the first moment is determined. Based on the eighth fusion sensing region, the target sensing region at the first moment is determined.

10. A method for determining a driving area, comprising: The initial perception region at a first moment, the first fused perception region at a second moment, and the second fused perception region at a second moment are obtained; wherein the second moment is earlier than the first moment. Based on the initial sensing region, the first fused sensing region, and the second fused sensing region, the third fused sensing region and the fourth fused sensing region at the first moment are determined. Based on the third fusion sensing region and the fourth fusion sensing region, the target sensing region at the first moment is determined; Based on the target perception area, the driving area at the first moment is determined.

11. The method according to claim 10, wherein, The step of determining the third and fourth fusion sensing regions at the first moment based on the initial sensing region, the first fusion sensing region, and the second fusion sensing region includes: Based on the vehicle's motion state data, the first fused perception region, and the second fused perception region, the fifth fused perception region and the sixth fused perception region at the first moment are determined. Based on the initial sensing region and the fifth fused sensing region, the third fused sensing region at the first moment is determined; Based on the initial sensing region, the third fusion sensing region, and the sixth fusion sensing region, the fourth fusion sensing region at the first moment is determined.

12. The method according to claim 11, wherein, Determining the third fusion sensing region at the first moment based on the initial sensing region and the fifth fusion sensing region includes: Based on the initial sensing area, the road surface area and non-road surface area at the first moment are generated; Based on the road surface area, the non-road surface area, and the fifth fusion sensing area, the third fusion sensing area at the first moment is determined.

13. The method according to claim 11, wherein, Determining the fourth fusion sensing region at the first moment based on the initial sensing region, the third fusion sensing region, and the sixth fusion sensing region includes: Based on the third fused sensing region, the sensing region mask at the first moment is determined; Based on the initial sensing region and the sixth fused sensing region, the seventh fused sensing region at the first moment is determined; Based on the sensing region mask and the seventh fused sensing region, the fourth fused sensing region at the first moment is determined.

14. The method of claim 10, wherein, Determining the target sensing region at the first moment based on the third fused sensing region and the fourth fused sensing region includes: Based on the third fused sensing region, the main region at the first moment is determined; Based on the main region and the fourth fusion sensing region, the eighth fusion sensing region at the first moment is determined. Based on the eighth fusion sensing region, the target sensing region at the first moment is determined.

15. An electronic device, the electronic device comprising: The device for determining the driving area as described in any one of claims 1 to 9.

16. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method for determining the driving area as described in any one of claims 10 to 14.

17. A computer-readable storage medium storing a computer program that is executed by a processor to perform the method for determining a driving area as described in any one of claims 10 to 14.

Images