Method and device for determining perception level of roadside perception system

Abstract

The embodiment of the invention provides a method and device for determining the perception level of a roadside perception system. When the perception level is determined, obtaining perception capability influence data in a sensing visual field range of the roadside perception system in real time; and determining the real-time perception level of the roadside perception system according to the obtained perception capability influence data and the nominal perception level of the roadside perception system. By utilizing the method, the real-time sensing level of the roadside sensing system can be adjusted according to the sensing capability influence data acquired in real time, so that the sensing data sensed by the roadside sensing system is ensured to be applied to a proper application scene during downstream application, and an application decision error caused by insufficient sensing level is avoided.

Description

Technical Field

[0001] The embodiments described in this specification generally relate to the field of vehicle-road cooperation, and more specifically, to methods and apparatus for determining the perception level of a roadside perception system. Background Technology

[0002] Roadside perception technology is a core technology in the field of vehicle-road cooperation. During roadside perception, roadside sensors such as cameras, millimeter-wave radar, and lidar are used to collect road traffic data from the vehicle's surroundings and at a greater distance ahead. Edge computing is then performed based on roadside edge computing units to identify and track static and / or dynamic targets, thereby achieving instantaneous intelligent perception of road traffic information such as traffic participants and road conditions.

[0003] When deploying roadside perception systems, different perception levels can be assigned based on technical requirements, such as Perception Level 1 (SL1), Perception Level 2 (SL2), and Perception Level 3 (SL3). Data sensed at different perception levels can be used by different levels of applications. For example, data sensed at Perception Level 1 can be used for data statistics and platform management applications. Data sensed at Perception Level 2 can be used for driver assistance applications to help drivers make on-site driving decisions. Data sensed at Perception Level SL3 can be used by autonomous driving systems to make autonomous driving decisions.

[0004] However, the sensing capabilities of roadside sensing systems can be affected by factors such as road conditions and the condition of the roadside sensing devices, often resulting in the sensing capabilities not reaching the nominal sensing level. In such cases, using the sensed data according to the nominal sensing level would affect the accuracy of subsequent application decisions using the data. Therefore, it is necessary to determine the sensing level of the roadside sensing system in real time to decide whether the sensed data needs to be downgraded for use. Summary of the Invention

[0005] The following brief introduction is provided to present some of the selected concepts in a simplified manner, which will be further described in the detailed description that follows. This brief introduction is not intended to highlight the key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0006] According to one aspect of an embodiment of this specification, a method for determining the perception level of a roadside perception system is provided, comprising: acquiring in real time perception capability impact data within the sensing field of view of the roadside perception system; and determining the real-time perception level of the roadside perception system based on the perception capability impact data and the nominal perception level of the roadside perception system.

[0007] According to another aspect of the embodiments of this specification, an apparatus for determining the perception level of a roadside perception system is provided, comprising: a perception capability influence data acquisition unit configured to acquire perception capability influence data within the sensing field of view of the roadside perception system in real time; and a perception level determination unit configured to determine the real-time perception level of the roadside perception system based on the perception capability influence data and the nominal perception level of the roadside perception system.

[0008] According to another aspect of the embodiments of this specification, a computing device is provided, comprising: one or more processors; and a memory storing computer-executable instructions that, when executed, cause the one or more processors to perform a method as described above for determining the perception level of a roadside perception system.

[0009] According to another aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores executable instructions, which, when executed, cause a processor to perform the method described above for determining the perception level of a roadside perception system.

[0010] According to another aspect of the embodiments of this specification, a computer program product is provided, including a computer program that is executed by a processor to implement the method described above for determining the perception level of a roadside perception system.

[0011] The method for determining the perception level of a roadside perception system according to embodiments of this specification acquires perception capability impact data within the sensing field of view of the roadside perception system in real time, and determines the real-time perception level of the roadside perception system based on the perception capability impact data and the nominal perception level of the roadside perception system. The real-time perception level of the roadside perception system can be adjusted according to the real-time acquired perception capability impact data, thereby ensuring that the sensing data sensed by the roadside perception system is applied to suitable application scenarios in downstream applications, and avoiding application decision errors due to insufficient perception level. Attached Figure Description

[0012] A further understanding of the nature and advantages of the embodiments described in this specification can be achieved by referring to the following accompanying drawings. In the drawings, similar components or features may have the same reference numerals.

[0013] Figure 1 A schematic diagram of an example architecture for a roadside perception system is shown.

[0014] Figure 2 An example flowchart of a method for determining the perception level of a roadside perception system according to an embodiment of this specification is shown.

[0015] Figure 3An example flowchart of a real-time perception level determination process according to an embodiment of this specification is shown.

[0016] Figure 4 An example schematic diagram of the sensing field of view after region division according to an embodiment of this specification is shown.

[0017] Figure 5 An example flowchart of the process for determining the first perception ability influence factor according to an embodiment of this specification is shown.

[0018] Figure 6 An example block diagram of a perception level determination device according to an embodiment of this specification is shown.

[0019] Figure 7 A block diagram of a perception level determination apparatus implemented using a computing device according to an embodiment of this specification is shown. Detailed Implementation

[0020] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed merely to enable those skilled in the art to better understand and implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. The function and arrangement of the elements discussed may be changed without departing from the scope of this disclosure. Various processes or components may be omitted, substituted, or added as needed in the various examples. For example, the described methods may be performed in a different order than described, and steps may be added, omitted, or combined. Furthermore, features described in some examples may be combined in other examples.

[0021] As used herein, the term "comprising" and its variations are open terms meaning "including but not limited to". The term "based on" means "at least partially based on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other definitions, whether explicit or implicit, may be included below. Unless explicitly indicated by the context, the definition of a term shall remain consistent throughout the specification.

[0022] Figure 1 A schematic diagram of an example architecture of a roadside perception system 100 is shown.

[0023] like Figure 1As shown, the roadside perception system 100 includes roadside sensors, an edge computing device 140, a roadside unit (RSU) 150, and an on-board unit (OBU) 160. Roadside sensors may include, for example, a camera 110, a millimeter-wave radar 120, and a lidar 130.

[0024] Camera 110, millimeter-wave radar 120, lidar 130, edge computing device 140, roadside device 150, and vehicle-mounted device 160 can communicate with each other via network 170 to transmit data. In some embodiments, network 170 can be any one or more of wired or wireless networks. Examples of network 170 may include, but are not limited to, cable networks, fiber optic networks, telecommunications networks, enterprise intranets, the Internet, local area networks (LANs), wide area networks (WANs), wireless local area networks (WLANs), metropolitan area networks (MANs), public switched telephone networks (PSTNs), Bluetooth networks, ZigZee networks, near field communication (NFC), device bus, device wiring, etc., or any combination thereof. In some embodiments, network 170 can be implemented as a vehicle-to-everything (V2X) network. In some embodiments, some or all of the components among camera 110, millimeter-wave radar 120, lidar 130, edge computing device 140, roadside device 150, and vehicle-mounted device 160 can communicate directly without network 170.

[0025] Roadside sensors are deployed on the roadside to sense road traffic information within the field of view of the roadside perception system. For example, camera 110 can sense information such as the type of traffic participants (e.g., pedestrians, vehicles, cyclists), millimeter-wave radar 120 can sense the position and speed of targets, and lidar 130 can sense the position, speed, and size of stationary and moving targets. The data sensed by camera 110 is image data, which is greatly affected by weather and light intensity. The data sensed by millimeter-wave radar 120 is point cloud data, which is not affected by weather conditions, but it is prone to missing stationary targets and has a lot of noise in the detected targets. The data sensed by lidar 130 is point cloud data, which is highly sensitive to the environment and is easily affected by heavy snow, dust, etc.

[0026] Edge computing device 140 acquires raw sensing data from camera 110, millimeter-wave radar 120, and lidar 130 via interfaces such as CAN and Ethernet, and simultaneously receives connected vehicle information data collected by RSU 150 via air interface. Edge computing device 140 is equipped with data fusion computing software. After receiving data from roadside sensors, edge computing device 140 uses the data fusion computing software to perform data fusion calculations on the received data to calculate information about traffic participants on the entire road. For example, after receiving raw sensing data (point cloud data, image data, etc.), target recognition and target localization can be performed on the raw sensing data. Target recognition refers to obtaining the target type through machine learning or deep learning algorithms; and target localization refers to calibrating targets within the sensing area using specific algorithms to obtain the transformation relationship between the detected target's coordinates in the sensor's relative coordinate system and its coordinates in the geodetic coordinate system. Then, the obtained target feature information is synchronized, associated with, matched with, estimated in state, and tracked to obtain precise information such as the location, speed, type, and size of traffic participants under different weather and lighting conditions. This information is then sent to the RSU 150 through agreed interfaces and protocols. After acquiring the information of all traffic participants fused by the edge computing device 140, the RSU 150 sends the traffic participant information within its sensing field of view to the OBU units on surrounding intelligent connected vehicles through a network such as a vehicle network, for intelligent decision-making by the intelligent connected vehicles. In some embodiments, the edge computing device 140 can be deployed in an edge cloud.

[0027] To enable roadside perception systems to serve different application scenarios, they can be classified based on their sensing capabilities. For example, they can be classified into Perception Level 1 (SL1), Perception Level 2 (SL2), and Perception Level 3 (SL3) based on technical specifications. SL1 refers to the perception capability level for platform data applications; the technical specifications of a roadside perception system meeting SL1 should support data statistics and platform management applications. SL2 refers to the perception capability level for driver assistance applications; the technical specifications of a roadside perception system meeting SL2 should support applications that provide real-time road information to drivers to assist them in making on-site driving decisions. SL3 refers to the perception capability level for autonomous driving applications; the technical specifications of a roadside perception system meeting SL3 should enable it to function as an external sensor, providing valuable real-time road information to autonomous driving systems and supporting autonomous driving decision-making applications. The technical specifications for perception levels will change in different application scenarios. Table 1 shows the positioning accuracy classification requirements for the traffic participant type determination scenario.

[0028]

[0029] Table 1

[0030] The sensing capability used for sensing level classification can also be referred to as the sensing performance of a roadside sensing system. Sensing capability can include, for example, precision and recall. Precision is the percentage of correctly identified positive targets (TP) to the total number of positive targets actually identified (TP + incorrectly identified positive targets (FP)), i.e., Precision = TP / (TP + FP). Recall is the percentage of correctly identified positive targets (TP) to the total number of correctly identified targets (TP + incorrectly identified negative targets (FN)), i.e., Recall = TP / (TP + FN).

[0031] However, the sensing capabilities of roadside perception systems can be affected by factors such as road conditions and the condition of the roadside sensing devices, often resulting in the system's sensing capabilities falling short of its nominal capabilities. In such cases, using the sensed data according to the nominal sensing level would negatively impact the accuracy of subsequent decision-making in applications utilizing that data.

[0032] In view of this, embodiments of this specification propose a perception level determination scheme. In this scheme, current perception capability impact data within the sensing field of view of the roadside perception system is acquired, and the real-time perception level of the roadside perception system is determined based on the current perception capability impact data and the nominal perception level of the roadside perception system. According to this perception level determination method, the real-time perception level of the roadside perception system can be adjusted based on the real-time acquired perception capability impact data, thereby ensuring that the real-time sensing data sensed by the roadside perception system is applied to appropriate application scenarios in downstream applications, avoiding application decision errors due to insufficient perception level.

[0033] The perception level determination method and perception level determination apparatus according to embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0034] Figure 2 An example flowchart of a method 200 for determining the perception level of a roadside perception system according to an embodiment of this specification is shown.

[0035] like Figure 2 As shown, in S210, real-time acquisition of perception capability impact data within the sensing field of view of the roadside perception system is performed. In this specification, the term "perception capability impact data" refers to data that can affect the sensing capability of the roadside perception system (e.g., degrade its performance). In some embodiments, perception capability impact data may include at least one of current road environment data, raw sensing data, and data processing results.

[0036] Current road environment data may include, for example, weather data and light intensity. Weather data may include, for example, road rainfall data and fog level data. Road environment data includes road environment data sensed by onboard environmental sensors, road environment data sensed by roadside environmental sensors, and / or road environment data acquired from external sources via vehicle-to-everything (V2X) networks. Examples of environmental sensors may include, but are not limited to, rain sensors and fog sensors. The sensing capability of roadside sensors in a roadside perception system varies under different road conditions. For example, compared to sunny days, the sensing capability of roadside sensors gradually decreases in light, moderate, or heavy rain. Therefore, road environment data affects the sensing capability of a roadside perception system.

[0037] Raw sensing data refers to the real-time sensing data obtained by roadside sensors within their field of view, such as image data captured by cameras, and point cloud data captured by millimeter-wave radar and lidar. When roadside sensors are damaged, for example, if a camera is covered in dust or oil, the quality of the data sensed by the roadside sensors will degrade. For instance, the image captured by the camera may be unclear or have poor resolution. Sensing processing based on this degraded raw sensing data will also result in a deterioration in the perception results, thus reducing the perception capability of the roadside perception system.

[0038] Data processing results refer to the results obtained by the roadside perception system using its built-in perception data processing software to process the raw sensing data. In some embodiments, data processing results may include object recognition results. For example, if multiple objects are identified in the target processing area, and at least two of these objects are occluded, the degree of occlusion between the objects will affect the perception capability of the roadside perception system. Identified objects may include, for example, people, roadside structures such as flower beds, etc.

[0039] In S220, the real-time perception level of the roadside perception system is determined based on the acquired perception capability impact data and the nominal perception level of the roadside perception system.

[0040] In some embodiments, a perception level determination model can be pre-trained using a training dataset. Each training data point in the dataset includes perception capability impact data, the nominal perception level of the roadside perception system, and label data. Here, the label data refers to the real-time perception level output by the perception level determination model. After training the perception level determination model, the perception capability impact data and the nominal perception level of the roadside perception system are provided to the model to determine the real-time perception level.

[0041] In some embodiments, the equipment composition information of the roadside sensing devices in the roadside sensing system can also be obtained. For example, the roadside sensing system consists of several types of roadside sensing devices, the number of devices of each type, and the sensing characteristics of each type. Here, the sensing characteristics can reflect, for example, the impact of various sensing capability influence data on the sensing capability of the roadside sensing devices. For example, cameras are greatly affected by weather and light intensity. In some embodiments, the sensing characteristics can quantitatively reflect the degree of influence of various sensing capability influence data on the sensing capability of the roadside sensing devices. For example, a functional relationship curve can be fitted between the data values ​​of sensing capability influence data and their degree of influence on the sensing capability of the roadside sensing devices.

[0042] In this scenario, a perception level determination model can be pre-trained using a training dataset. Each training data point in the dataset includes perception capability impact data, equipment composition information of the roadside perception system's roadside sensing devices, the nominal perception level of the roadside perception system, and label data. Here, the label data refers to the real-time perception level output by the perception level determination model. After training the perception level determination model, the perception capability impact data, the equipment composition information of the roadside perception system's roadside sensing devices, and the nominal perception level of the roadside perception system are provided to the model to determine the real-time perception level.

[0043] In some embodiments, the impact of perception capability on the roadside perception system can be quantitatively determined based on perception capability impact data, and the perception level of the roadside perception system can be determined in real time based on the perception capability impact.

[0044] Figure 3 An example flowchart of a real-time perception level determination process 300 according to an embodiment of this specification is shown.

[0045] like Figure 3 As shown, in S310, the perception capability impact factor of the roadside perception system is determined based on the perception capability impact data.

[0046] When the data influencing perception capability includes road environment data, a first perception capability influence factor for the roadside perception system can be determined based on the road environment data. This first perception capability influence factor reflects the degree of influence of the road environment on perception capability. Different road environment data will have different impacts on the perception capability of the roadside perception system. For example, when the road environment data is rainfall data, different rainfall amounts will have different impacts on the perception capability of the roadside perception system. In some embodiments, rainfall data can be quantized based on hourly rainfall. Different quantization results correspond to different first perception capability influence factors. For example, hourly rainfall less than 2.5 mm can be quantized as light rain, hourly rainfall of not less than 2.5 mm and less than 7.5 mm can be quantized as moderate rain, and hourly rainfall of not less than 7.5 mm can be quantized as heavy rain. Furthermore, light rain, moderate rain, and heavy rain can each be assigned a real value within a given data range (e.g., between 0 and 1). The real value obtained after quantization is used as the first perception capability influence factor. In other embodiments, rainfall data can also be quantized in other suitable ways to obtain a real value within a given data range, which is then used as the first perception capability influence factor. In addition, other types of road environment data can also be quantified using appropriate quantification methods, and the first perception ability influencing factor can be determined based on the quantification results.

[0047] Furthermore, the perception capabilities of different types of roadside sensing devices may vary under the same road environment data. In some embodiments, a first perception capability influence factor for the roadside sensing system can be determined based on road environment data and the device type of the roadside sensing devices. In some embodiments, the roadside sensing system may include at least two types of roadside sensing devices. In this case, for each type of roadside sensing device, a first perception capability influence factor can be determined based on road environment data and the device type of that roadside sensing device. Subsequently, the first perception capability influence factors of various roadside sensing devices are factor-integrated to obtain the first perception capability influence factor of the roadside sensing system. For example, different weighting factors can be assigned to various roadside sensing devices based on the device composition of the roadside sensing system, and then the first perception capability influence factors of various roadside sensing devices can be weighted and summed to obtain the first perception capability influence factor of the roadside sensing system.

[0048] When the perception capability impact data includes raw sensing data, a second perception capability impact factor for the roadside perception system can be determined based on the raw sensing data. Since roadside perception devices exhibit different perception capabilities under different hardware states, they sense raw sensing data of varying quality. By performing data quality analysis on the raw sensing data, the real-time hardware state of the roadside perception devices can be analyzed, and a second perception capability impact factor reflecting the influence of the hardware state of the roadside perception devices on the perception capability can be determined based on the real-time hardware state of the roadside perception devices during real-time sensing. For example, the perception capability of roadside perception devices will deteriorate under conditions of device damage, contamination (e.g., dust or oil stains), or adverse external sensing conditions (e.g., excessively high or low ambient temperatures).

[0049] In some embodiments, the raw sensing data can be assessed for data quality based on a given data quality evaluation method, and a second sensing capability influencing factor of the roadside perception system can be determined based on the data quality assessment results. For example, a data quality assessment dimension can be selected, and a data quality assessment score can be calculated based on a given data quality assessment function. The calculated data quality assessment score serves as the second sensing capability influencing factor. In some embodiments, the data quality assessment score can be normalized to a real value between 0 and 1. In some embodiments, the selected data quality assessment dimension can be different for different types of roadside perception devices. When the roadside perception system includes multiple types of roadside perception devices, the data quality assessment dimension can be determined based on the device type composition of the roadside perception devices in the roadside perception system.

[0050] In some embodiments, such as for image data captured by a camera, the selected data quality assessment dimensions for evaluating sensing data may include, for example, image distortion coefficients, spot parameters, peak signal-to-noise ratio (PSNR), etc. Image distortion coefficients may include, for example, radial distortion coefficients, tangential distortion coefficients, etc. For example, the degree of distortion can be quantified by calculating the magnitude of the image distortion coefficients to determine the multi-order distortion. The larger the absolute value of the image distortion coefficient, the more severe the image distortion. Spot parameters may include, for example, spot intensity, spot size, and spot shape. For example, the spot intensity of an image can be quantified by measuring the pixel intensity of a specific region in the image. Spot size and shape can be characterized as the size and shape of a specific spot pixel region in the image (e.g., circular, elliptical, etc.). The peak signal-to-noise ratio of the image can be used to measure the reconstruction quality of the image. For point cloud data acquired by millimeter-wave radar and lidar, the selected data quality assessment dimensions for evaluating data quality may include, for example, point cloud density, point cloud uniformity, point cloud accuracy, etc. Point cloud density is used to represent the number of points per unit area or volume. Point cloud density can be used to assess the level of detail in the data. Point cloud uniformity reflects whether the distribution of points in a point cloud is uniform. It can be used to assess the likelihood of ghosting. Point cloud accuracy can be measured using the root mean square error (RMSE).

[0051] When perception capability influences data, including data processing results, a third perception capability influencing factor can be determined based on these results. Because the processing capabilities of the data processing software installed in roadside perception systems differ, the resulting data processing will also vary. Different data processing results lead to different final perception outcomes, thus affecting the perception capability of the roadside perception system. In this case, the third perception capability influencing factor determined based on the data processing results can reflect the degree of impact of the roadside perception system's software processing capabilities on its perception capability.

[0052] In some embodiments, the data processing results include object recognition results. In this case, the occlusion degree between the identified objects can be determined based on the main boundary information (e.g., object length, width, and height) and position information of the identified objects, and the obtained object occlusion degree is used as a third perception capability influencing factor.

[0053] In S320, the real-time perception level of the roadside perception system is determined based on the determined perception capability influencing factors and the nominal perception level of the roadside perception system.

[0054] In some embodiments, when the determined perception capability influencing factors include at least two of the first perception capability influencing factor, the second perception capability influencing factor, and the third perception capability influencing factor, factor fusion can be performed on the determined perception capability influencing factors to obtain the perception capability influencing factors of the roadside perception system.

[0055] For example, based on the equipment composition of the roadside sensing devices in the roadside sensing system (e.g., equipment type composition and / or the number of devices of each type), different weighting factors can be assigned to the first sensing capability influence factor, the second sensing capability influence factor, and the third sensing capability influence factor. Then, the first sensing capability influence factor, the second sensing capability influence factor, and the third sensing capability influence factor can be weighted and summed to obtain the sensing capability influence factor of the roadside sensing system.

[0056] After obtaining the perception capability impact factor of the roadside perception system, the real-time perception level of the roadside perception system can be determined based on the perception capability impact factor and the nominal perception level of the roadside perception system. For example, assuming the nominal perception level of the roadside perception system is N (e.g., SL3), N-1 (2) thresholds can be set, such as 0.4 and 0.7. Different thresholds correspond to different perception levels. When the perception capability impact factor of the roadside perception system reaches a certain threshold, the perception level of the roadside perception system is downgraded to the perception level corresponding to that threshold. For example, when the perception capability impact factor of the roadside perception system is no greater than 0.4, SL3 remains unchanged. When the perception capability impact factor of the roadside perception system is greater than 0.4 but no greater than 0.7, the perception level is downgraded to SL2. When the perception capability impact factor of the roadside perception system is greater than 0.7, the perception level is downgraded to SL1.

[0057] In some embodiments, a perception level determination model can be trained using a training dataset. Each training data point in the training dataset includes a perception capability influencing factor, the nominal perception level of the roadside perception system, and label data. After training the perception level determination model, the perception capability influencing factor and the nominal perception level of the roadside perception system are provided to the model to determine the real-time perception level. Here, in the example of integrating the perception capability influencing factor, the input perception capability influencing factor to the perception level determination model is the integrated perception capability influencing factor. In some embodiments, the integration of the perception capability influencing factor may not be performed. In this case, multiple unintegrated perception capability influencing factors can be provided as model inputs to the perception level determination model for perception level determination.

[0058] Since the road environment, the real-time device status of the roadside sensing equipment, and / or the data processing capabilities of the roadside sensing system's data processing software affect the real-time sensing level of the roadside sensing system, in the above embodiments, by analyzing the road environment data, raw sensing data, and / or data processing results respectively, the influence of the road environment data, raw sensing data, and / or data processing results on the roadside sensing capability of the roadside sensing system is determined. Incorporating the influence of the road environment data, raw sensing data, and / or data processing results on the roadside sensing capability of the roadside sensing system when determining the real-time sensing level allows for real-time adjustment of the roadside sensing system's sensing level based on the actual sensing conditions. This ensures that the real-time sensing data sensed by the roadside sensing system is applied to suitable application scenarios in downstream applications, avoiding application decision errors due to insufficient sensing level.

[0059] In some embodiments, the equipment composition information of the roadside sensing devices in the roadside sensing system can also be obtained. For example, the roadside sensing system consists of several types of roadside sensing devices, the number of devices of each type, and the sensing characteristics of each type of roadside sensing device. In this case, a sensing level determination model can be pre-trained using a training dataset. Each training data point in the training dataset includes a sensing capability influencing factor, equipment composition information of the roadside sensing devices in the roadside sensing system, the nominal sensing level of the roadside sensing system, and label data. After the sensing level determination model is trained, the sensing capability influencing factor, the equipment composition information of the roadside sensing devices in the roadside sensing system, and the nominal sensing level of the roadside sensing system are provided to the sensing level determination model to determine the real-time sensing level.

[0060] In some embodiments, the sensing field of view can be divided into regions to obtain at least two divided regions, each of which is covered by some or all of the roadside sensing devices of the roadside sensing system.

[0061] Figure 4 An example schematic diagram of the sensing field of view after region division according to an embodiment of this specification is shown.

[0062] like Figure 4As shown, the sensing field of view of the roadside perception system is the intersection, which is divided into four sub-regions: 410, 420, 430, and 440. Furthermore, the roadside perception system includes a camera 450, a lidar sensor 460, a camera 470, a lidar sensor 480, and a lidar sensor 490. Camera 450 and lidar sensor 460 cover sub-region 410, lidar sensor 490 covers sub-region 420, and camera 470 and lidar sensor 480 cover sub-region 440. It should be noted that sub-region 430 can also be covered by the roadside perception device. As shown, sub-regions 410, 420, 430, and 440 can consist of 12 grid areas. In some embodiments, sub-regions 410, 420, 430, and 440 can be further subdivided to obtain smaller sub-regions; for example, sub-region 410 can be further subdivided to obtain sub-regions 411, 413, and 415, each consisting of four grids.

[0063] In this scenario, real-time perception levels can be determined for each defined area. In some embodiments, for each defined area, perception capability impact data is determined, and the real-time perception level of the roadside perception system within that area is determined based on the perception capability impact data and the nominal perception level of the roadside perception system. In some embodiments, for each defined area, perception capability impact data is determined, and a perception capability impact factor is determined based on the perception capability impact data. Then, the real-time perception level of the roadside perception system within that area is determined based on the perception capability impact factor and the nominal perception level of the roadside perception system.

[0064] In some embodiments, the roadside sensing system includes at least two types of roadside sensing devices, and each segmented area is covered by different roadside sensing devices. In this case, for each segmented area, the real-time sensing level of the roadside sensing system within that segmented area can be determined based on the sensing capability influence factor corresponding to that segmented area, the device sensing characteristics of the roadside sensing devices covering that segmented area, and the nominal sensing level of the roadside sensing system.

[0065] For example, a perception level determination model can be trained using a training dataset. Each training data point in the dataset includes perception capability influencing factors within a defined area, the device perception characteristics of roadside sensing equipment covering that area, the nominal perception level of the roadside sensing system, and label data. After training the perception level determination model, the perception capability influencing factors within the defined area, the device perception characteristics of roadside sensing equipment covering that area, and the nominal perception level of the roadside sensing system are provided to the model to determine the real-time perception level of the roadside sensing system within that defined area.

[0066] In some embodiments, the roadside perception system includes at least two types of roadside perception devices, and each segmented area is covered by different roadside perception devices. In some examples, for each segmented area, a first perception capability influence factor of the roadside perception system in that segmented area can be determined based on the road environment data of that segmented area and the device perception environment attributes of the roadside perception devices covering that segmented area. The device perception environment attributes can quantitatively reflect the degree of influence of road environment data on the perception capability of the roadside perception devices. In some examples, the segmented area is covered by different types of roadside perception devices. In this case, a first perception capability influence factor of each type of roadside perception device can be determined based on the road environment data of that segmented area, and then the first perception capability influence factors of the various types of roadside perception devices can be fused to obtain the first perception capability influence factor of the roadside perception system in that segmented area.

[0067] Figure 5 An example flowchart of a first perception ability influence factor determination process 500 according to an embodiment of this specification is shown. Figure 5 The example shown is used to determine the first sensing capability influence factor of a roadside sensing system within a defined area.

[0068] like Figure 5 As shown, in step S510, road environment data within the defined area is acquired. This road environment data can be acquired, for example, through a road environment data acquisition device deployed on the intelligent vehicle (e.g., a rain sensor), or from an external road environment data acquisition device via a vehicle-to-everything (V2X) network.

[0069] In some embodiments, the defined region may include multiple grid regions. When acquiring road environment data within the defined region, road environment data from each grid region can be acquired, and then the acquired road environment data can be fused to obtain the road environment data for the defined region.

[0070] In S520, the device composition information of the roadside sensing devices covering the defined area is determined. The acquired device composition information may include, for example, the device types of the roadside sensing devices covering the defined area. Optionally, in some embodiments, the device composition information may also include the number of each type of roadside sensing device.

[0071] In step S530, based on road environment data within the defined area, a first perception capability influence factor for each type of roadside sensing device is determined. In some embodiments, the device perception environment attributes of each type of roadside sensing device can be obtained. Subsequently, based on the road environment data within the defined area and the device perception attributes of each type of roadside sensing device, a first perception capability influence factor for each type of roadside sensing device is determined. In some embodiments, the first perception capability influence factor for each type of roadside sensing device can be determined based on the road environment data within the defined area, using a mapping table of road environment data-device type perception capability influence factors. The mapping table of road environment data-device type perception capability influence factors can be pre-determined based on historical data. In some embodiments, a relationship function (e.g., a relationship curve) between road environment data and device type perception capability influence factors can be fitted based on historical data. Subsequently, the perception capability influence factor relationship function is used to determine the first perception capability influence factor for each type of roadside sensing device based on the road environment data within the defined area.

[0072] In S540, the first perception capability influence factors of various types of roadside sensing devices are fused to obtain the first perception capability influence factor of the roadside sensing system within the defined area. In some embodiments, the first perception capability influence factors of various types of roadside sensing devices can be averaged to obtain the first perception capability influence factor of the roadside sensing system within the defined area. In some embodiments, the first perception capability influence factors of various types of roadside sensing devices can also be weighted and summed to obtain the first perception capability influence factor of the roadside sensing system within the defined area. The weight factor for each type of roadside sensing device can be determined based on the device type of the roadside sensing device. In some embodiments, if the device composition information also includes the number of each type of roadside sensing device, the weight factor for each type of roadside sensing device can be determined based on the device type and the number of that type of roadside sensing device.

[0073] After determining the real-time perception level of the roadside perception system as described above, the subsequent use cases or applications of the sensed data can be determined based on this level. For example, the sensed data obtained by the roadside perception system identifies road traffic data for object A in area A, such as object A's speed, shape, size, and location. The nominal perception level of the roadside perception system is SL3, and the road traffic data for object A is designated for use by the autonomous driving system for autonomous driving control. When the real-time perception level of the roadside perception system is determined to be SL3, the road traffic data for object A can be provided to the vehicle's autonomous driving system for autonomous driving control. When the real-time perception level of the roadside perception system is determined to be SL2, the road traffic data for object A cannot be provided to the autonomous driving system for autonomous driving control. However, since the warning function of the vehicle's warning system requires a perception level of SL2, the road traffic data for object A can be provided to the warning system for warning decisions.

[0074] Figure 6 An example block diagram of a perception level determination device 600 according to an embodiment of this specification is shown. Figure 6 As shown, the perception level determination device 600 includes a perception capability influence data acquisition unit 610 and a perception level determination unit 620.

[0075] The perception capability impact data acquisition unit 610 is configured to acquire perception capability impact data within the sensing field of view of the roadside perception system in real time. The operation of the perception capability impact data acquisition unit 710 can be referenced above. Figure 2 The operation described in S210. In some embodiments, the sensing capability impact data acquisition unit 610 can be implemented to be communicatively connected to various sensing capability impact data acquisition devices, and to acquire sensing capability impact data within the sensing field of view of the roadside sensing system in real time from the various sensing capability impact data acquisition devices. In some embodiments, the sensing capability impact data acquisition unit 610 can be implemented as various sensing capability impact data acquisition devices.

[0076] The perception level determination unit 620 is configured to determine the real-time perception level of the roadside perception system based on the acquired perception capability impact data and the nominal perception level of the roadside perception system. The operation of the perception level determination unit 620 can be referenced above. Figure 2 S210 and Figures 3 to 5 The described operation.

[0077] In some embodiments, the perception level determination device may be deployed in an edge computing device or independently of the edge computing device, for example, independently of the edge computing device in an edge cloud.

[0078] As per the above reference Figures 1 to 6The present specification describes a perception level determination method and a perception level determination apparatus according to embodiments thereof. The perception level determination apparatus described above can be implemented in hardware, software, or a combination of hardware and software.

[0079] Figure 7 A schematic diagram of an example of a perception level determination device 700 implemented using a computing device according to an embodiment of this specification is shown. Figure 7 As shown, the perception level determination device 700 may include at least one processor 710, a memory (e.g., non-volatile memory) 720, a main memory 730, and a communication interface 740, and the at least one processor 710, memory 720, main memory 730, and communication interface 740 are connected together via a bus 760. The at least one processor 710 executes at least one computer-readable instruction (i.e., the elements implemented in software above) stored or encoded in the memory.

[0080] In one embodiment, computer-executable instructions are stored in a memory that, when executed, cause at least one processor 710 to: acquire perception capability impact data within the sensing field of view of the roadside perception system; and determine the real-time perception level of the roadside perception system based on the perception capability impact data and the nominal perception level of the roadside perception system.

[0081] It should be understood that the computer-executable instructions stored in memory, when executed, cause at least one processor 710 to perform the above-described combinations in the various embodiments of this specification. Figures 2-6 The description includes various operations and functions.

[0082] According to one embodiment, a program product, such as a machine-readable medium (e.g., a non-transitory machine-readable medium), is provided. The machine-readable medium may have instructions (i.e., the elements implemented in software as described above), which, when executed by a machine, cause the machine to perform the above-described combinations of the various embodiments of this specification. Figures 2-6 The various operations and functions described. Specifically, a system or apparatus equipped with a readable storage medium storing software program code that implements the functions of any of the embodiments described above, and enabling the computer or processor of the system or apparatus to read and execute the instructions stored in the readable storage medium.

[0083] In this case, the program code itself, which can be read from a readable medium, can perform the functions of any of the above embodiments. Therefore, the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.

[0084] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer or the cloud via a communication network.

[0085] According to one embodiment, a computer program product is provided, the computer program product including a computer program, which, when executed by a processor, causes the processor to perform the above-described combinations of the various embodiments of this specification. Figures 2-6 The description includes various operations and functions.

[0086] Those skilled in the art will understand that the various embodiments disclosed above can be modified and varied without departing from the spirit of the invention. Therefore, the scope of protection of this invention should be defined by the appended claims.

[0087] It should be noted that not all steps and units in the above process and system structure diagrams are mandatory; some steps or units can be omitted according to actual needs. The execution order of each step is not fixed and can be determined as needed. The device structure described in the above embodiments can be a physical structure or a logical structure; that is, some units may be implemented by the same physical entity, or some units may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0088] In the above embodiments, the hardware units or modules can be implemented mechanically or electrically. For example, a hardware unit, module, or processor may include permanent dedicated circuitry or logic (such as a dedicated processor, FPGA, or ASIC) to perform the corresponding operation. The hardware unit or processor may also include programmable logic or circuitry (such as a general-purpose processor or other programmable processor), which can be temporarily configured by software to perform the corresponding operation. The specific implementation method (mechanical, dedicated permanent circuitry, or temporarily configured circuitry) can be determined based on cost and time considerations.

[0089] The specific embodiments described above with reference to the accompanying drawings are exemplary embodiments, but do not represent all embodiments that can be implemented or fall within the scope of the claims. The term "exemplary" as used throughout this specification means "serving as an example, instance, or illustration" and does not imply that it is "preferred" or "advantageous" compared to other embodiments. Specific details are included to provide an understanding of the described techniques. However, these techniques can be practiced without these specific details. In some instances, well-known structures and apparatuses are shown in block diagram form to avoid obscuring the concepts of the described embodiments.

[0090] The foregoing description of this disclosure is provided to enable any person skilled in the art to implement or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein can be applied to other variations without departing from the scope of this disclosure. Therefore, this disclosure is not limited to the examples and designs described herein, but is consistent with the widest scope of the principles and novel features disclosed herein.

Claims

1. A method for determining the perception level of a roadside perception system, comprising: Real-time acquisition of data on the impact of perception capabilities within the sensing field of view of the roadside perception system; as well as Based on the perception capability impact data and the nominal perception level of the roadside perception system, the real-time perception level of the roadside perception system is determined.

2. The method as described in claim 1, wherein, Based on the perception capability impact data and the nominal perception level of the roadside perception system, the real-time perception level of the roadside perception system is determined as follows: Based on the aforementioned perception capability impact data, the perception capability impact factor of the roadside perception system is determined; and The real-time perception level of the roadside perception system is determined based on the perception capability influence factor and the nominal perception level of the roadside perception system.

3. The method as described in claim 1 or 2, wherein, The perception capability impact data includes at least one of current road environment data, raw sensing data, and data processing results. The data processing results are obtained by processing the raw sensing data using the perception data processing software of the roadside perception system.

4. The method of claim 3, wherein, Based on the aforementioned perception capability impact data, the perception capability impact factors of the roadside perception system are determined as follows: Based on the road environment data, a first perception capability influence factor of the roadside perception system is determined. The first perception capability influence factor is used to reflect the degree of influence of the road environment on the perception capability. Based on the original sensing data, a second sensing capability influence factor is determined for the roadside sensing system. This second sensing capability influence factor reflects the degree of influence of the hardware status of the sensing devices in the roadside sensing system on the sensing capability; and / or Based on the data processing results, a third sensing capability influence factor for the roadside perception system is determined. This third sensing capability influence factor reflects the software processing capabilities of the roadside perception system. The degree to which ability influences perception ability.

5. The method of claim 4, wherein, The data processing results include object recognition results.

6. The method of claim 4, wherein, Based on the aforementioned perception capability impact data, determining the perception capability impact factor of the roadside perception system further includes: The perception capability influence factor of the roadside perception system is determined based on the first perception capability influence factor, the second perception capability influence factor, and / or the third perception capability influence factor.

7. The method of claim 2, wherein, The roadside sensing system includes at least two types of roadside sensing devices, the sensing field of view includes at least two divided regions, and the sensing capability influence factor of the roadside sensing system includes the sensing capability influence factor of the roadside sensing system in each divided region. The sensing capability influence factor of the roadside sensing system in each divided region is determined based on the sensing capability influence data in each divided region. Determining the real-time perception level of the roadside perception system based on the perception capability influencing factor and the nominal perception level of the roadside perception system includes: For each defined area, the real-time perception level of the roadside perception system within that area is determined based on the perception capability influence factor corresponding to that area, the device perception characteristics of the roadside perception equipment covering that area, and the nominal perception level.

8. The method of claim 7, wherein, Based on the perception capability influence factor corresponding to the defined area, the device perception characteristics of the roadside perception equipment covering the defined area, and the nominal perception level, the real-time perception level of the roadside perception system within the defined area is determined as follows: The perception capability influence factor corresponding to the divided area, the device perception characteristics of the roadside perception equipment covering the divided area, and the nominal perception level are provided to the perception level determination model to determine the real-time perception level of the roadside perception system in the divided area.

9. The method of claim 4, wherein, The roadside perception system includes at least two types of roadside perception devices, and the sensing field of view includes at least two divided regions. Based on the road environment data, the first perception capability influencing factor of the roadside perception system includes: For each defined area, based on the road environment data of that area and the device perception environment attributes of the roadside perception devices covering that area, the first perception capability influence factor of the roadside perception system in that area is determined.

10. The method of claim 3, wherein, The road environment data includes road environment data sensed by onboard environmental sensors, road environment data sensed by roadside environmental sensors, and / or road environment data obtained from external sources via vehicle-to-everything (V2X) networks.

11. An apparatus for determining the sensing level of a roadside sensing system, comprising: The perception capability impact data acquisition unit is configured to acquire perception capability impact data within the sensing field of view of the roadside perception system in real time. as well as The perception level determination unit is configured to determine the real-time perception level of the roadside perception system based on the perception capability impact data and the nominal perception level of the roadside perception system.

12. A computing device, comprising: One or more processors; as well as A memory storing computer-executable instructions that, when executed, cause the one or more processors to perform the method for determining the perception level of a roadside perception system as described in any one of claims 1 to 10.

13. A computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform a method for determining the perception level of a roadside perception system as described in any one of claims 1 to 10.

14. A computer program product comprising a computer program executed by a processor to implement the method for determining the perception level of a roadside perception system as described in any one of claims 1 to 10.

Images