Method, device, and electronic device for evaluating driving behavior of an autonomous vehicle

Abstract

The present disclosure provides a method, apparatus, electronic device, and medium for evaluating the driving behavior of an autonomous vehicle, relating to the field of autonomous driving, and more particularly, to the testing and evaluation of autonomous vehicles. A method for evaluating the driving behavior of an autonomous vehicle includes: obtaining driving data collected by the vehicle during autonomous driving; determining the driving behavior of the vehicle and at least one indicator value based on the driving data, wherein the at least one indicator value is used to characterize the determined driving behavior; determining a scene complexity associated with the driving behavior; and determining an evaluation result for the driving behavior based on the at least one indicator value and the scene complexity.

Description

Technical Field

[0001] The present disclosure relates to the field of autonomous driving technology, and in particular to the testing and evaluation of autonomous driving vehicles, and specifically to a method, apparatus, electronic device, computer-readable storage medium, and computer program product for evaluating the driving behavior of an autonomous driving vehicle. Background Art

[0002] When testing autonomous vehicles on the road, changes in the road's characteristics and traffic environment can impact the evaluation results. Autonomous vehicles face different driving scenarios at different times and locations, and the key factors for each scenario vary depending on the time of day and location. If autonomous vehicle road tests are conducted on different road sections or at different times, the results can vary significantly due to the surrounding road conditions. Summary of the Invention

[0003] The present disclosure provides a method, apparatus, electronic device, computer-readable storage medium, and computer program product for evaluating the driving behavior of an autonomous vehicle.

[0004] According to one aspect of the present disclosure, a method for evaluating the driving behavior of an autonomous vehicle is provided, comprising: obtaining driving data collected by the vehicle during autonomous driving; determining the driving behavior of the vehicle and at least one index value based on the driving data, the at least one index value being used to characterize the determined driving behavior; determining a scene complexity associated with the driving behavior; and determining an evaluation result for the driving behavior based on the at least one index value and the scene complexity.

[0005] According to another aspect of the present disclosure, there is provided an apparatus for evaluating the driving behavior of an autonomous driving vehicle, comprising: a driving data acquisition unit for acquiring driving data collected by the vehicle during autonomous driving; an index determination unit for determining the driving behavior of the vehicle and at least one index value based on the driving data, the at least one index value being used to characterize the determined driving behavior; a complexity determination unit for determining the scene complexity associated with the driving behavior; and an evaluation unit for determining an evaluation result for the driving behavior based on the at least one index value and the scene complexity.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute a method for evaluating the driving behavior of an autonomous vehicle according to one or more embodiments of the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute a method for evaluating the driving behavior of an autonomous vehicle according to one or more embodiments of the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, implements a method for evaluating a driving behavior of an autonomous vehicle according to one or more embodiments of the present disclosure.

[0009] An autonomous driving vehicle includes a processor that controls the autonomous driving behavior of the autonomous driving vehicle according to control logic, wherein the control logic is based on an evaluation result of a method for evaluating the driving behavior of an autonomous driving vehicle according to one or more embodiments of the present disclosure.

[0010] According to one or more embodiments of the present disclosure, the driving behavior performance of an autonomous driving vehicle can be evaluated more accurately.

[0011] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings illustrate exemplary embodiments and constitute a part of the specification. Together with the description of the specification, they serve to explain exemplary implementation of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals designate similar, but not necessarily identical, elements.

[0013] Figure 1 A schematic diagram illustrating an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure;

[0014] Figure 2 A flowchart of a method for evaluating driving behavior of an autonomous vehicle according to an embodiment of the present disclosure is shown;

[0015] Figure 3A-3C shows an example of a range of interest according to an embodiment of the present disclosure;

[0016] Figure 4 A structural block diagram of an apparatus for evaluating driving behavior of an autonomous vehicle according to an embodiment of the present disclosure is shown;

[0017] Figure 5 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0018] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0019] In this disclosure, unless otherwise specified, the use of terms such as "first" and "second" to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements. Such terms are only used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of the element, while in some cases, based on the context of the description, they may also refer to different instances.

[0020] The terms used in the descriptions of the various examples described in this disclosure are for the purpose of describing specific examples only and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element may be one or more. In addition, the term "and / or" used in this disclosure encompasses any one and all possible combinations of the listed items.

[0021] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0022] Figure 1 FIG2 is a schematic diagram of an exemplary system 100 in which the various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a motor vehicle 110, a server 120, and one or more communication networks 110 coupling the one or more client devices 110 to the server 120. The client devices 101, 102, 103, 104, 105, and 106 can be configured to execute one or more applications 130.

[0023] In an embodiment of the present disclosure, the motor vehicle 110 may include a computing device according to an embodiment of the present disclosure and / or be configured to perform a method according to an embodiment of the present disclosure.

[0024] The server 120 may run one or more services or software applications that enable execution of the method for evaluating the driving behavior of an autonomous vehicle according to the present disclosure.

[0025] In some embodiments, server 120 may also provide other services or software applications that may include non-virtualized environments and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0026] exist Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105 and / or 106 of the motor vehicle 110 may, in turn, utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0027] The user can use client devices 101, 102, 103, 104, 105 and / or 106 to evaluate the driving behavior of the autonomous vehicle, view the evaluation results, adjust the control parameters or control logic of the vehicle, etc. The client device can provide an interface that enables the user of the client device to interact with the client device. The client device can also output information to the user via the interface. Figure 1 Only six client devices are depicted, but one skilled in the art will appreciate that the present disclosure can support any number of client devices.

[0028] Client devices 101, 102, 103, 104, 105, and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (such as Google Chrome OS); or include various mobile operating systems, such as Microsoft Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include cellular phones, smartphones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Gaming systems may include various handheld gaming devices, internet-enabled gaming devices, etc. Client devices are capable of executing a variety of different applications, such as various internet-related applications, communication applications (such as email applications), and short message service (SMS) applications, and may use various communication protocols.

[0029] The network 110 may be any type of network known to those skilled in the art that can support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0030] Server 120 may include one or more general-purpose computers, specialized server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0031] The computing units in the server 120 may run one or more operating systems including any of the operating systems described above as well as any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, and the like.

[0032] In some embodiments, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from user motor vehicles 110 of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106 motor vehicles 110.

[0033] In some embodiments, server 120 may be a distributed system server or a server integrated with blockchain. Server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host equipped with artificial intelligence technology. A cloud server is a host product within the cloud computing service system that addresses the management difficulties and poor scalability of traditional physical hosts and virtual private servers (VPS) services.

[0034] The network 130 may be any type of network known to those skilled in the art that can support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a satellite communication network, a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (including, for example, Bluetooth, WiFi), and / or any combination of these and other networks.

[0035] The system 100 may also include one or more databases 130-150. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130-150 may be used to store information such as audio files and video files. The database 130 data repository 150 may reside in a variety of locations. For example, the database data repository used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The database 130 data repository 150 may be of different types. In some embodiments, the data repository used by the server 120 may be a database, such as a relational database. One or more of these databases may store, update, and retrieve data to and from the database in response to commands.

[0036] In some embodiments, one or more of the databases 130150 may also be used by an application to store application data. The databases used by the application may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0037] Motor vehicle 110 may include sensors 111 for sensing its surroundings. Sensors 111 may include one or more of the following: visual cameras, infrared cameras, ultrasonic sensors, millimeter-wave radar, and laser radar (LiDAR). Different sensors offer different detection accuracy and range. Cameras may be mounted on the front, rear, or other locations of the vehicle. Visual cameras can capture real-time information about the vehicle's interior and exterior and present it to the driver and / or passengers. Furthermore, by analyzing the images captured by the visual cameras, information such as traffic light indications, intersection conditions, and the operating status of other vehicles can be obtained. Infrared cameras can detect objects in night vision conditions. Ultrasonic sensors can be mounted on all sides of the vehicle, utilizing the strong directionality of ultrasonic waves to measure the distance of external objects from the vehicle. Millimeter-wave radars can be mounted on the front, rear, or other locations of the vehicle, utilizing the properties of electromagnetic waves to measure the distance of external objects from the vehicle. LiDARs can be mounted on the front, rear, or other locations of the vehicle, detecting object edges and shapes for object recognition and tracking. Due to the Doppler effect, radar devices can also measure changes in the speed of the vehicle and moving objects.

[0038] The motor vehicle 110 may also include a communication device 112. The communication device 112 may include a satellite positioning module that can receive satellite positioning signals (e.g., Beidou, GPS, GLONASS, and GALILEO) from satellites 141 and generate coordinates based on these signals. The communication device 112 may also include a module for communicating with a mobile communication base station 142. The mobile communication network may implement any suitable communication technology, such as GSM / GPRS, CDMA, LTE, and other current or evolving wireless communication technologies (e.g., 5G technology). The communication device 112 may also have a vehicle-to-everything (V2X) module that is configured to implement vehicle-to-vehicle (V2V) communication with other vehicles 143 and vehicle-to-infrastructure (V2I) communication with infrastructure 144, for example. In addition, the communication device 112 may also include a module configured to communicate with a user terminal 145 (including but not limited to a smartphone, tablet computer, or wearable device such as a watch) via a wireless local area network or Bluetooth using the IEEE 802.11 standard, for example. Using the communication device 112, the motor vehicle 110 may also access the server 120 via the network 130.

[0039] The motor vehicle 110 may also include a control device 113. The control device 113 may include a processor that communicates with various types of computer-readable storage devices or media, such as a central processing unit (CPU) or a graphics processing unit (GPU), or other dedicated processors. The control device 113 may include an autonomous driving system for automatically controlling various actuators in the vehicle. The autonomous driving system is configured to control the powertrain, steering system, and braking system of the motor vehicle 110 (not shown) via multiple actuators in response to input from multiple sensors 111 or other input devices to control acceleration, steering, and braking, respectively, without human intervention or limited human intervention. Some processing functions of the control device 113 may be implemented through cloud computing. For example, some processing may be performed using an on-board processor, while other processing may be performed using computing resources in the cloud. The control device 113 may be configured to execute the method according to the present disclosure. In addition, the control device 113 may be implemented as an example of a computing device on the motor vehicle side (client) according to the present disclosure.

[0040] Figure 1 The system 100 may be configured and operated in various ways to enable application of the various methods and apparatuses described in accordance with the present disclosure.

[0041] Reference below Figure 2A method 200 for evaluating driving behavior of an autonomous vehicle according to an exemplary embodiment of the present disclosure is described.

[0042] In step S201 , driving data collected by the vehicle during automatic driving is acquired.

[0043] In step S202 , a driving behavior of the vehicle and at least one index value are determined based on the driving data, where the at least one index value is used to characterize the determined driving behavior.

[0044] In step S203 , the scene complexity associated with the driving behavior is determined.

[0045] In step S204 , an evaluation result for the driving behavior is determined based on the at least one indicator value and the scene complexity.

[0046] According to the embodiments of the present disclosure, the performance of autonomous driving can be judged based on both the vehicle's operational complexity and indicators, enabling a more accurate assessment of the autonomous vehicle's driving behavior. This accurate assessment can also be used to optimize operational performance.

[0047] As a non-limiting example, consider two test results: one with a congested road and therefore high actual complexity, and a low vehicle speed, i.e., a low index value; and another with an open road and therefore low complexity, and a moderate vehicle speed, i.e., an index value. If only the index value is considered, the latter may be concluded to be superior; however, combining complexity with the index value may reveal that the former performed well, while the latter was too slow. Therefore, combining complexity and index value may yield a more accurate evaluation result. According to some embodiments, combining complexity and index value may be a quantitative evaluation, as described further below. As another non-limiting example, for a case with a complexity of 78 (assuming the maximum value is 100), when a vehicle passes through an intersection at a slower time (e.g., 10 meters), the vehicle's performance is actually relatively good; whereas, for a case with a complexity of only 10, when the vehicle passes through at a slower time (e.g., 8 meters), the vehicle's performance may be significantly poorer. Failure to consider complexity may lead to erroneous results.

[0048] As a non-limiting example, driving behavior can be going straight, turning, merging, reversing, parking, etc. Different monitoring indicators can be used for different driving behaviors. For example, examples of indicators for merging can include the time of crossing the line, the horizontal and vertical distances at the time of crossing the line, the pass rate, the lane change time, reasonable braking, the average lane change speed, etc. As another example, for normal driving behavior, or also known as "following the vehicle in front" behavior, the evaluation indicators can include vehicle speed, distance to the vehicle in front, etc. It will be understood that this is only an example, and the present disclosure is not limited to this.

[0049] The method according to the present disclosure can be applied to road testing, for example, as a data mining tool to evaluate vehicle driving conditions for subsequent vehicle function optimization. However, it is understood that the present disclosure is not limited to this and can be applied, for example, to offline review and online vehicle evaluation, etc.

[0050] According to some embodiments, determining the scene complexity associated with the driving behavior may include: determining a road condition complexity and an interactive element complexity, wherein the road condition complexity indicates the complexity of the road on which the vehicle is located, and the interactive element complexity indicates the complexity of the elements surrounding the vehicle; and determining the scene complexity based on the road condition complexity and the interactive element complexity.

[0051] According to such an embodiment, the complexity of road conditions may include road information, intersection information, overall road condition information such as road congestion, etc., and the complexity of interactive elements may include other vehicles, pedestrians, non-motor vehicles, obstacles, etc. appearing around the vehicle. Thus, various influencing factors of the vehicle driving scene can be comprehensively considered to obtain more accurate complexity results.

[0052] According to some embodiments, determining the scene complexity based on the road condition complexity and the interactive element complexity may include: determining a weighted sum of the road condition complexity and the interactive element complexity as the scene complexity.

[0053] According to such an embodiment, an example of how to quantitatively calculate complexity is given, that is, the complexity can be calculated separately and then weighted and summed to fully consider the influence of both the road itself and the surrounding interactive elements.

[0054] According to some embodiments, determining the weighted sum of the road condition complexity and the interactive element complexity may include giving the interactive element complexity a higher weight than the road condition complexity.

[0055] Such an embodiment is based on the consideration that, during vehicle travel, the elements appearing around the vehicle (e.g., oncoming vehicles, pedestrians crossing the road) have a greater impact on vehicle operation than the overall road conditions (e.g., how many vehicles are on this road on average), and therefore can be given a greater weight. Interactive elements may include elements within a predetermined range of the vehicle (e.g., within a perception range), or elements within other ranges. Alternatively, interactive elements may include elements within a range of orientation specific to the current state of the vehicle, which will be described in the optional embodiments below.

[0056] According to some embodiments, determining the traffic complexity may include determining static traffic complexity based on road network information.

[0057] According to such an embodiment, the road condition complexity may include static road condition complexity, and the static road condition information may include intersection information (intersection type, traffic light type, intersection size, etc.) and road information (lane width, number of lanes, lane type, lane speed limit), etc.

[0058] According to some embodiments, determining the road condition complexity may include determining dynamic road condition complexity based on roadside sensing information acquired by roadside equipment. Dynamic road conditions may include traffic flow (vehicle flow density, non-motor vehicle flow density, pedestrian flow density, congestion) and traffic violation information (non-motor vehicles, pedestrians, vehicles driving against traffic, parking in the road, pedestrians and non-motor vehicles blocking the road), etc.

[0059] For example, traffic density refers to the density of obstacles on a single lane within a region of interest (ROI). Road congestion refers to the average speed distribution of surrounding vehicles when traffic density reaches a certain level. The congestion level distribution varies depending on the lane speed limit.

[0060] According to one or more embodiments, a quantitative description of the complexity characteristics of autonomous driving scenarios can be generated by comprehensively considering at least one or more of multiple factors, including static road conditions, traffic flow, traffic behavior, and autonomous vehicles. This description can provide a basis for assessing the difficulty of driving tasks, enable horizontal comparison of driving scenario environments, and provide constraints for evaluating autonomous driving capabilities.

[0061] Driving scenarios vary widely. Quantitatively analyzing driving scenarios based on different roads and driving behaviors, and extracting key features of the road and traffic environment based on specific driving scenarios, are key technical challenges in the development of autonomous vehicles. The complexity of driving scenarios is often closely related to the driving situation and traffic environment.

[0062] By considering the situation from the perspective of an autonomous vehicle, extracting key features from static, dynamic, and interactive factors, and modeling the complexity of the vehicle's surroundings using these key scenario features, we can quantify the scene complexity of the autonomous vehicle's driving tasks. Road tests and evaluations of the vehicle's driving capabilities can then be conducted based on the complexity of autonomous driving scenarios at the same level, achieving more accurate evaluations.

[0063] More specifically, scene complexity can be quantified based on the characteristics of the autonomous vehicle's test environment, taking into account static road conditions, traffic behavior, traffic flow, and autonomous vehicle characteristics. For example, scene complexity can include both static and dynamic elements. Static elements can include intersection data and road data. More specifically, the former can include intersection size, intersection type, and signal type, while the latter can include the number of lanes, lane width, and road type. Dynamic elements can include obstacle data and host vehicle data. The former can further include obstacle speed, number, and type, while the latter can include the host vehicle's position and speed. Furthermore, consideration can be given to more interactive complexity factors for the vehicle, such as intersection data, number of roads, number of interacting vehicles, number of interacting non-motorized vehicles, interactive pedestrian data, and traffic violation data. Data sources can include map data, host vehicle data, and obstacle data. It should be understood that the above are merely examples and the present disclosure is not limited to these.

[0064] The scene complexity calculation tool can comprehensively analyze the main vehicle data, perception data or obstacle data, map data, etc. for calculation. As a non-limiting example, an open road network may include highways, auxiliary roads, roads and intersections. Driving scenarios may include following, turning left at an intersection, turning right at an intersection, going straight at an intersection, changing lanes left, changing lanes right, etc. Based on universal data processing capabilities, data mining can be performed, such as road network portrait mining and scene indicator mining. The former may include dynamic elements and static elements, and the latter may include interactive elements and main vehicle strategies, etc. In this way, an objective evaluation of driving rationality can be formed, and driving efficiency as well as safety, intelligence, comfort, etc. can be improved.

[0065] In conjunction with Table 1, some example factors are listed below for high-frequency driving scenarios, including right turns at intersections, left turns at intersections, going straight at intersections, following, changing lanes, and responding to cutting vehicles.

[0066] Table 1

[0067]

[0068]

[0069] It should be understood that the above factors are examples, and in practice, more, fewer, or completely different factors may be used to determine complexity. Furthermore, the methods according to embodiments of the present disclosure may also be applied to driving behavior scenarios not listed in the table. For example, as a non-limiting example, static elements, dynamic elements, and interactive obstacle data can be determined based on map data, obstacle data, and vehicle data. Static elements may include intersection size, intersection type, and intersection signal type. Dynamic elements may include vehicle flow density, non-motorized vehicle flow density, and pedestrian flow density. A road condition difficulty model can be established based on these static and dynamic elements. Similarly, interactive obstacle data may include, for example, the number of vehicles within the ROI, the number of non-motorized vehicles within the ROI, the number of pedestrians within the ROI, and whether there are following vehicles ahead, and thus, an interactive obstacle model can be established. Different weights can be assigned to the road condition model and the interactive obstacle model to determine scenario complexity. This is an example of how complexity can be determined based on data using mining algorithms, indicator systems, and statistical algorithms.

[0070] According to some embodiments, determining the complexity of interactive elements may include: obtaining vehicle-side perception information; determining perceived element information in the vehicle-side perception information; and determining the complexity of the interactive elements based on the element information. The element information may include, for example, the number of elements, element type, and element attributes (such as speed and direction). In such embodiments, vehicle perception information can be fully utilized.

[0071] It is understandable that in other embodiments, element information can also be determined based on or in combination with data perceived by other vehicles or data perceived by roadside equipment, and the present disclosure is not limited to this. Other technologies for obtaining information on elements around the vehicle that can be imagined by those skilled in the art can be applied to the method of the present disclosure.

[0072] According to some optional embodiments, interactive dynamic factors can be considered, including dynamic factors that have a direct impact on the autonomous vehicle, and the number of moving vehicles, pedestrians, and non-motor vehicles that have an impact on the autonomous vehicle. By considering the distance between the autonomous vehicle and the moving vehicles, pedestrians, and non-motor vehicles, and the angle between the speed direction of the autonomous vehicle and the moving vehicles, pedestrians, and non-motor vehicles. The interactive dynamic elements emphasize the dynamic characteristics, and the interactive element indicator mining is the number of motor vehicles, non-motor vehicles, and pedestrians that appear in the ROI perception range. At different position moments, the influence of traffic participants on the main vehicle is different, so the position of the perceived ROI will also change accordingly depending on the direction and position of the vehicle.

[0073] According to some embodiments, determining the perceived element information in the vehicle-side perception information may include: determining a range of interest based on the type of driving behavior; and determining element information of elements in the vehicle-side perception information that are within the range of interest of the vehicle.

[0074] The range of interest can also be referred to as the interactive orientation range. According to such an embodiment, different interactive areas (e.g., front, turning direction, etc.) can be set for different types of driving behaviors to be evaluated (e.g., straight, turning, etc.), and the characteristics that elements in specific interactive areas have a greater impact on driving are fully considered to obtain more accurate complexity. Through such an embodiment, different regions of interest can be set according to the direction and position of the vehicle, different driving behaviors, etc., thereby resolving the impact of obstacles in unrelated lanes on the algorithm results, and fully considering the objective factors of different states of the host vehicle at different positions and times, obtaining a universal design for all scenarios based on the change of range.

[0075] According to some embodiments, the range of interest may include a different azimuth range for each of the at least two behavior sub-stages in the driving behavior, thereby fully considering the most critical influencing factors under different vehicle states.

[0076] According to some other optional embodiments, the range of interest may include a different range for each of a plurality of predetermined element types, wherein the plurality of predetermined element types are selected from the group consisting of: vehicles, non-motorized vehicles, pedestrians, and static obstacles. This is because different types of elements may have different degrees of impact on the vehicle within the same distance and angle.

[0077] Figure 3A-3C An example of the scope of interest is shown by taking a left turn scenario as an example. In such a non-limiting example, the left turn scenario is divided into three stages according to the position of the intersection where the host vehicle is located: the start period, the middle period, and the end period.

[0078] The start period may include a period from the left turn start time to the time when 1 / 4 of the left turn duration occurs.

[0079] like Figure 3A As shown, during the start period, the range of interest may be radius R0 = 10m, angle range (0°, 30°), radius R1 = 5m (30°, 60°), and R2 = 3m (60°, 120°). It will be understood that although the angle ranges are indicated in parentheses, the angle ranges here may be open or closed, and the present disclosure is not limited thereto.

[0080] As an optional embodiment, the above-mentioned range of interest can be the range of interest for vehicles, and the range of interest for non-motor vehicles can be radius R0 = 7m (0°, 30°), radius R1 = 5m (30°, 60°), and radius R2 = 2m (60°, 120°).

[0081] The intermediate period may include a period from a quarter of the left turn duration to a half of the left turn duration.

[0082] like Figure 3B As shown, during the middle period, the range of interest may be radius R0 = 12 m, angular range (0°, 30°), radius R1 = 7 m, angular range (30°, 60°), radius R2 = 3.5 m, angular range (60°, 120°).

[0083] As an optional embodiment, the above-mentioned range of interest can be the range of interest for vehicles, and the range of interest for non-motor vehicles can be radius R0 = 7m, angle range (0°, 30°), radius R1 = 5m, angle range (30°, 60°), radius R2 = 3m, angle range (60°, 120°).

[0084] The end period may include the period from 1 / 2 of the left turn duration to the end time.

[0085] like Figure 3C As shown, during the end period, the range of interest may be radius R0 = 12 m, angular range (0°, 30°), radius R1 = 7 m, angular range (30°, 60°), radius R2 = 3.5 m, angular range (60°, 120°).

[0086] As an optional embodiment, the above-mentioned range of interest can be the range of interest for vehicles, and the range of interest for non-motor vehicles. Radius R0 = 8m, angle range (0°, 30°), radius R1 = 8m, angle range (30°, 60°), radius R2 = 3m, angle range (60°, 120°).

[0087] It will be appreciated that the above-described stage segmentation, region of interest shapes, values, element types, and the like are merely examples, and the present disclosure is not limited thereto. As an alternative, non-limiting example, a left-turn scenario can be divided into fewer or more stages. As another alternative, non-limiting example, a separate region of interest can be set for pedestrians, pedestrians can share the same region of interest with non-motor vehicles, and so on.

[0088] It is understandable that when only elements within the area of ​​interest are selected, all perception data can still be obtained (for example, the vehicle can obtain perception data within a circle with a radius of 100m), but only the elements in the "range of interest" are processed during analysis, or a larger weight is given to the elements in this area, and so on.

[0089] As another example, for a scene of merging to the left, the range of interest may include the rear, the left front, etc., while the right may be ignored or given a lower weight.

[0090] According to some embodiments, determining an evaluation result for the driving behavior based on the at least one indicator and the scene complexity may include: determining an indicator score for each of the at least one indicator value, including: obtaining a predetermined value distribution range for the indicator corresponding to the indicator value; determining an indicator score based on the indicator value and the predetermined value distribution range; and determining the evaluation result based on the determined at least one indicator score and the scene complexity. Thus, an evaluation result that takes complexity into account can be determined based on the indicator value, the indicator distribution curve, and the current complexity.

[0091] According to some embodiments, determining the indicator score based on the indicator value and the predetermined value distribution range may include obtaining a normalized indicator value based on the indicator value and the predetermined value distribution range as the indicator score. By determining the normalized indicator value based on the position of the indicator value in the distribution curve, the level of the phenomenon represented by the indicator can be conveniently quantified.

[0092] According to some embodiments, determining the evaluation result based on the determined at least one indicator score and the scene complexity may include: determining the evaluation result based on a ratio of the at least one indicator score to the scene complexity.

[0093] The ratio of the indicator score to the complexity is used to obtain the evaluation result, which is equivalent to a normalization based on complexity. As an example, the complexity can be in numerical form, such as between 0 and 100, with 80 or above indicating very complex.

[0094] As other optional embodiments, other methods may be used, for example, different score lines (intervals) may be set according to different complexities to determine different evaluation levels such as "excellent / good / poor", etc. It is understood that the present disclosure is not limited thereto.

[0095] Currently, testing of autonomous vehicle driving behavior on open roads largely relies on on-board problem submissions from road test engineers. This results in highly subjective evaluations and poor generalization. Encountering unique scenarios or road conditions can lead to significant deviations from the original evaluation results. Furthermore, such evaluations fail to adequately consider the dynamic nature of the road's traffic environment. Even when the complexity of driving scenarios is considered, the evaluation results are primarily informed by qualitative analysis, which is imprecise and lacks objective data support, resulting in low credibility. Furthermore, most current road testing methods for autonomous vehicles evaluate the entire driving mission, ignoring the occurrence of manual takeover, the completion of individual missions, and any issues encountered within them. These methods lack horizontal comparisons across long-distance, complex traffic environments, thus diminishing the actual value of the evaluations.

[0096] According to some embodiments, the method may further include adjusting control parameters of the vehicle based on the evaluation results. Adjusting autonomous driving control parameters based on the evaluation results of the vehicle test run can achieve better performance, suitable for various levels of complexity, and thus achieve a better performing autonomous driving vehicle.

[0097] Now refer to Figure 4 An apparatus 400 for evaluating the driving behavior of an autonomous vehicle according to an embodiment of the present disclosure is described. The apparatus 400 for evaluating the driving behavior of an autonomous vehicle may include a driving data acquisition unit 401, an index determination unit 402, a complexity determination unit 403, and an evaluation unit 404. The driving data acquisition unit 401 may be used to acquire driving data collected by the vehicle during autonomous driving. The index determination unit 402 may be used to determine the driving behavior of the vehicle and at least one index value based on the driving data, the at least one index value being used to characterize the determined driving behavior. The complexity determination unit 403 may be used to determine the scene complexity associated with the driving behavior. The evaluation unit 404 may be used to determine an evaluation result for the driving behavior based on the at least one index value and the scene complexity.

[0098] According to the device described in the embodiment of the present disclosure, the driving behavior performance of an autonomous driving vehicle can be evaluated more accurately.

[0099] Optionally, the complexity determination unit 403 may include: a unit for determining the road condition complexity and the interactive element complexity, wherein the road condition complexity indicates the complexity of the road on which the vehicle is located, and the interactive element complexity indicates the complexity of the elements around the vehicle; and a unit for determining the scene complexity based on the road condition complexity and the interactive element complexity.

[0100] In the technical solutions disclosed herein, the collection, acquisition, storage, use, processing, transmission, provision and public application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0101] According to an embodiment of the present disclosure, an electronic device, a readable storage medium, and a computer program product are also provided.

[0102] According to another aspect of the present disclosure, an autonomous driving vehicle is also provided, which includes a processor, which controls the autonomous driving behavior of the autonomous driving vehicle according to control logic, wherein the control logic is based on an evaluation result of a method for evaluating the driving behavior of an autonomous driving vehicle according to an embodiment of the present disclosure (for example, obtained, determined or adjusted based on the evaluation result, etc.).

[0103] According to another aspect of the present disclosure, an edge computing device is also provided. Optionally, the edge computing device may include not only electronic devices but also communication components, and the electronic devices and communication components may be integrated or provided separately. The electronic devices can obtain data, such as images and videos, from roadside sensing devices (such as roadside cameras), perform image and video processing and data calculations, and then transmit the processing and calculation results to the cloud control platform via the communication components.

[0104] Optionally, the edge computing device can also be a roadside computing unit (RSCU). Optionally, the electronic device itself can also have sensory data acquisition and communication functions, such as an AI camera. The electronic device can directly perform image and video processing and data calculation based on the acquired sensory data, and then transmit the processing and calculation results to the cloud control platform.

[0105] Optionally, the cloud control platform performs processing in the cloud, performing image and video processing and data calculation. The cloud control platform can also be called a vehicle-road collaborative management platform, V2X platform, cloud computing platform, central system, cloud server, etc.

[0106] refer to Figure 5, a block diagram of an electronic device 500 that can serve as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0107] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 can also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0108] Multiple components within electronic device 500 are connected to I / O interface 505, including an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. Input unit 506 can be any type of device capable of inputting information into electronic device 500. Input unit 506 can receive input numeric or character information and generate key signal input related to user settings and / or function control of the electronic device. It may include, but is not limited to, a mouse, keyboard, touch screen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 507 can be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. Storage unit 508 may include, but is not limited to, a magnetic disk or an optical disk. Communication unit 509 allows electronic device 500 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks. It may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and / or chipset, such as a Bluetooth™ device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0109] The computing unit 501 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as method 200 and its variations. For example, in some embodiments, method 200 and its variations can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of method 200 and its variations described above can be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the method 200 and its variations, etc., in any other appropriate manner (eg, by means of firmware).

[0110] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0111] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0112] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0113] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0114] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0115] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0116] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0117] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but is only limited by the claims after authorization and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. In addition, the steps may be performed in an order different from that described in this disclosure. Further, the various elements in the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced by equivalent elements that appear after this disclosure.

Claims

1. A method for evaluating driving behavior of an autonomous vehicle, comprising: Acquiring driving data collected by the vehicle during autonomous driving; determining a driving behavior of the vehicle and at least one index value based on the driving data, wherein the at least one index value is used to characterize the determined driving behavior; determining a scenario complexity associated with the driving behavior; as well as determining an evaluation result for the driving behavior based on the at least one indicator value and the scene complexity, The scene complexity includes static road complexity, dynamic road complexity and interactive element complexity. The static road condition complexity includes lane-related information, including at least one of lane width, lane number, and lane type. The dynamic road condition complexity includes traffic flow information, including at least one of vehicle flow density, non-motor vehicle flow density, pedestrian flow density, and congestion degree, and The interactive element complexity includes element information of elements in a range of interest determined based on the type of driving behavior, wherein the range of interest includes a different range for each element type of multiple predetermined element types, wherein the multiple predetermined element types are selected from the group consisting of: vehicles, non-motor vehicles, pedestrians, and static obstacles.

2. The method according to claim 1, wherein Determining the scene complexity associated with the driving behavior includes determining the scene complexity based on road condition complexity and the complexity of the interactive elements, where the road condition complexity includes the static road condition complexity and the dynamic road condition complexity.

3. The method according to claim 2, wherein: Determining the scene complexity based on the road condition complexity and the interactive element complexity includes: determining a weighted sum of the road condition complexity and the interactive element complexity as the scene complexity. 4 . The method according to claim 3 , wherein determining the weighted sum of the road condition complexity and the interactive element complexity comprises giving the interactive element complexity a higher weight than the road condition complexity.

5. The method according to any one of claims 2 to 4, wherein Determining the road condition complexity includes determining static road condition complexity based on road network information.

6. The method according to any one of claims 2 to 5, wherein: Determining the road condition complexity includes determining the dynamic road condition complexity based on roadside perception information acquired by the roadside equipment.

7. The method according to any one of claims 1 to 6, wherein Determining the complexity of interactive elements includes: Obtain vehicle side perception information; Determining the sensed element information in the vehicle-side sensed information; and The interaction element complexity is determined based on the element information.

8. The method according to claim 7, wherein: Determining the perceived element information in the vehicle-side perception information includes: determining a range of interest based on the type of driving behavior; and Determine element information of elements in the vehicle-side perception information that are within the range of interest of the vehicle.

9. The method according to claim 8, wherein The range of interest includes a different orientation range for each of at least two behavior sub-phases in the driving behavior.

10. The method according to any one of claims 1 to 9, wherein Determining an evaluation result for the driving behavior based on the at least one indicator and the scene complexity includes: Determining an indicator score for each indicator value of the at least one indicator value includes: Obtaining a predetermined value distribution range of an indicator corresponding to the indicator value; and An indicator score is determined based on the indicator value and the predetermined value distribution range; and the evaluation result is determined based on the determined at least one indicator score and the scene complexity.

11. The method according to claim 10, wherein: Determining the indicator score based on the indicator value and the predetermined value distribution range includes acquiring a normalized indicator value as the indicator score based on the indicator value and the predetermined value distribution range.

12. The method according to claim 10 or 11, wherein: Determining the evaluation result based on the determined at least one indicator score and the scene complexity includes: determining the evaluation result based on a ratio of the at least one indicator score to the scene complexity.

13. The method according to any one of claims 1-12, further comprising adjusting a control parameter of the vehicle based on the evaluation result.

14. A device for evaluating driving behavior of an autonomous vehicle, comprising: a driving data acquisition unit, configured to acquire driving data collected by the vehicle during autonomous driving; an index determination unit, configured to determine a driving behavior of the vehicle and at least one index value based on the driving data, wherein the at least one index value is used to characterize the determined driving behavior; a complexity determination unit, configured to determine a scene complexity associated with the driving behavior; as well as an evaluation unit, configured to determine an evaluation result for the driving behavior based on the at least one indicator value and the scene complexity, The scene complexity includes static road complexity, dynamic road complexity and interactive element complexity. The static road condition complexity includes lane-related information, including at least one of lane width, lane number, and lane type. The dynamic road condition complexity includes traffic flow information, including at least one of vehicle flow density, non-motor vehicle flow density, pedestrian flow density, and congestion degree, and The interactive element complexity includes element information of elements in a range of interest determined based on the type of driving behavior, wherein the range of interest includes a different range for each element type of multiple predetermined element types, wherein the multiple predetermined element types are selected from the group consisting of: vehicles, non-motor vehicles, pedestrians, and static obstacles.

15. The device according to claim 14, wherein The complexity determination unit includes a unit for determining the scene complexity based on the road condition complexity and the interactive element complexity, and the road condition complexity includes the static road condition complexity and the dynamic road condition complexity.

16. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 13.

17. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-13.

18. A computer program product comprising a computer program, wherein When the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

19. An autonomous driving vehicle, comprising a processor, wherein the processor controls the autonomous driving behavior of the autonomous driving vehicle according to control logic, wherein: The control logic is based on the evaluation result of the method according to any one of claims 1-13.

Images