Method and apparatus for autonomous vehicle

Abstract

According to embodiments of this disclosure, a method and apparatus for autonomous vehicles are provided. The method includes determining predictive information corresponding to a target time based on multiple images acquired by the autonomous vehicle, the predictive information indicating predictive attributes of a traffic light; updating the predictive information based on historical attribute information of the traffic light to determine a target attribute of the traffic light; comparing the target attribute of the traffic light with a reference attribute indicated by map data; and determining a control strategy for the autonomous vehicle based on a confidence level of the target attribute in response to a difference between the target attribute and the reference attribute. Therefore, embodiments of this disclosure can control autonomous vehicles more efficiently and improve driving safety.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to methods, apparatus, devices, computer-readable storage media, and computer program products for autonomous vehicles. Background Technology

[0002] Autonomous driving is a technology that uses computers to replace or assist human drivers in perceiving the vehicle's surroundings, planning the vehicle's trajectory, and controlling the vehicle to reach a designated destination.

[0003] Traffic light perception is a crucial aspect of safe driving for autonomous vehicles. As an essential component of traffic regulations, traffic lights determine whether a vehicle needs to stop, continue, or make other safety decisions, impacting the safety, traffic efficiency, and passenger comfort of autonomous vehicles. Therefore, accurate and timely traffic light recognition is vital for enabling autonomous vehicles to react correctly in complex road environments. Summary of the Invention

[0004] In a first aspect of this disclosure, a method for an autonomous vehicle is provided. The method includes: determining prediction information corresponding to a target time based on multiple images acquired by the autonomous vehicle, the prediction information indicating a prediction attribute of a traffic light; updating the prediction information based on historical attribute information of the traffic light to determine a target attribute of the traffic light; comparing the target attribute of the traffic light with a reference attribute indicated by map data; and determining a control strategy for the autonomous vehicle based on a confidence level of the target attribute in response to the target attribute differing from the reference attribute.

[0005] In a second aspect of this disclosure, an apparatus for an autonomous vehicle is provided. The apparatus includes: a first determining module configured to determine prediction information corresponding to a target time based on multiple images acquired by the autonomous vehicle, the prediction information indicating a prediction attribute of a traffic light; an updating module configured to update the prediction information based on historical attribute information of the traffic light to determine a target attribute of the traffic light; a comparison module configured to compare the target attribute of the traffic light with a reference attribute indicated by map data; and a second determining module configured to determine a control strategy for the autonomous vehicle based on a confidence level of the target attribute in response to a difference between the target attribute and the reference attribute.

[0006] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the first aspect.

[0007] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the first aspect.

[0008] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product includes computer-executable instructions that, when executed by a processor, implement the first aspect.

[0009] It should be understood that the content described in this summary section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0011] Figure 1 An example scenario of changing traffic light attributes is shown;

[0012] Figure 2 A schematic diagram illustrating an example environment in which embodiments of this disclosure can be implemented is shown;

[0013] Figure 3 A schematic diagram illustrating an example process for determining the attributes of an autonomous vehicle according to some embodiments of the present disclosure is shown;

[0014] Figure 4 A schematic diagram illustrating an example process for controlling an autonomous vehicle according to some embodiments of the present disclosure is shown;

[0015] Figure 5 A flowchart illustrating an example process for an autonomous vehicle according to some embodiments of this disclosure is shown;

[0016] Figure 6 A schematic structural block diagram of an apparatus for an autonomous vehicle according to certain embodiments of the present disclosure is shown; and

[0017] Figure 7 A block diagram of an apparatus capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0018] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0019] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0020] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0021] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0022] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information other than that necessary for basic functions will not affect the user's use of basic functions.

[0023] As used in this paper, the term "model" refers to a system that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. In this paper, "model" may also be referred to as a "machine learning model," a "machine learning network," or simply a "network," and these terms are used interchangeably.

[0024] As mentioned earlier, traffic light perception is one of the key aspects of safe vehicle operation in autonomous driving.

[0025] Road change issues refer to takeover problems caused by discrepancies between the map data acquired by autonomous vehicles and the real-world environment information, such as changes in traffic light attributes (inconsistencies between real-world attributes and reference attributes indicated by map data) or changes in the service status of traffic lights (e.g., map data indicates that a traffic light is off, but in reality, the traffic light is back on). These issues have a significant impact on the average takeover mileage (MPI) of autonomous vehicles.

[0026] Figure 1 Example scenario 100 of traffic light attribute changes is shown. For example... Figure 1 As shown, scene 100 contains three sets of traffic lights: traffic light 101 is a left-turn light, traffic light 102 is a round light, and traffic light 103 is a right-turn light. However, in the map data, traffic light 101 is a left-turn light, traffic light 102 is a round light, and traffic light 103 is a round light. The actual attributes of traffic light 103 are inconsistent with the reference attributes indicated by the map data.

[0027] In the problem of traffic light status changes, scenarios involving attribute changes occur frequently, and there is a certain risk of running a red light when an autonomous vehicle encounters such a situation for the first time. Therefore, accurately detecting changes in traffic light status is of significant safety importance. On the other hand, detecting changes in traffic light status also enhances the real-time performance of high-precision map updates for autonomous driving, allowing for continuous updates to traffic light attributes.

[0028] Embodiments of this disclosure propose a scheme for autonomous vehicles. According to various embodiments of this disclosure, predictive information corresponding to a target time is determined based on multiple images acquired by the autonomous vehicle. The predictive information is updated based on historical attribute information of the traffic lights to determine the target attributes of the traffic lights. Then, the target attributes of the traffic lights are compared with reference attributes indicated by map data. If the target attributes differ from the reference attributes, a control strategy for the autonomous vehicle is determined based on the confidence level of the target attributes.

[0029] In the embodiments of this disclosure, the current-time prediction information is updated based on the historical attribute information of the traffic lights, and the target attributes of the traffic lights are determined, thereby improving the accuracy of traffic light attribute identification. Simultaneously, by comparing the target attributes of the traffic lights with the reference attributes indicated by map data and accordingly determining the control strategy for the autonomous vehicle, the rapid response and accurate decision-making of the autonomous vehicle in the face of changes in the actual environment can be improved, thus enhancing the safety of autonomous driving.

[0030] Example Environment

[0031] Figure 2 A schematic diagram of an example environment 200 in which embodiments of the present disclosure can be implemented is shown. As shown, environment 200 may include vehicle 210. Vehicle 210 may be an autonomous vehicle, that is, a vehicle with autonomous driving capability (or driverless capability), also known as a driverless car, autonomous driving vehicle, etc.

[0032] In some scenarios, vehicle 210 can be assigned to provide travel services to users. For example, users can obtain travel services provided by vehicle 210 through a travel application. In some scenarios, vehicle 210 may also be referred to as a driverless taxi or robotaxi. During the process of vehicle 210 providing travel services to users, vehicle 210 may be equipped with a safety operator. The safety operator can, for example, take over vehicle 210 in case of an emergency. Alternatively, vehicle 210 may also be in an unmanned state.

[0033] The vehicle 210 may be equipped with at least one sensor (e.g., at least one vehicle-mounted camera) to collect real-time environmental information while the vehicle 210 is in motion. For example, when the vehicle 210 approaches an intersection, each vehicle-mounted camera may take a picture of the traffic light panel 151 to obtain at least one image 221 of the traffic light panel 151.

[0034] In some scenarios, vehicle 210 may also have one or more display devices installed inside the vehicle to provide human-computer interaction functions.

[0035] In some scenarios, electronic device 230 can process the perceived image 221 to obtain attribute information 241 of the traffic lights in the traffic light panel. Such attribute information 241 includes, but is not limited to, the identification signals in the traffic lights (e.g., straight, left turn, right turn, etc.), the corresponding colors of the traffic lights, the numbers in the traffic lights, etc. Such electronic device 230 may include a terminal and / or a server.

[0036] Such a terminal can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, electronic device 230 may also support any type of user-facing interface (such as "wearable" circuitry).

[0037] Such servers can be standalone physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Servers can include, for example, computing systems / servers such as mainframes, edge computing nodes, and computing devices in cloud environments, etc.

[0038] Electronic device 230 can be installed in vehicle 210, or deployed in any electronic unit of vehicle 210 (such as sensors, control units, etc.), or it can exist independently of vehicle 210.

[0039] When the electronic device 230 and the vehicle 210 exist independently, a communication connection can be established between the vehicle 210 and the electronic device 230. This communication connection can be established via wired or wireless means. The communication connection may include, but is not limited to, Bluetooth connections, mobile network connections, Universal Serial Bus connections, Wi-Fi connections, etc., and the embodiments of this disclosure are not limited in this respect. Based on this, the vehicle 210 and the electronic device 230 can achieve signaling interaction through the communication connection between them.

[0040] It should be understood that the structure and function of environment 200 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0041] The following will be referenced Figure 3 This describes an example process for determining the attributes of an autonomous vehicle according to some embodiments of the present disclosure. Figure 3 A schematic diagram of an example process 300 for determining the attributes of an autonomous vehicle according to some embodiments of the present disclosure is shown. Example process 300 can be implemented at electronic device 130. Reference will be made below. Figure 1 Describe example process 300.

[0042] During its operation, vehicle 210 can acquire multiple images of itself from different perspectives in real time via a camera. For example, when vehicle 210 approaches intersection 261, multiple images of the traffic light panel 251 at the current time (e.g., the target time) can be acquired, such as image 312-1, image 312-2, etc. In some embodiments, the traffic light panel refers to the entire external structure or frame of the traffic light, including all bulbs, digital lights, light shields, and surrounding fixed parts. The traffic light panel can be used to determine the presence and location of the traffic light.

[0043] Based on multiple images captured by multiple cameras, electronic device 230 can determine predictive information corresponding to a target time. This predictive information can indicate the predictive attributes of traffic lights. The attributes of traffic lights can refer to their type, such as round lights, arrow lights, left-turn lights, right-turn lights, etc. These attributes determine the semantic information of the traffic light panel and the associated lane marking information.

[0044] In some embodiments, since multiple cameras have different focal lengths and orientations, the electronic device 230 performs fusion processing on multiple images to accurately determine the prediction information for the traffic lights. The electronic device 230 can determine multiple classification results corresponding to the multiple images, and determine weight information corresponding to the multiple classification results based on the confidence level of the multiple classification results and the size of the corresponding image regions. Then, based on the weight information and the multiple classification results, the prediction information is determined.

[0045] For example, the electronic device 230 can input multiple images into a panel classification model and a panel detection model respectively to obtain multiple classification results and confidence scores of the multiple classification results. Then, based on the multiple classification results and confidence scores of the multiple classification results, multiple model recognition scores (det_scores) of the traffic light panel 251 are determined. In some embodiments, the image region can be the bounding box of the traffic light panel 251, and the size of the image region can be the area of ​​the bounding box. The electronic device 230 can determine the areas of multiple bounding boxes corresponding to multiple images. Then, based on the product of the model recognition score (det_scores) and the area of ​​the corresponding bounding box (area), the weighted score of the traffic light attribute recognition result is determined: weight_scores = det_scores * area, thereby obtaining multiple weighted scores. Taking image 1 as an example, the electronic device 230 can determine the classification result 1 and the confidence score 1 of image 1, and determine the model recognition score 1 of image 1 based on the classification result 1 and the confidence score 1. The electronic device 230 can also determine the area 1 of the bounding box of the traffic light panel 251 in image 1. Then, based on the product of the model recognition score 1 and the area 1, a weighted score for image 1 is obtained.

[0046] In this disclosure, the panel detection model and the panel classification model can be any model capable of detecting and classifying traffic light panels, and this disclosure does not limit them. In some embodiments, multiple weighted scores can be sorted, and the highest weighted score can be selected (314) from the multiple weighted scores, and the attribute recognition result corresponding to the highest weighted score can be used as the predicted attribute 316 of the traffic light.

[0047] In some embodiments, in order to improve the accuracy of traffic light attribute prediction, this disclosure also utilizes historical attribute information of the traffic light to update the current predicted attribute 316, thereby determining the target attribute of the traffic light.

[0048] Electronic device 230 can determine a set of historical attributes of a traffic light over at least one historical period. For example, such as Figure 3 Attributes 1 to 8 are shown. Then, based on this set of historical attributes and the predicted attribute 316 (e.g., attribute 0) indicated by the predicted information, scores for multiple candidate attributes are determined. In some embodiments, the multiple candidate attributes may include the predicted attribute 316 and at least one of the historical attributes.

[0049] In some embodiments, at least one historical period may include multiple historical periods corresponding to different time lengths, such as... Figure 3 The first historical period and the second historical period are shown. The length of the first historical period can be longer than the length of the second historical period. The electronic device 230 can determine the first set of historical attributes of the traffic light (e.g., historical attribute 1 to historical attribute 7) within the first historical period, and determine the second set of historical attributes of the traffic light (e.g., historical attribute 1 to historical attribute 3) within the second historical period.

[0050] Based on the first historical time period and the predicted attribute 316, the electronic device 230 can determine the first set of scores for the first group of candidate attributes. Based on the second historical time period and the predicted attribute 316, the electronic device 230 can determine the second set of scores for the second group of candidate attributes. In some embodiments, the electronic device 230 can perform a weighted summation (318) of the first set of scores and the second set of scores to determine the scores of multiple candidate attributes. The weight corresponding to the first set of scores is less than the weight corresponding to the second set of scores.

[0051] The percentage score of a candidate attribute can be the ratio of the number of times the candidate attribute appears in the group of candidate attributes to which it belongs, `type_count`, to the total length of the group of candidate attributes, `total_count`. For example, taking candidate attribute A as an example, its first score in the first group of candidate attributes can be: `long_type_count / long_term_total_count`, where `long_type_count` is the number of times candidate attribute A appears in the first group of candidate attributes, and `long_term_total_count` is the total length of the first group of candidate attributes. Similarly, taking candidate attribute A as an example, its second score in the second group of candidate attributes can be: `short_type_count / short_term_total_count`, where `short_type_count` is the number of times candidate attribute A appears in the second group of candidate attributes, and `short_term_total_count` is the total length of the second group of candidate attributes. For example, assuming the length of the first group of candidate attributes is 8, and candidate attribute A appears 3 times in the first group of candidate attributes, then the first score of candidate attribute A is 3 / 8. Assuming the length of the second group of candidate attributes is 4, and candidate attribute A appears 3 times in the second group of candidate attributes, then the second score of candidate attribute A is 3 / 4.

[0052] Based on the first score and the second score of candidate attribute A, as well as the weights of the first set of scores and the corresponding weights of the second set of scores, the score of candidate attribute A can be determined. For example, the score of candidate attribute A can be:

[0053] score=long_type_count / long_term_total_count*LongTermScoreWeight+short_type_count / short_term_total_count*ShortTermScoreWeight.

[0054] Where LongTermScoreWeight is the weight corresponding to the first group of scores, and ShortTermScoreWeight is the weight corresponding to the second group of scores.

[0055] In some embodiments, the electronic device 230 may select a target candidate attribute 320 from multiple candidate attributes based on their scores. For example, the electronic device 230 may determine the candidate attribute with a score greater than a predetermined threshold as the target candidate attribute 320. Optionally or alternatively, the electronic device 230 may also determine the candidate attribute with the highest score among the multiple candidate attributes as the target candidate attribute 320.

[0056] In some embodiments, the electronic device 230 further determines the stability of the target attribute predicted at the current moment based on the attributes of historical time periods. If it is determined that the attributes of the traffic light in a continuous target time period are all the target attributes predicted at the current moment, or in other words, the target attribute predicted at the current moment is consistent with the attributes of the traffic light in a continuous target time period, the electronic device 230 determines the confidence level of the target attribute. In some embodiments, the target time period may include the current moment (i.e., the target moment) and the length of the target time period is greater than a threshold.

[0057] For example, electronic device 230 introduces a parameter called `same_prediction_times` to indicate the number of predictions. If the target attribute predicted at the current time is consistent with the attribute predicted at the previous time, then `same_prediction_times` is increased. This continues until `same_prediction_times` exceeds a preset threshold, at which point the confidence level of the target attribute is determined. In some embodiments, the confidence level can be the average score of the target attribute over the target time period. For example, `confidence = type_score_sum_ / same_prediction_times_`, where `type_score_sum_` is the sum of the scores predicted for all consecutive identical attributes.

[0058] In some embodiments, after determining the target candidate attribute 320, the electronic device 230 further determines (322) whether the type of the traffic light indicated by the target candidate attribute 320 is unknown. If the type of the traffic light is unknown, it means that the traffic light may be off or none of the cameras have detected the light color, so the attribute is not updated and the traffic light will use the attribute from the previous moment. If the type of the traffic light is not unknown, the electronic device 230 further determines (324) whether the distance between the vehicle 210 and the traffic light exceeds the set maximum distance threshold (MaxDist). This determination is to prevent the accuracy of attribute recognition from being too low due to excessive distance. If it is determined that the distance between the vehicle 210 and the traffic light does not exceed the maximum distance threshold, the target candidate attribute is determined as the target attribute 326.

[0059] In some embodiments, after determining the target attribute 326 of the traffic light, the electronic device 230 compares the target attribute 326 of the traffic light with the reference attribute 418 of the traffic light indicated by map data. If the target attribute 326 of the traffic light is determined to be different from the reference attribute, or if the target attribute of the traffic light is inconsistent with the reference attribute, the electronic device 230 re-determines the control strategy of the autonomous vehicle.

[0060] Figure 4A schematic diagram of an example process 400 for controlling an autonomous vehicle according to some embodiments of the present disclosure is shown. Figure 4 As shown, this example process 400 includes a perception module 410, a map module 420, and a planning module 430. In some embodiments, the perception module 410 may be... Figure 2 An example of electronic device 230 is shown.

[0061] In some embodiments, after determining the target attribute 424 of the traffic light (e.g., as referenced...), Figure 3 In the example process, given the determined target attribute 326, the perception module 410 first determines whether the target attribute 424 of the traffic light is consistent with the reference attribute 418. If the target attribute 424 is inconsistent with the reference attribute 418, to ensure the accuracy of the target attribute 424 and reduce the problem of map data being changed due to target attribute identification errors, the perception module 410 will not directly send an attribute change message 416 to the map module 420.

[0062] The sensing module 410 first determines whether the target attribute 424 meets preset conditions. In some embodiments, the preset conditions may be related to at least one of the following: the confidence level of the target attribute 424, the attribute jump ratio of the traffic light within a predetermined time period, and the duration of the target attribute 424.

[0063] For example, if the perception module 410 determines that the confidence level of the target attribute 424 is higher than a predetermined confidence threshold, the attribute jump ratio of the traffic light within a predetermined time period is lower than a predetermined ratio threshold, or the duration of the target attribute remains stable, the perception module 410 will determine that the target attribute 424 meets the preset conditions, and the target attribute 424 is a stable and usable result. In some embodiments, the attribute jump ratio jump_ratio is used to indicate the stability of attribute prediction. For example, a jump count variable jump_count_ can be determined. If the attribute identification result at the current moment is inconsistent with the attribute identification result at the previous moment, then jump_count_ is incremented by one. Then the jump ratio jump_ratio_ = jump_count_ / age_, where age_ is the total number of judgments.

[0064] In some embodiments, if the perception module 410 determines (414) that the confidence level of the target attribute 424 is high, such as higher than a predetermined confidence threshold, the perception module 410 will send an attribute change message 416 to the map module 420. The attribute change message 416 may include the identifier (signal_id) of the current traffic light, the target attribute 424 of the current traffic light, and the confidence level of the target attribute 424, etc.

[0065] In some embodiments, if the confidence level of the target attribute is determined to be less than or equal to a threshold, the perception module 410 sends a remote assistance request 412 related to the traffic light to a remote device, requesting the remote device to assist in confirming the target attribute 424. The perception module 410 receives confirmation information from the remote device and then determines whether to continue sending attribute change messages 416 to the map module 420.

[0066] The map module 420 receives an attribute change message 416 from the perception module 410 and determines whether to update the map data. For example, the map module 420 will decide whether to modify the lane information bound to a traffic light based on the attributes before and after the traffic light change. If the map module 420 determines to update the map data, it will send a change signal to the planning module 430. Upon receiving the change signal, the planning module 430 will generate a stop line to ensure that the autonomous vehicle stops before the stop line, and then execute the passage logic after the map module 420 updates the data. At the same time, based on the updated map data, the planning module 430 will regenerate the planned route 422.

[0067] According to embodiments of this disclosure, when the detected target attribute is inconsistent with the reference attribute indicated by the map data, stability verification is performed using confidence level and attribute jump ratio. Based on this result, the system chooses to automatically update the map data or request remote assistance, thereby achieving accurate processing of traffic light attribute changes and real-time updates of map data. Furthermore, it ensures rapid response and accurate decision-making of the autonomous driving system when facing changes in the actual environment.

[0068] Example process

[0069] Figure 5 A schematic diagram of a process 400 of an autonomous vehicle according to some embodiments of the present disclosure is shown. Process 500 can be implemented at electronic device 230. References below... Figure 1 Describe the process 500.

[0070] In frame 510, electronic device 230 determines prediction information corresponding to the target time based on multiple images collected by the autonomous vehicle. The prediction information indicates the prediction attributes of the traffic light.

[0071] In box 520, electronic device 230 updates prediction information based on the historical attribute information of the traffic light to determine the target attribute of the traffic light.

[0072] In box 530, electronic device 230 compares the target attributes of the traffic light with the reference attributes indicated by the map data.

[0073] In box 540, electronic device 230, in response to a target attribute differing from a reference attribute, determines a control strategy for the autonomous vehicle based on the confidence level of the target attribute.

[0074] In some embodiments, determining prediction information corresponding to a target time based on multiple images acquired by an autonomous vehicle includes: determining multiple classification results corresponding to the multiple images; determining weight information corresponding to the multiple classification results based on the confidence level of the multiple classification results and the size of the corresponding image region; and determining prediction information based on the weight information and the multiple classification results.

[0075] In some embodiments, updating prediction information based on historical attribute information of the traffic light to determine the target attribute of the traffic light includes: determining a set of historical attributes of the traffic light within at least one historical period based on the historical attribute information; determining scores of a plurality of candidate attributes based on the set of historical attributes and prediction attributes indicated by the prediction information, wherein the plurality of candidate attributes include prediction attributes and at least one attribute from the set of historical attributes; and determining the target attribute of the traffic light based on the scores of the plurality of candidate attributes.

[0076] In some embodiments, at least one historical period includes multiple historical periods corresponding to different time lengths.

[0077] In some embodiments, the multiple historical time periods include a first historical time period and a second historical time period, wherein the duration of the first historical time period is longer than that of the second historical time period. Determining the scores of multiple candidate attributes based on a set of historical attributes and predictive attributes indicated by predictive information includes: determining a first set of scores for a first set of candidate attributes based on the first historical time period and the predictive attributes; determining a second set of scores for a second set of candidate attributes based on the second historical time period and the predictive attributes; and determining the scores of the multiple candidate attributes based on a weighted sum of the first set of scores and the second set of scores, wherein the weight corresponding to the first set of scores is less than that of the second set of scores.

[0078] In some embodiments, determining a target attribute of a traffic light based on the scores of multiple candidate attributes includes: determining a target candidate attribute from the multiple candidate attributes, wherein the score of the target candidate attribute is greater than a threshold; determining the distance from the autonomous vehicle to the traffic light in response to the target candidate attribute indicating that the type of the traffic light is unknown; and determining the target candidate attribute as the target attribute in response to the distance being less than a threshold.

[0079] In some embodiments, process 500 further includes: in response to the fact that the attributes of the traffic light are all target attributes in a continuous target time period, determining the confidence level of the target attribute, wherein the target time period includes the target time and the length of the target time period is greater than a threshold.

[0080] In some embodiments, determining the confidence level of a target attribute includes: determining the confidence level of the target attribute based on the average score of the target attribute of the traffic light during the target time period.

[0081] In some embodiments, comparing the target attribute of a traffic light with a reference attribute indicated by map data includes: in response to the target attribute of the traffic light satisfying a preset condition, comparing the target attribute of the traffic light with the reference attribute indicated by map data. The preset condition relates to at least one of the following: the confidence level of the target attribute, the percentage change in the attribute of the traffic light within a predetermined time period, and the duration of the target attribute.

[0082] In some embodiments, determining the control strategy of an autonomous vehicle based on the confidence level of a target attribute includes: updating map data based on the target attribute in response to a confidence level exceeding a threshold; and generating a stop line corresponding to a traffic light to wait for the map data update to complete.

[0083] In some embodiments, determining the control strategy of an autonomous vehicle based on the confidence level of a target attribute includes: sending an assistance request related to a traffic light to a remote device in response to a confidence level that is less than or equal to a threshold.

[0084] Example devices and equipment

[0085] Figure 6 A schematic structural block diagram of a device 600 for an autonomous vehicle according to certain embodiments of the present disclosure is shown. The device 600 may be implemented as or included in an electronic device 230. Various modules / components in the device 600 may be implemented by hardware, software, firmware, or any combination thereof.

[0086] As shown in the figure, the device 600 includes a first determining module 610, configured to determine prediction information corresponding to a target time based on multiple images collected by the autonomous vehicle, wherein the prediction information indicates the prediction attribute of the traffic light; an updating module 620, which updates the prediction information based on the historical attribute information of the traffic light to determine the target attribute of the traffic light; a comparison module 630, configured to compare the target attribute of the traffic light with the reference attribute indicated by map data; and a second determining module 640, configured to determine the control strategy of the autonomous vehicle based on the confidence level of the target attribute in response to the target attribute being different from the reference attribute.

[0087] In some embodiments, the first determining module 610 is further configured to: determine multiple classification results corresponding to multiple images; determine weight information corresponding to multiple classification results based on the confidence level of the multiple classification results and the size of the corresponding image region; and determine prediction information based on the weight information and the multiple classification results.

[0088] In some embodiments, the update module 620 is further configured to: determine a set of historical attributes of the traffic light within at least one historical period based on historical attribute information; determine scores of a plurality of candidate attributes based on the set of historical attributes and predicted attributes indicated by prediction information, wherein the plurality of candidate attributes include predicted attributes and at least one attribute from the set of historical attributes; and determine a target attribute of the traffic light based on the scores of the plurality of candidate attributes.

[0089] In some embodiments, at least one historical period includes multiple historical periods corresponding to different time lengths.

[0090] In some embodiments, the multiple historical time periods include a first historical time period and a second historical time period, wherein the duration of the first historical time period is longer than that of the second historical time period. The update module 620 is further configured to: determine a first set of scores for a first group of candidate attributes based on the first historical time period and the predicted attribute; determine a second set of scores for a second group of candidate attributes based on the second historical time period and the predicted attribute; and determine the scores of multiple candidate attributes based on a weighted sum of the first and second set of scores, wherein the weight corresponding to the first set of scores is less than that of the second set of scores.

[0091] In some embodiments, the update module 620 is further configured to: determine a target candidate attribute from a plurality of candidate attributes, wherein the score of the target candidate attribute is greater than a threshold; determine the distance from the autonomous vehicle to the traffic light in response to the target candidate attribute indicating that the type of the traffic light is unknown; and determine the target candidate attribute as the target attribute in response to the distance being less than the threshold.

[0092] In some embodiments, the apparatus 600 further includes a third determining module configured to determine the confidence level of the target attribute in response to the fact that the attributes of the traffic light are all target attributes in a continuous target time period, wherein the target time period includes the target time and the length of the target time period is greater than a threshold.

[0093] In some embodiments, the third determining module is further configured to: determine the confidence level of the target attribute based on the average score of the target attribute of the traffic light during the target time period.

[0094] In some embodiments, the comparison module 630 is further configured to: compare the target attribute of the traffic light with a reference attribute indicated by map data in response to the target attribute of the traffic light meeting a preset condition. The preset condition relates to at least one of the following: the confidence level of the target attribute, the attribute jump ratio of the traffic light within a predetermined time period, and the duration of the target attribute.

[0095] In some embodiments, the second determining module 640 is further configured to: update map data based on target attributes in response to a confidence level higher than a threshold; and generate a stop line corresponding to a traffic light to wait for the map data update to complete.

[0096] In some embodiments, the second determining module 640 is further configured to send an assistance request related to the traffic light to a remote device in response to a confidence level that is lower than or equal to a threshold.

[0097] Figure 7 A block diagram is shown illustrating a computing device 700 in which one or more embodiments of the present disclosure may be implemented. It should be understood that... Figure 7 The computing device 700 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 7 The computing device 700 shown can be used to implement Figure 2 230 electronic devices.

[0098] like Figure 7 As shown, computing device 700 is in the form of a general-purpose computing device. Components of computing device 700 may include, but are not limited to, one or more processors or processing units 710, memory 720, storage devices 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. Processing unit 710 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 720. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of computing device 700.

[0099] Computing device 700 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to computing device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 720 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 730 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within computing device 700.

[0100] The computing device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 7As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 720 may include computer program product 725 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0101] The communication unit 740 enables communication with other computing devices via a communication medium. Additionally, the components of the computing device 700 can function as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the computing device 700 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or another network node.

[0102] Input device 750 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 760 can be one or more output devices, such as a monitor, speaker, printer, etc. Computing device 700 can also communicate as needed with one or more external devices (not shown) via communication unit 740. These external devices, such as storage devices, display devices, etc., can communicate with one or more devices that enable user interaction with computing device 700, or with any device (e.g., network card, modem, etc.) that enables computing device 700 to communicate with one or more other computing devices. Such communication can be performed via input / output (I / O) interface (not shown).

[0103] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0104] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0105] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0106] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0108] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for an autonomous vehicle, comprising: Based on multiple images collected by the autonomous vehicle, predictive information corresponding to the target time is determined, and the predictive information indicates the predictive attributes of the traffic light. Based on the historical attribute information of the traffic light, the prediction information is updated to determine the target attribute of the traffic light; Compare the target attributes of the traffic light with the reference attributes indicated by the map data; as well as In response to the target attribute differing from the reference attribute, a control strategy for the autonomous vehicle is determined based on the confidence level of the target attribute.

2. The method according to claim 1, wherein determining the prediction information corresponding to the target time based on multiple images acquired by the autonomous vehicle includes: Determine multiple classification results corresponding to the multiple images; Based on the confidence scores of the multiple classification results and the corresponding image region sizes, weight information corresponding to the multiple classification results is determined. as well as The prediction information is determined based on the weight information and the multiple classification results.

3. The method according to claim 1, wherein updating the prediction information based on the historical attribute information of the traffic light to determine the target attribute of the traffic light includes: Based on the historical attribute information, a set of historical attributes of the traffic light within at least one historical period are determined; Based on the set of historical attributes and the predicted attributes indicated by the predicted information, scores for multiple candidate attributes are determined, wherein the multiple candidate attributes include at least one attribute from the predicted attributes and the set of historical attributes. as well as The target attribute of the traffic light is determined based on the scores of the multiple candidate attributes.

4. The method according to claim 3, wherein at least one historical period includes multiple historical periods corresponding to different time lengths.

5. The method of claim 4, wherein the plurality of historical time periods includes a first historical time period and a second historical time period, the first historical time period having a longer duration than the second historical time period, and determining the scores of the plurality of candidate attributes based on the set of historical attributes and the prediction attributes indicated by the prediction information includes: Based on the first historical period and the predicted attribute, determine the first set of scores for the first set of candidate attributes; Based on the second historical period and the predicted attributes, a second set of scores for the second set of candidate attributes is determined; as well as The scores of the plurality of candidate attributes are determined based on the weighted sum of the scores of the first group and the scores of the second group, wherein the weight corresponding to the scores of the first group is less than that of the scores of the second group.

6. The method of claim 3, wherein determining the target attribute of the traffic light based on the scores of the plurality of candidate attributes comprises: A target candidate attribute is determined from the plurality of candidate attributes, wherein the score of the target candidate attribute is greater than a threshold. In response to the target candidate attribute indicating that the type of the traffic light is unknown, the distance from the autonomous vehicle to the traffic light is determined; as well as In response to the distance being less than a threshold, the target candidate attribute is determined as the target attribute.

7. The method according to claim 1, further comprising: In response to the fact that the attributes of the traffic light are all the target attributes during a continuous target time period, the confidence level of the target attribute is determined, wherein the target time period includes the target time and the length of the target time period is greater than a threshold.

8. The method of claim 7, wherein determining the confidence level of the target attribute comprises: The confidence level of the target attribute is determined based on the average score of the target attribute of the traffic light during the target time period.

9. The method of claim 1, wherein comparing the target attribute of the traffic light with the reference attribute indicated by the map data comprises: In response to the target attribute of the traffic light satisfying a preset condition, the target attribute of the traffic light is compared with the reference attribute indicated by the map data. The preset conditions are related to at least one of the following: the confidence level of the target attribute, the attribute jump ratio of the traffic light within a predetermined time period, and the duration of the target attribute.

10. The method of claim 1, wherein determining the control strategy of the autonomous vehicle based on the confidence level of the target attribute comprises: In response to the confidence level being higher than the threshold, the map data is updated based on the target attribute; as well as A stop line corresponding to the traffic light is generated to wait for the map data update to complete.

11. The method of claim 1, wherein determining the control strategy of the autonomous vehicle based on the confidence level of the target attribute comprises: In response to the confidence level being lower than or equal to a threshold, an assistance request related to the traffic light is sent to a remote device.

12. An apparatus for an autonomous vehicle, comprising: The first determining module is configured to determine prediction information corresponding to the target time based on multiple images collected by the autonomous vehicle, wherein the prediction information indicates the prediction attribute of the traffic light. The update module updates the prediction information based on the historical attribute information of the traffic light to determine the target attribute of the traffic light; The comparison module is configured to compare the target attributes of the traffic light with the reference attributes indicated by the map data; as well as The second determining module is configured to determine the control strategy of the autonomous vehicle based on the confidence level of the target attribute in response to the target attribute being different from the reference attribute.

13. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 11.

15. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 11.

Images