Traffic light determination method and device for autonomous vehicle, electronic equipment and computer program product

Abstract

The invention discloses a traffic light determination method and device for an autonomous vehicle, electronic equipment and a computer program product, and the method comprises the steps: obtaining and detecting a current road image collected by the autonomous vehicle, and obtaining a traffic light detection result; acquiring lane center line data of a road where the vehicle is currently located; according to the traffic light detection result and the lane center line data, determining a corresponding relation between the lane of the current road and the traffic light by using a corresponding relation generation model between the lane and the traffic light; and according to the current lane of the vehicle, the navigation route information and the corresponding relationship between the lane and the traffic light, determining the light state information of the current corresponding target traffic light of the vehicle. The corresponding relation between the lane of the road where the vehicle is currently located and the traffic light can be dynamically generated in real time, the state information of the traffic light concerned by the vehicle is further determined by combining the lane where the vehicle is currently located and the driving route information, and a more accurate basis is provided for driving decision making.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a traffic light determination method and device for an autonomous driving vehicle, an electronic device, and a computer program product. Background Art

[0002] In an autonomous driving scenario, when an autonomous driving vehicle drives to an intersection, it is necessary to identify the traffic light information ahead from the image captured by the vehicle's camera, and then determine the correspondence between the autonomous driving vehicle and the traffic light, so that the autonomous driving vehicle can select the corresponding traffic light information based on its current location and make correct planning decisions.

[0003] The current solution is to use manual or semi-manual methods to draw the correspondence between traffic lights and lanes in high-precision maps in advance, and then detect traffic lights through target detection algorithms during vehicle driving. The current vehicle position is obtained based on the positioning device, and the center line of the lane where the vehicle is located is determined based on the vehicle position. The position of the traffic light to be used by the current vehicle is found based on the lane centerline. Based on the obtained traffic light position, the internal and external parameters of the vehicle camera are used to project the traffic light coordinates onto the image to find the traffic light closest to the vehicle, thereby determining the corresponding traffic light state information when the current vehicle reaches the intersection.

[0004] However, the above solution has at least the following technical problems:

[0005] 1) Drawing the correspondence relationship manually or semi-manually is costly and slow, and the accuracy of the manual drawing results is difficult to measure;

[0006] 2) To obtain the final light status information, the vehicle needs to go through two traditional algorithm matches, which is closely related to the accuracy of lane centerline drawing and camera parameter calibration. This places higher demands on the accuracy of each link. If any link is not accurate enough, it may lead to the wrong light status of the vehicle.

[0007] 3) After the road on which the vehicle is traveling changes, the correspondence between the traffic lights and lanes needs to be manually redrawn. This method is not universal, and the correspondence cannot be generated in real time. It needs to be manually drawn in advance, which has poor timeliness. Summary of the invention

[0008] Embodiments of the present application provide a traffic light determination method and device, an electronic device, and a computer program product for an autonomous driving vehicle to improve the accuracy and real-time performance of traffic light determination in an autonomous driving scenario.

[0009] The present application embodiment adopts the following technical solutions:

[0010] In a first aspect, an embodiment of the present application provides a traffic light determination method for an autonomous driving vehicle, wherein the traffic light determination method for an autonomous driving vehicle includes:

[0011] Acquire a current road image collected by the autonomous driving vehicle, and detect traffic lights in the current road image to obtain a traffic light detection result;

[0012] Obtain lane centerline data of the road the autonomous vehicle is currently on;

[0013] According to the traffic light detection result and the lane centerline data of the road where the autonomous driving vehicle is currently located, a model for generating a correspondence between lanes and traffic lights is used to determine a correspondence between the lanes and traffic lights of the road where the autonomous driving vehicle is currently located;

[0014] According to the lane in which the autonomous driving vehicle is currently located, navigation route information, and the correspondence between the lane and the traffic light, the target traffic light currently corresponding to the autonomous driving vehicle and the light state information of the target traffic light are determined, so as to control the driving of the autonomous driving vehicle at the intersection according to the light state information of the target traffic light.

[0015] Optionally, the lane-to-traffic light correspondence generation model is trained in the following manner:

[0016] Acquire a current road image collected by the autonomous driving vehicle, and detect traffic lights in the current road image to obtain a traffic light detection result;

[0017] Obtain lane centerline data of the road the autonomous vehicle is currently on;

[0018] Constructing training sample data of a model for generating a correspondence between lanes and traffic lights according to the traffic light detection result and the lane centerline data;

[0019] The lane-traffic-light correspondence generation model is trained using the training sample data of the lane-traffic-light correspondence generation model to obtain a trained lane-traffic-light correspondence generation model.

[0020] Optionally, obtaining lane centerline data of a road where the autonomous driving vehicle is currently located includes:

[0021] Get the current position of the autonomous vehicle;

[0022] Acquire corresponding high-precision map data according to the current position of the autonomous driving vehicle;

[0023] The lane centerline data of the road where the autonomous driving vehicle is currently located is obtained from the high-precision map data corresponding to the current position of the autonomous driving vehicle.

[0024] Optionally, the training sample data for constructing a model for the correspondence between lanes and traffic lights according to the traffic light detection result and the lane centerline data includes:

[0025] Encoding the traffic light detection result to obtain a traffic light encoding vector;

[0026] Encoding the lane centerline data to obtain a lane centerline encoding vector;

[0027] The traffic light coding vector and the lane centerline coding vector are fused to obtain training sample data for generating a model for the correspondence relationship between the lane and the traffic light.

[0028] Optionally, encoding the lane centerline data to obtain a lane centerline encoding vector includes:

[0029] Preprocessing the lane centerline data using a preset lane line preprocessing strategy to obtain preprocessed lane centerline data;

[0030] The preprocessed lane centerline data is encoded to obtain a lane centerline encoding vector.

[0031] Optionally, the using the training sample data of the lane-traffic light correspondence generation model to train the lane-traffic light correspondence generation model includes:

[0032] Inputting the training sample data of the lane-traffic light correspondence generation model into a multi-layer perceptron to obtain the correspondence between the lane and the traffic light output by the multi-layer perceptron;

[0033] Calculating a loss value based on the correspondence between lanes and traffic lights output by the multi-layer perceptron and the correspondence between pre-labeled lanes and traffic lights;

[0034] The loss value is used to update the parameters of the multilayer perceptron to obtain a trained lane-traffic light correspondence generation model.

[0035] Optionally, the using the training sample data of the lane-traffic light correspondence generation model to train the lane-traffic light correspondence generation model includes:

[0036] Inputting the training sample data into the corresponding relationship generation model between the lane and the traffic light;

[0037] Based on the preset output structure of the correspondence relationship between the lanes and the traffic lights, determining whether the number of traffic lights and the number of lane center lines corresponding to the training sample data are less than the corresponding preset numbers;

[0038] When the number of traffic lights and / or lane center lines is less than the corresponding preset number, a fill-in vector is used to supplement and a road correspondence matrix consisting of the preset number of lane center lines multiplied by the preset number of traffic lights is output.

[0039] In a second aspect, an embodiment of the present application further provides a traffic light determination device for an autonomous driving vehicle, wherein the traffic light determination device for an autonomous driving vehicle comprises:

[0040] A traffic light detection unit, used to obtain a current road image collected by the autonomous driving vehicle, and detect traffic lights in the current road image to obtain a traffic light detection result;

[0041] An acquisition unit, used to acquire lane centerline data of a road where the autonomous driving vehicle is currently located;

[0042] a first determination unit, configured to determine a correspondence between a lane and a traffic light of a road where the autonomous driving vehicle is currently located, based on the traffic light detection result and lane centerline data of the road where the autonomous driving vehicle is currently located, by using a lane-traffic light correspondence generation model;

[0043] The second determination unit is used to determine the target traffic light currently corresponding to the autonomous driving vehicle and the light state information of the target traffic light based on the lane in which the autonomous driving vehicle is currently located, navigation route information, and the correspondence between the lane and the traffic light, so as to control the driving of the autonomous driving vehicle at the intersection according to the light state information of the target traffic light.

[0044] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0045] Processor; and

[0046] A memory arranged to store computer executable instructions, which when executed cause the processor to perform any of the aforementioned traffic light determination methods for an autonomous driving vehicle.

[0047] In a fourth aspect, an embodiment of the present application further provides a computer program product, comprising a computer program or instructions, which, when executed by a processor, implements any of the aforementioned traffic light determination methods for an autonomous driving vehicle.

[0048] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: the traffic light determination method for an autonomous driving vehicle in the embodiments of the present application first obtains the current road image collected by the autonomous driving vehicle, and detects the traffic lights in the current road image to obtain the traffic light detection result; obtains the lane centerline data of the road where the autonomous driving vehicle is currently located; according to the traffic light detection result and the lane centerline data of the road where the autonomous driving vehicle is currently located, a model is generated based on the correspondence between the lane and the traffic light to determine the correspondence between the lane and the traffic light of the road where the autonomous driving vehicle is currently located; according to the lane where the autonomous driving vehicle is currently located, the navigation route information, and the correspondence between the lane and the traffic light, the target traffic light currently corresponding to the autonomous driving vehicle and the light state information of the target traffic light are determined, so as to control the driving of the autonomous driving vehicle at the intersection according to the light state information of the target traffic light. The traffic light determination method for an autonomous driving vehicle in the embodiments of the present application can dynamically generate the correspondence between the lane and the traffic light of the road where the vehicle is currently located in real time, and further determine the light state information of the traffic light that the vehicle should pay attention to in combination with the lane where the vehicle is currently located and the driving route information, providing a more accurate basis for driving decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0050] Figure 1 A flow chart of a method for determining a traffic light for an autonomous driving vehicle in an embodiment of the present application;

[0051] Figure 2 This is a schematic diagram of the structure of a traffic light determination device for an autonomous driving vehicle in an embodiment of the present application;

[0052] Figure 3 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0054] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0055] The present application embodiment provides a traffic light determination method for an autonomous driving vehicle, such as Figure 1 As shown, a flow chart of a method for determining a traffic light for an autonomous driving vehicle in an embodiment of the present application is provided. The method for determining a traffic light for an autonomous driving vehicle at least includes the following steps S110 to S140:

[0056] Step S110, obtaining a current road image collected by the autonomous driving vehicle, and detecting traffic lights in the current road image to obtain a traffic light detection result.

[0057] When determining the traffic light information that the autonomous driving vehicle currently needs to pay attention to, the embodiment of the present application needs to first obtain a road image collected by the autonomous driving vehicle, and then use a traffic light detection algorithm to detect the traffic lights in the road image to obtain light state information such as the position of the traffic light detection frame, the color of the traffic lights, and the attributes of the traffic lights. The specific traffic light detection algorithm to be used can be flexibly selected by those skilled in the art according to actual needs and is not specifically limited here.

[0058] Step S120, obtaining lane centerline data of the road where the autonomous driving vehicle is currently located.

[0059] The embodiment of the present application also needs to obtain the lane centerline data of the road where the autonomous driving vehicle is currently located. Each lane corresponds to a unique lane centerline. Here, it is necessary to obtain the lane centerline data of all lanes contained in the current road section.

[0060] Step S130, based on the traffic light detection result and the lane centerline data of the road where the autonomous driving vehicle is currently located, a model for generating the correspondence between lanes and traffic lights is used to determine the correspondence between the lanes and traffic lights of the road where the autonomous driving vehicle is currently located.

[0061] The traffic light detection results obtained in the above steps include all the traffic light information detected in real time in front of the road where the autonomous driving vehicle is currently located, and the lane centerline data also includes the centerline information of all lanes on the road where the autonomous driving vehicle is currently located. Therefore, it is necessary to further determine the correspondence between all lanes and all detected traffic lights.

[0062] The embodiment of the present application has trained the lane-traffic light correspondence generation model in advance, and inputs the above traffic light detection results and the lane centerline data of the road where the autonomous driving vehicle is currently located into the lane-traffic light correspondence generation model, so that the lane-traffic light correspondence of the road where the autonomous driving vehicle is currently located can be automatically generated. Since the correspondence here is generated in real time and dynamically based on the actual detection results, it can flexibly adapt to the changes in the intersection scene and lane, thereby improving the accuracy of the subsequent determination of the traffic light state information.

[0063] Step S140, based on the lane the autonomous driving vehicle is currently in, navigation route information, and the correspondence between the lane and the traffic light, determine the target traffic light that the autonomous driving vehicle currently corresponds to and the light state information of the target traffic light, so as to control the driving of the autonomous driving vehicle at the intersection according to the light state information of the target traffic light.

[0064] Taking into account that in actual road scenarios, there is not a one-to-one correspondence between lanes and traffic lights, and a lane may allow straight driving, left turns, or even right turns at the same time, and for an autonomous vehicle, what it needs to pay most attention to is the traffic lights ahead of its planned driving route. Therefore, after determining the correspondence between the lanes and traffic lights of the current road, the embodiment of the present application needs to determine the lane in which the autonomous vehicle is currently located based on the current position of the autonomous vehicle, and then determine which traffic lights correspond to the current lane based on the correspondence between the lanes and traffic lights of the current road, and then determine in combination with the navigation route of the autonomous vehicle whether the autonomous vehicle will go straight, turn left, make a U-turn, or turn right at the next intersection, and finally determine the target traffic light that the autonomous vehicle is paying attention to, and then obtain the light state data of the target traffic light for the autonomous vehicle to make correct decision planning.

[0065] The traffic light determination method for an autonomous driving vehicle in an embodiment of the present application can dynamically generate a correspondence between the lane of the road where the vehicle is currently located and the traffic light in real time, and further determine the traffic light state information that the vehicle should pay attention to in combination with the vehicle's current lane and driving route information, thereby providing a more accurate basis for driving decisions.

[0066] In some embodiments of the present application, the lane-traffic light correspondence generation model is trained in the following manner: obtaining a current road image collected by an autonomous driving vehicle, and detecting traffic lights in the current road image to obtain traffic light detection results; obtaining lane centerline data of the road where the autonomous driving vehicle is currently located; constructing training sample data for the lane-traffic light correspondence generation model based on the traffic light detection results and the lane centerline data; training the lane-traffic light correspondence generation model using the training sample data of the lane-traffic light correspondence generation model to obtain a trained lane-traffic light correspondence generation model.

[0067] When training a model for generating the correspondence between lanes and traffic lights, the embodiment of the present application needs to first obtain a road image captured by the autonomous driving vehicle, and then use a traffic light detection algorithm to detect the traffic lights in the road image to obtain light state information such as the position of the traffic light detection frame, the color and attributes of the traffic lights. The specific traffic light detection algorithm to be used can be flexibly selected by those skilled in the art according to actual needs and is not specifically limited here.

[0068] The embodiment of the present application also needs to obtain the lane centerline data of the road where the autonomous driving vehicle is currently located. Each lane corresponds to a unique lane centerline. Here, it is necessary to obtain the lane centerline data of all lanes contained in the current road section.

[0069] Based on the above steps, it is equivalent to obtaining all the traffic light data and all the lane centerline data that can be detected on the road section where the autonomous driving vehicle is located. In order to learn the intrinsic connection between the traffic light data and the lane centerline data, it is necessary to combine all the traffic light detection results that can be detected on the road section where the autonomous driving vehicle is located with the corresponding all lane centerline data to construct training sample data for training the lane and traffic light correspondence generation model.

[0070] After the training sample data is constructed, the pre-selected basic network structure can be iteratively trained using the training sample data. When the training effect of the model meets the accuracy requirements, the iteration is stopped and the trained lane-traffic light correspondence generation model is output. The basic network structure here can be flexibly selected in combination with existing technologies, for example, a multi-layer perceptron network structure can be used. The specific selection is not specifically limited here.

[0071] The traffic light determination method for an autonomous driving vehicle in an embodiment of the present application utilizes traffic light detection results and lane centerline data to construct training samples for training a lane-traffic light correspondence generation model. The trained lane-traffic light correspondence generation model can be directly used for real-time generation of correspondences without relying on manual drawing, thereby improving the real-time and accuracy of road correspondence generation.

[0072] In some embodiments of the present application, obtaining lane centerline data of the road where the autonomous driving vehicle is currently located includes: obtaining the current position of the autonomous driving vehicle; obtaining corresponding high-precision map data based on the current position of the autonomous driving vehicle; and obtaining lane centerline data of the road where the autonomous driving vehicle is currently located from the high-precision map data corresponding to the current position of the autonomous driving vehicle.

[0073] High-precision map data, also known as high-definition map data (HD MapData), is a digital map that provides accurate and detailed road information and environmental perception data based on advanced surveying and mapping technology and high-precision positioning data. This data not only contains the basic geographic information of traditional maps, such as road networks and directions, but also provides detailed data such as lane-level geometry, traffic rules, traffic signs, and traffic flow.

[0074] As an important element in high-precision maps, lane centerline data is crucial for autonomous driving systems. Lane centerline not only represents the geometric centerline of the lane, but also contains attribute information such as position sequence and lane ID. This information is recorded in detail in high-precision maps, allowing the autonomous driving system to accurately identify the lane where the vehicle is located and perform path planning and driving control based on the lane centerline.

[0075] Based on this, the embodiments of the present application can use high-precision map data as a source for obtaining lane centerline data. Since the amount of global high-precision map data is very large, the embodiments of the present application can combine the current position of the autonomous driving vehicle to determine the local high-precision map corresponding to the current road section, and obtain the lane centerline data of the current road section of the autonomous driving vehicle.

[0076] Although the lane centerline data of the actual road may change, the lane centerline data provided in the HD map data has not been updated in time, resulting in the lane centerline data in the HD map not being consistent with the actual situation. However, during the training phase, targeted sections where the lanes have not changed can be selected for data sample collection. This method can improve the efficiency of training data collection to a certain extent, thereby improving the efficiency of model training.

[0077] In some embodiments of the present application, constructing the training sample data of the lane-traffic light correspondence generation model based on the traffic light detection result and the lane centerline data includes: encoding the traffic light detection result to obtain a traffic light encoding vector; encoding the lane centerline data to obtain a lane centerline encoding vector; fusing the traffic light encoding vector and the lane centerline encoding vector to obtain the training sample data of the lane-traffic light correspondence generation model.

[0078] When constructing training sample data, in order to meet the data structure requirements of subsequent model input, the traffic light detection results and lane centerline data can be encoded and processed separately to obtain traffic light encoding vectors and lane centerline encoding vectors.

[0079] For example, the traffic light detection result may include the coordinates of the traffic light detection frame, the color of the traffic light such as red, green, yellow, black, and others, and the direction of the traffic light such as straight ahead, left turn, right turn, and U-turn. Therefore, the coordinates of the traffic light detection frame can be encoded into a 4-dimensional vector, the color of the traffic light can be encoded into a 5-dimensional vector, and the direction of the traffic light can be encoded into a 4-dimensional vector. These encoded traffic light feature vectors are concatenated to obtain a 13-dimensional vector as the final traffic light encoding vector.

[0080] It should be noted that those skilled in the art can flexibly select the specific encoding method according to actual needs. For example, it can be implemented by One-Hot encoding and other methods, which are not specifically limited here.

[0081] In some embodiments of the present application, encoding the lane centerline data to obtain a lane centerline encoding vector includes: preprocessing the lane centerline data using a preset lane line preprocessing strategy to obtain preprocessed lane centerline data; encoding the preprocessed lane centerline data to obtain a lane centerline encoding vector.

[0082] In order to meet the requirements of subsequent model training, the embodiment of the present application can first pre-process the lane centerline data obtained from the high-precision map. The pre-processing here can include first converting the UTM coordinates of the high-precision map data to the vehicle body coordinate system, and then using Bezier curves or Douglas-Peucker algorithm to process the converted lane centerline to obtain processed lane centerline data, and finally encoding the processed lane centerline data into a vector. Of course, the specific pre-processing operations can be flexibly set by those skilled in the art according to actual needs, and are not specifically limited here.

[0083] In some embodiments of the present application, the use of the training sample data of the lane-traffic light correspondence generation model to train the lane-traffic light correspondence generation model includes: inputting the training sample data of the lane-traffic light correspondence generation model into a multilayer perceptron to obtain the correspondence between the lanes and the traffic lights output by the multilayer perceptron; calculating the loss value based on the correspondence between the lanes and the traffic lights output by the multilayer perceptron and the pre-labeled correspondence between the lanes and the traffic lights; and using the loss value to update the parameters of the multilayer perceptron to obtain the trained lane-traffic light correspondence generation model.

[0084] The basic network structure used in the embodiment of the present application to train the corresponding relationship generation model between lanes and traffic lights adopts a multilayer perceptron (MLP for short). The multilayer perceptron is a feedforward artificial neural network model that introduces one or more hidden layers on the basis of a single-layer neural network, so that the neural network has multiple network layers, and the output of the hidden layer is converted by an activation function. The number of network layers of the multilayer perceptron set in the embodiment of the present application can be dynamically adjusted according to the actual training situation. For example, a maximum number of network layers can be set, and the number of network layers taken can be gradually reduced to evaluate the effect of the model, thereby obtaining the optimal network structure.

[0085] During specific training, the training sample data constructed above is input into the multi-layer perceptron, and the correspondence between lanes and traffic lights output by the multi-layer perceptron can be obtained. Then, the loss value of the model output result is calculated in combination with the correspondence between lanes and traffic lights marked in advance. The specific loss function used can be, for example, a cross entropy loss function, which is not specifically limited here. Finally, the parameters of the multi-layer perceptron are iteratively updated using the loss value until a lane-traffic light correspondence generation model that meets the requirements is obtained.

[0086] In some embodiments of the present application, the method of training the lane-traffic light correspondence generation model using the training sample data of the lane-traffic light correspondence generation model includes: inputting the training sample data into the lane-traffic light correspondence generation model; based on the preset output structure of the lane-traffic light correspondence generation model, determining whether the number of traffic lights and the number of lane center lines corresponding to the training sample data are less than the corresponding preset number; in the case that the number of traffic lights and / or the number of lane center lines are less than the corresponding preset number, supplementing with a padding vector, and outputting a road correspondence matrix formed by multiplying the preset number of lane center lines by the preset number of traffic lights.

[0087] Considering that the number of traffic lights and lanes deployed on different road sections are not necessarily the same, and there is not a one-to-one correspondence between traffic lights and lanes, for example, some road sections are deployed with 3 lanes and 3 traffic lights, and some road sections are deployed with 4 lanes and 3 traffic lights.

[0088] Based on this, in order to make the lane-traffic light correspondence generation model trained in the embodiment of the present application applicable to various road scenarios, when training the lane-traffic light correspondence generation model, the output data structure of the lane-traffic light correspondence generation model is defined in advance. The output data structure here can be understood as a unified and universal data structure set according to the lane deployment conditions and traffic light deployment conditions of various road scenarios. For example, it can be expressed as: a matrix of the preset number of traffic lights × the preset number of lane center lines. The preset number of traffic lights and the preset number of lane center lines can be set to a larger value according to the actual road conditions, so as to cover and include the number of traffic lights and the number of lanes in all road scenarios.

[0089] When processing the training sample data, since the training sample data is input data consisting of traffic light detection results and lane centerline data obtained based on actual scenarios, the number of traffic lights actually detected and the number of lane centerlines obtained may not necessarily reach the maximum number set above. For example, the preset number is 10 traffic lights and 10 lanes, and 3 traffic lights and 4 lanes are actually detected. In this case, the insufficient part can be filled with a padding vector to ensure that the final output result is still a 10×10 matrix.

[0090] The present application also provides a traffic light determination device 200 for an autonomous driving vehicle. Figure 2 As shown, a schematic diagram of the structure of a traffic light determination device for an autonomous driving vehicle in an embodiment of the present application is provided. The traffic light determination device 200 for an autonomous driving vehicle includes:

[0091] The traffic light detection unit 210 is used to obtain a current road image collected by the autonomous driving vehicle, and detect traffic lights in the current road image to obtain a traffic light detection result;

[0092] An acquisition unit 220 is used to acquire lane centerline data of a road where the autonomous driving vehicle is currently located;

[0093] A first determination unit 230 is used to determine the correspondence between the lane and the traffic light of the road where the autonomous driving vehicle is currently located, based on the traffic light detection result and the lane centerline data of the road where the autonomous driving vehicle is currently located, by using a lane-traffic light correspondence generation model;

[0094] The second determination unit 240 is used to determine the target traffic light currently corresponding to the autonomous driving vehicle and the light state information of the target traffic light according to the lane in which the autonomous driving vehicle is currently located, navigation route information, and the correspondence between the lane and the traffic light, so as to control the driving of the autonomous driving vehicle at the intersection according to the light state information of the target traffic light.

[0095] In some embodiments of the present application, the traffic light determination device for an autonomous driving vehicle also includes a training unit, which is specifically used to: obtain a current road image collected by the autonomous driving vehicle, and detect traffic lights in the current road image to obtain traffic light detection results; obtain lane centerline data of the road where the autonomous driving vehicle is currently located; construct training sample data of a lane-traffic light correspondence generation model based on the traffic light detection results and the lane centerline data; train the lane-traffic light correspondence generation model using the training sample data of the lane-traffic light correspondence generation model to obtain a trained lane-traffic light correspondence generation model.

[0096] In some embodiments of the present application, the training unit is also used to: obtain the current position of the autonomous driving vehicle; obtain corresponding high-precision map data based on the current position of the autonomous driving vehicle; and obtain lane centerline data of the road where the autonomous driving vehicle is currently located from the high-precision map data corresponding to the current position of the autonomous driving vehicle.

[0097] In some embodiments of the present application, the training unit is also used to: encode the traffic light detection result to obtain a traffic light encoding vector; encode the lane centerline data to obtain a lane centerline encoding vector; fuse the traffic light encoding vector and the lane centerline encoding vector to obtain training sample data for generating a model for the correspondence relationship between the lane and the traffic light.

[0098] In some embodiments of the present application, the training unit is also used to: preprocess the lane centerline data using a preset lane line preprocessing strategy to obtain preprocessed lane centerline data; and encode the preprocessed lane centerline data to obtain a lane centerline encoding vector.

[0099] In some embodiments of the present application, the training unit is also used to: input the training sample data of the lane-traffic light correspondence generation model into a multilayer perceptron to obtain the correspondence between the lanes and traffic lights output by the multilayer perceptron; calculate the loss value based on the correspondence between the lanes and traffic lights output by the multilayer perceptron and the correspondence between the pre-labeled lanes and traffic lights; use the loss value to update the parameters of the multilayer perceptron to obtain the trained lane-traffic light correspondence generation model.

[0100] In some embodiments of the present application, the training unit is also used to: input the training sample data into the lane-traffic light correspondence generation model; based on the preset output structure of the lane-traffic light correspondence generation model, determine whether the number of traffic lights and the number of lane center lines corresponding to the training sample data are less than the corresponding preset number; when the number of traffic lights and / or the number of lane center lines is less than the corresponding preset number, use a padding vector to supplement and output a road correspondence matrix formed by multiplying the preset number of lane center lines by the preset number of traffic lights.

[0101] It can be understood that the above-mentioned traffic light determination device for autonomous driving vehicles can implement the various steps of the traffic light determination method for autonomous driving vehicles provided in the aforementioned embodiments, and the relevant explanations about the traffic light determination method for autonomous driving vehicles are applicable to the traffic light determination device for autonomous driving vehicles, which will not be repeated here.

[0102] Figure 3This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 3 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0103] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0104] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0105] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a traffic light determination device for an autonomous driving vehicle at the logical level. The processor executes the program stored in the memory and is specifically used to execute:

[0106] Acquire a current road image collected by the autonomous driving vehicle, and detect traffic lights in the current road image to obtain a traffic light detection result;

[0107] Obtain lane centerline data of the road the autonomous vehicle is currently on;

[0108] According to the traffic light detection result and the lane centerline data of the road where the autonomous driving vehicle is currently located, a model for generating a correspondence between lanes and traffic lights is used to determine a correspondence between the lanes and traffic lights of the road where the autonomous driving vehicle is currently located;

[0109] According to the lane in which the autonomous driving vehicle is currently located, navigation route information, and the correspondence between the lane and the traffic light, the target traffic light currently corresponding to the autonomous driving vehicle and the light state information of the target traffic light are determined, so as to control the driving of the autonomous driving vehicle at the intersection according to the light state information of the target traffic light.

[0110] The above application Figure 1 The method performed by the traffic light determination device for an autonomous driving vehicle disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0111] The present application also provides a computer program product, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method performed by the traffic light determination device for an autonomous driving vehicle in the illustrated embodiment is specifically used to perform:

[0112] Acquire a current road image collected by the autonomous driving vehicle, and detect traffic lights in the current road image to obtain a traffic light detection result;

[0113] Obtain lane centerline data of the road the autonomous vehicle is currently on;

[0114] According to the traffic light detection result and the lane centerline data of the road where the autonomous driving vehicle is currently located, a model for generating a correspondence between lanes and traffic lights is used to determine a correspondence between the lanes and traffic lights of the road where the autonomous driving vehicle is currently located;

[0115] According to the lane in which the autonomous driving vehicle is currently located, navigation route information, and the correspondence between the lane and the traffic light, the target traffic light currently corresponding to the autonomous driving vehicle and the light state information of the target traffic light are determined, so as to control the driving of the autonomous driving vehicle at the intersection according to the light state information of the target traffic light.

[0116] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0120] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0121] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0122] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0123] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0124] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0125] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A traffic light determination method for an autonomous vehicle, wherein: The signal light determination method for an autonomous driving vehicle comprises: Acquire a current road image collected by the autonomous driving vehicle, and detect traffic lights in the current road image to obtain a traffic light detection result; Obtain lane centerline data of the road the autonomous vehicle is currently on; According to the traffic light detection result and the lane centerline data of the road where the autonomous driving vehicle is currently located, a model for generating a correspondence between lanes and traffic lights is used to determine a correspondence between the lanes and traffic lights of the road where the autonomous driving vehicle is currently located; According to the lane in which the autonomous driving vehicle is currently located, navigation route information, and the correspondence between the lane and the traffic light, the target traffic light currently corresponding to the autonomous driving vehicle and the light state information of the target traffic light are determined, so as to control the driving of the autonomous driving vehicle at the intersection according to the light state information of the target traffic light.

2. The traffic light determination method for an autonomous driving vehicle according to claim 1, wherein: The lane-to-traffic light correspondence generation model is trained in the following way: Acquire a current road image collected by the autonomous driving vehicle, and detect traffic lights in the current road image to obtain a traffic light detection result; Obtain lane centerline data of the road the autonomous vehicle is currently on; Constructing training sample data of a model for generating a correspondence between lanes and traffic lights according to the traffic light detection result and the lane centerline data; The lane-traffic-light correspondence generation model is trained using the training sample data of the lane-traffic-light correspondence generation model to obtain a trained lane-traffic-light correspondence generation model.

3. The traffic light determination method for an autonomous driving vehicle according to claim 2, wherein: The acquisition of lane centerline data of the road where the autonomous driving vehicle is currently located includes: Get the current position of the autonomous vehicle; Acquire corresponding high-precision map data according to the current position of the autonomous driving vehicle; The lane centerline data of the road where the autonomous driving vehicle is currently located is obtained from the high-precision map data corresponding to the current position of the autonomous driving vehicle.

4. The traffic light determination method for an autonomous driving vehicle according to claim 2, wherein: The training sample data for constructing a lane-traffic light correspondence generation model based on the traffic light detection result and the lane centerline data includes: Encoding the traffic light detection result to obtain a traffic light encoding vector; Encoding the lane centerline data to obtain a lane centerline encoding vector; The traffic light coding vector and the lane centerline coding vector are fused to obtain training sample data for generating a model for the correspondence relationship between the lane and the traffic light.

5. The traffic light determination method for an autonomous driving vehicle according to claim 4, wherein: The encoding process of the lane centerline data to obtain the lane centerline encoding vector comprises: Preprocessing the lane centerline data using a preset lane line preprocessing strategy to obtain preprocessed lane centerline data; The preprocessed lane centerline data is encoded to obtain a lane centerline encoding vector.

6. The traffic light determination method for an autonomous driving vehicle according to claim 2, wherein: The method of training the lane-traffic light correspondence generation model using the training sample data of the lane-traffic light correspondence generation model comprises: Inputting the training sample data of the lane-traffic light correspondence generation model into a multi-layer perceptron to obtain the correspondence between the lane and the traffic light output by the multi-layer perceptron; Calculating a loss value based on the correspondence between lanes and traffic lights output by the multi-layer perceptron and the correspondence between pre-labeled lanes and traffic lights; The loss value is used to update the parameters of the multilayer perceptron to obtain a trained lane-traffic light correspondence generation model.

7. The traffic light determination method for an autonomous driving vehicle according to claim 6, wherein: The method of training the lane-traffic light correspondence generation model using the training sample data of the lane-traffic light correspondence generation model comprises: Inputting the training sample data into the corresponding relationship generation model between the lane and the traffic light; Based on the preset output structure of the correspondence relationship between the lanes and the traffic lights, determining whether the number of traffic lights and the number of lane center lines corresponding to the training sample data are less than the corresponding preset numbers; When the number of traffic lights and / or lane center lines is less than the corresponding preset number, a fill-in vector is used to supplement and a road correspondence matrix consisting of the preset number of lane center lines multiplied by the preset number of traffic lights is output.

8. A traffic light determination device for an autonomous driving vehicle, wherein: The traffic light determination device for an autonomous driving vehicle comprises: A traffic light detection unit, used to obtain a current road image collected by the autonomous driving vehicle, and detect traffic lights in the current road image to obtain a traffic light detection result; An acquisition unit, used to acquire lane centerline data of a road where the autonomous driving vehicle is currently located; a first determination unit, configured to determine a correspondence between a lane and a traffic light of a road where the autonomous driving vehicle is currently located, based on the traffic light detection result and lane centerline data of the road where the autonomous driving vehicle is currently located, by using a lane-traffic light correspondence generation model; The second determination unit is used to determine the target traffic light currently corresponding to the autonomous driving vehicle and the light state information of the target traffic light based on the lane in which the autonomous driving vehicle is currently located, navigation route information, and the correspondence between the lane and the traffic light, so as to control the driving of the autonomous driving vehicle at the intersection according to the light state information of the target traffic light.

9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the traffic light determination method for an autonomous driving vehicle as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, which, when executed by a processor, implements the traffic light determination method for an autonomous driving vehicle as described in any one of claims 1 to 7.

Images