Association method, electronic device, storage medium and program product

Abstract

The invention relates to the technical field of computers, and discloses an association method, electronic equipment, a storage medium and a program product. In the method, an electronic device determines a target vehicle whose driving state matches the indication information of a target traffic light from a plurality of vehicles based on the driving track information of each vehicle in the plurality of vehicles and the indication information of the target traffic light at different moments when the target vehicle is located at a target position (such as a stop line corresponding to the target traffic light and other road marking lines). And taking a lane corresponding to the driving track information of the target vehicle as an associated lane of the target traffic light. According to the method, the associated lane of the target traffic light is determined based on the driving state of the vehicle at the target position and the indication information of the target traffic light, and the accuracy of the determined associated lane is high.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an association method, electronic equipment, storage medium and program product. Background Art

[0002] In the field of autonomous driving, when an autonomous driving vehicle is about to pass through a road intersection with multiple traffic lights, in order to ensure driving safety, it is necessary to determine the traffic light corresponding to the current lane. Then, the autonomous driving vehicle determines whether it can pass through the current road intersection based on the indication information of the corresponding traffic light (such as red, yellow, and green). For example, if the indication information of the traffic light corresponding to the current lane is green, the autonomous driving vehicle determines that it can pass through the current road intersection; if the indication information of the traffic light corresponding to the current lane is red, the autonomous driving vehicle determines that it is prohibited to pass through the current road intersection.

[0003] Therefore, in order to enable the autonomous driving vehicle to determine the traffic light corresponding to the current lane, multiple data pairs consisting of lanes and traffic lights can be used to train the autonomous driving vehicle. The lanes and traffic lights in each data pair are interrelated, that is, the indication information of the traffic lights in each data pair is used to indicate whether the vehicle traveling in the lane in the data pair can pass.

[0004] In some solutions, the association between lanes and traffic lights in each data pair is manually annotated in the 3D point cloud data. However, manual annotation may lead to misdetermination of the association between lanes and traffic lights, reducing the accuracy of the obtained association. In addition, manual annotation may also lead to low efficiency in determining the association. Summary of the invention

[0005] The embodiment of the present application provides an association method, electronic device, storage medium and program product. The method can automatically determine the associated lane of the target traffic light, thereby improving the determination efficiency and accuracy of the associated lane.

[0006] In a first aspect, the present application provides an association method, which is applied to an electronic device, and the method includes: obtaining indication information of a target traffic light at different times; obtaining driving trajectory information of each vehicle among multiple vehicles, wherein the driving trajectory information is used to indicate the position and driving status of each vehicle at different times, and one driving trajectory information corresponds to one lane; based on the driving trajectory information of multiple vehicles and the indication information of the target traffic lights at different times, determining a target vehicle from multiple vehicles, wherein when the target vehicle is at a target position, the driving status of the target vehicle matches the indication information of the target traffic light; determining the lane corresponding to the driving trajectory information of the target vehicle as an associated lane of the target traffic light, wherein the target position includes the position of the road marking corresponding to the target traffic light.

[0007] The association method provided by the present application can automatically determine the associated lane of the target traffic light by an electronic device. Compared with the manual determination method, the method can improve the determination efficiency of the associated lane of the target traffic light, that is, improve the association efficiency of the target traffic light and the lane. In addition, the method determines the associated lane of the target traffic light based on the matching of the driving state of the vehicle and the indication information of the target traffic light when the vehicle is at the target position. The accuracy of the associated lane determined in this way is higher.

[0008] In a possible implementation of the first aspect, a target vehicle is determined from multiple vehicles based on driving trajectory information of multiple vehicles and indication information of target traffic lights at different times, including: determining a driving state to be matched when the vehicle to be matched is at a target position based on driving trajectory information of the vehicle to be matched among the multiple vehicles; determining indication information to be matched of the target traffic light when the vehicle to be matched is at the target position based on indication information of the target traffic lights at different times; and determining the vehicle to be matched as the target vehicle by matching the driving speed indicated by the driving state to be matched with the indication information to be matched.

[0009] In a possible implementation of the first aspect, the driving speed corresponding to the driving state indication to be matched matches the indication information to be matched, and the vehicle to be matched is determined as the target vehicle, including: the driving speed corresponding to the driving state indication to be matched is greater than a speed threshold, and the indication information to be matched is green, and the vehicle to be matched is determined as the target vehicle.

[0010] It can be understood that if the to-be-matched vehicle is at the target position, the driving speed of the to-be-matched vehicle is greater than the speed threshold, and the indication information of the target traffic light is green, it means that when the target traffic light shows green, the to-be-matched vehicle has a driving trend, so the to-be-matched vehicle can be used as the target vehicle to further determine the associated lane of the target traffic light based on the target vehicle. This method can ensure the accuracy of the obtained target vehicle, and then ensure the accuracy of the associated lane of the obtained target traffic light.

[0011] In a possible implementation of the first aspect, the moment when the vehicle to be matched is at the target position includes a first moment and a second moment, the first moment and the second moment are different, the driving state to be matched includes a first driving state corresponding to the first moment and a second driving state corresponding to the second moment, the indication information to be matched includes first indication information corresponding to the first moment and second indication information corresponding to the second moment, the driving speed corresponding to the driving state indication to be matched matches the indication information to be matched, and the vehicle to be matched is determined as the target vehicle, including: a first driving speed corresponding to the first driving state indication is greater than a speed threshold, the first indication information is green, the second driving speed indicated by the second driving state is less than or equal to the speed threshold, and the second indication information is red, and the vehicle to be matched is determined as the target vehicle.

[0012] It can be understood that if a lane does not have a corresponding associated traffic light, that is, no matter whether the indication information of the target traffic light is red or green, the driving speed of the vehicle traveling in the lane may be greater than the speed threshold. If the target vehicle is determined only based on the driving speed of the vehicle being greater than the threshold when it is at the target position and the indication information of the target traffic light being green, the vehicle traveling in the lane without a corresponding associated traffic light will be mistakenly determined as the target vehicle. This method will result in a lower accuracy in determining the target vehicle, and further a lower accuracy in determining the associated lane of the target traffic light.

[0013] Therefore, the electronic device can determine that the vehicle is a target vehicle when it is determined that when the vehicle is at the target position at a first moment, the driving speed is greater than the speed threshold and the indication information of the target traffic light is green, and when the vehicle is at the target position at a second moment, the driving speed is less than or equal to the speed threshold and the indication information of the target traffic light is red.

[0014] This approach can avoid treating vehicles traveling on lanes without corresponding associated traffic lights as target vehicles, thereby improving the accuracy of determining target vehicles and further improving the accuracy of determining the associated lanes of target traffic lights.

[0015] In a possible implementation of the first aspect, the driving speed indicated by the driving state to be matched includes a driving rate and a driving direction of the vehicle to be matched, and the driving direction includes a direction in which the vehicle to be matched is traveling toward a target traffic light.

[0016] In a possible implementation of the first aspect, the target traffic light is located at a first position, and obtaining indication information of the target traffic light at different times includes: obtaining a captured image, the captured image is obtained based on a camera in each vehicle at different times, the captured image includes image information of the first traffic light, the image information includes position information of the first traffic light in the captured image and indication information of the first traffic light; based on the position of each vehicle at the first moment and the first position of the target traffic light, determining the predicted position information of the target traffic light in the captured image at the first moment; determining that the first position information and the predicted position information of the first traffic light in the first image information match, wherein the first image information is the image information of the captured image at the first moment; based on the indication information of the first traffic light in the captured image, determining the indication information of the target traffic light at different times.

[0017] It is understandable that the three-dimensional point cloud data acquired by the electronic device includes the first position of the target traffic light, but the electronic device cannot determine the indication information of the target traffic light at different times based on the three-dimensional point cloud data. Therefore, the electronic device needs to associate the target traffic light at the first position with the captured image including the first (or target) traffic light to determine the indication information of the target traffic light at different times.

[0018] In a possible implementation of the first aspect, the lane corresponding to the driving trajectory information of the first vehicle in each vehicle is determined based on the following method: determining the lane centerline information of each lane in a plurality of lanes; and taking the lane corresponding to the lane centerline information with the smallest distance difference between the lane centerline information and the driving trajectory information of the first vehicle among the plurality of lane centerline information as the lane corresponding to the driving trajectory information of the first vehicle.

[0019] It can be understood that the driving trajectory information of the first vehicle is used to indicate the position of the first vehicle at different times, so the electronic device can determine the sum of the shortest distances from the position of the first vehicle to the center line of each lane at each time. The sum of the shortest distances from the position of the first vehicle to the center line of each lane at each time can be the distance difference between the lane center line information and the driving trajectory information of the first vehicle.

[0020] Then, the lane corresponding to the lane centerline information corresponding to the smallest sum of the shortest distances among the multiple lane centerline information is used as the lane corresponding to the driving trajectory information of the first vehicle. The lane corresponding to the driving trajectory information of the first vehicle obtained in this way has a higher accuracy.

[0021] In a possible implementation of the first aspect, the method further includes: when a distance between a vehicle center point corresponding to the target vehicle and the target position is less than or equal to a distance threshold, determining that the target vehicle is at the target position.

[0022] Wherein, the road marking includes any one of a stop line and a zebra crossing. Corresponding to the case where the target position is a stop line, a zebra crossing, etc., the distance between the vehicle center point of the target vehicle and the target position may be the distance between the vehicle center point and a target point in the target position. Wherein, the target point in the target position may be, for example, the midpoint of the stop line, or the center point of the area corresponding to the zebra crossing.

[0023] In a second aspect, the present application provides an electronic device, comprising: one or more processors; one or more memories; one or more memories storing one or more programs, which, when the one or more programs are executed by one or more processors, enables the electronic device to execute the first aspect and any possible associated method of the first aspect.

[0024] In a third aspect, the present application provides a computer-readable storage medium having instructions stored thereon, which, when executed on an electronic device, causes the electronic device to execute the first aspect and any possible associated method of the first aspect.

[0025] In a fourth aspect, the present application provides a computer program product, which includes: computer instructions, when the computer instructions are executed on an electronic device, the electronic device executes the first aspect and any possible implementation of the associated method of the first aspect.

[0026] Among them, the beneficial effects of the second to fourth aspects can refer to the beneficial effects of the first aspect and any possible implementation of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 According to some embodiments of the present application, a flow chart of a first association method is shown; Figure 2 According to some embodiments of the present application, a flow chart of a method for determining indication information of a target traffic light at different times is shown; Figure 3 According to some embodiments of the present application, a schematic diagram of a scene including a plurality of vehicles and a first traffic light is shown; Figure 4 According to some embodiments of the present application, a schematic diagram of a scene including multiple vehicles and multiple target traffic lights is shown; Figure 5 According to some embodiments of the present application, a schematic diagram of a scene including multiple vehicles and a target traffic light is shown; Figure 6 According to some embodiments of the present application, a flow chart of a second association method is shown; Figure 7According to some embodiments of the present application, a schematic structural diagram of an electronic device is shown. DETAILED DESCRIPTION

[0028] Illustrative embodiments of the present application include, but are not limited to, associated methods, electronic devices, storage media, and program products.

[0029] It can be understood that the electronic devices to which the association method provided in the embodiments of the present application is applicable include, but are not limited to, electronic devices with the function of marking association relationships, such as tablet computers (Pad), computers with wireless transceiver functions, virtual reality (VR) devices, augmented reality (AR) devices, wireless devices in industrial control (industrial control), wireless devices in self-driving, wireless devices in remote medical surgery, wireless devices in smart grids, wireless devices in transportation safety, wireless devices in smart cities, wireless devices in smart homes, and the like.

[0030] Based on the above, the manual labeling method may lead to the misdetermination of the association between lanes and traffic lights, reducing the accuracy of the obtained association. In addition, manual labeling also leads to low efficiency in determining the association.

[0031] Therefore, in order to solve the above technical problems, the present application provides an association method. The method automatically determines the associated lane of the target traffic light based on the driving state of the vehicle at the target position and the indication information of the target traffic light when the vehicle is at the target position. Specifically, the electronic device can determine the target vehicle from multiple vehicles based on the driving trajectory information of multiple vehicles and the indication information of the target traffic lights at different times, and then determine the lane corresponding to the driving trajectory information of the target vehicle as the associated lane of the target traffic light. Among them, when the target vehicle is at the target position, the driving state indicated by the driving trajectory information of the target vehicle matches the indication information of the target traffic light.

[0032] In some embodiments, the target position includes the position of a road marking corresponding to the target traffic light, and the road marking includes any one of a stop line and a zebra crossing.

[0033] The association method provided by the present application can automatically determine the associated lane of the target traffic light by an electronic device. Compared with the manual determination method, the method can improve the determination efficiency of the associated lane of the target traffic light, that is, improve the association efficiency of the target traffic light and the lane. In addition, the method determines the associated lane of the target traffic light based on the matching of the driving state of the vehicle and the indication information of the target traffic light when the vehicle is at the target position. The accuracy of the associated lane determined in this way is higher.

[0034] The association method provided by the present application is described in detail below with reference to the accompanying drawings. Figure 1 A schematic flow chart of the first association method provided by the present application is shown. Figure 1 The execution subjects of each step of the process shown are all electronic devices. For the convenience of description, Figure 1 The execution entities of each step will not be described repeatedly in the steps of the process shown. Figure 1 As shown, the method includes but is not limited to the following schemes: S101: Obtain indication information of a target traffic light at different times.

[0035] It can be understood that the application scenario of the association method provided in the present application is to determine the associated lane of the target traffic light in the three-dimensional point cloud data. The three-dimensional point cloud data includes the target traffic light at the first position, that is, the electronic device can obtain the three-dimensional point cloud data, and then determine the target traffic light in the three-dimensional point cloud data and the first position of the target traffic light.

[0036] It is understandable that the electronic device can determine the target traffic light and the first position of the target traffic light based on the three-dimensional point cloud data, but cannot obtain the indication information of the target traffic light. Among them, the first position of the target traffic light is the three-dimensional position of the target traffic light, and the first position can be obtained based on the three-dimensional point cloud data. It is understandable that the three-dimensional point cloud data records objects in a specific area (such as the target traffic light in the present application) in the form of points, and each point corresponds to three-dimensional coordinate information. Therefore, the electronic device can obtain the three-dimensional coordinate information corresponding to the target traffic light in the three-dimensional point cloud data based on the three-dimensional point cloud data, that is, obtain the first position.

[0037] In some embodiments, the target traffic light and the first position in the three-dimensional point cloud data may be pre-marked. This application does not limit the marking method of the target traffic light and the first position, and the target traffic light and the first position may be manually marked or automatically marked.

[0038] Next, in the present application, the electronic device can determine the indication information of the target traffic light at different times based on the first position of the target traffic light and the captured images captured by the vehicle at different times. The indication information of the target traffic light may include red, green and yellow.

[0039] In some embodiments, Figure 2 As shown, the method for determining the indication information of the target traffic light at different times includes: S201: Acquire captured images, where the captured images are captured by a camera in each vehicle at different times, and the captured images include image information of a first traffic light, and the image information includes position information of the first traffic light in the captured images and indication information of the first traffic light.

[0040] Figure 3 A schematic diagram of a scene including multiple vehicles and a first traffic light is shown. It can be understood that the vehicles and the first traffic light in this application are obtained by annotating the three-dimensional point cloud data, but for the convenience of illustration and understanding, this application does not represent the vehicles and the first traffic light in the form of three-dimensional point cloud data.

[0041] like Figure 3 As shown, Figure 3 The first traffic light 301, vehicle A1, vehicle B1, vehicle C1 and vehicle D1 are included. Vehicle A1 and vehicle B1 are located on lane M1, and vehicle C1 and vehicle D1 are located on lane N1.

[0042] It can be understood that since vehicles A1, B1, C1 and D1 are traveling toward the first traffic light 301, the captured images acquired by the electronic device may be, for example, images of vehicles A1, B1, C1 and D1 taken at different times based on cameras in front of the vehicles.

[0043] The captured image includes image information of the first traffic light, for example, two-dimensional position information of the first traffic light in the captured image and indication information of the first traffic light (for example, red, green, yellow).

[0044] It can be understood that after the electronic device obtains the captured images captured by the camera of each vehicle at different times, it is necessary to further execute subsequent S202 and S203 to determine whether the first traffic light in the captured image is the target traffic light at the first position.

[0045] It can be understood that since the electronic device needs to determine whether the first traffic light is the target traffic light based on the captured image obtained by the vehicle, the vehicle in the present application can be a vehicle traveling towards the target traffic light and the distance between the vehicle and the target traffic light meets the distance condition, so as to avoid the situation where the vehicle is too far away from the target traffic light and the first traffic light different from the target traffic light is mistakenly regarded as the target traffic light.

[0046] S202: Based on the position of each vehicle at the first moment and the first position of the target traffic light, determine the predicted position information of the target traffic light in the captured image at the first moment.

[0047] For the first moment among the different moments, the electronic device can determine the predicted position information of the target traffic light in the captured image at the first moment based on the position of each vehicle at the first moment, the first position of the target traffic light, and the internal and external parameters of the camera on each vehicle.

[0048] Among them, the camera's internal parameters describe the geometric characteristics inside the camera, and the internal parameters usually include focal length, principal point, distortion parameters, etc. The camera's external parameters describe the camera's position and orientation in the world coordinate system. In this application, the camera's external parameters can be used to describe the camera's position and orientation relative to the corresponding vehicle. It can be understood that the camera's internal and external parameters determine how the camera maps the target traffic light into a two-dimensional image (such as a captured image).

[0049] Therefore, in S202, the electronic device can determine the predicted position information of the target traffic light in the captured image at the first moment based on the position of each vehicle at the first moment, the first position of the target traffic light, and the internal and external parameters of the camera on each vehicle.

[0050] The embodiment of the present application does not limit the method for determining the predicted position information of the target traffic light in the captured image at the first moment. For example, the electronic device can obtain the predicted position information of the target traffic light in the captured image at the first moment based on a pre-trained mapping model.

[0051] S203: Determine whether first position information of the first traffic light in the first image information matches the predicted position information, wherein the first image information is image information of an image captured at a first moment.

[0052] It can be understood that after the electronic device determines the predicted position information of the target traffic light in the captured image at the first moment, it matches the predicted position information with the first position information in the first image information of the captured image at the first moment. In some embodiments, if the first position information and the predicted position information are the same, or the difference between the first position information and the predicted position information is less than the difference threshold, it can be determined that the first position information and the predicted position information match.

[0053] The first image information is the image information of the captured image obtained by the vehicle capturing the first traffic light at the first moment, including the first position information of the first traffic light in the captured image. The predicted position information is the predicted position information of the target traffic light in the captured image at the first moment, which is predicted based on the position information of the vehicle at the first moment, the first position of the target traffic light, and the internal and external parameters of the camera. Specifically, the predicted position information can be understood as the position information of the target traffic light in the image obtained if the vehicle actually captures the target traffic light at the first position based on the camera corresponding to the internal and external parameters at the first moment.

[0054] Therefore, when the electronic device has the first image information, if it wants to determine whether the target traffic light at a known position (i.e., the first position) is the first traffic light, it can determine whether the first position information in the first image information matches the predicted position information corresponding to the target traffic light at the first moment.

[0055] If the electronic device determines that the predicted location information matches the first location information, it can be determined that the first traffic light corresponding to the first location information and the target traffic light corresponding to the predicted location information are the same traffic light, that is, the first traffic light is the target traffic light.

[0056] It is understandable that Figure 3 As shown, at the first moment, there may be multiple vehicles (such as vehicle A1, vehicle B1, vehicle C1, vehicle D1) taking pictures based on the camera, and each vehicle may be provided with multiple cameras, so the electronic device can obtain multiple captured images corresponding to the first moment, that is, obtain multiple first position information.

[0057] In some embodiments, the electronic device may determine that the first traffic light is the target traffic light when the number of first location information matching the predicted location information among multiple first location information is greater than a quantity threshold.

[0058] Taking the number of vehicles photographed at the first moment as a, and each vehicle having b cameras as an example, the electronic device can obtain a×b photographed images corresponding to the first moment, that is, obtain a×b first position information. Then, the number S of first position information matching the predicted position information among the a×b first position information is determined. If the number S is greater than the number threshold S', it can be determined that the first traffic light is the target traffic light.

[0059] The embodiment of the present application does not limit the method for determining the quantity threshold, and it can be set based on experience or flexibly adjusted according to actual application scenarios.

[0060] S204: Determine the indication information of the target traffic light at different times based on the indication information of the first traffic light in the captured image.

[0061] After the electronic device determines that the first position information and the predicted position information match, it can determine that the first traffic light in the captured image is the target traffic light at the first position, and then can use the indication information of the first traffic light in the first image information as the indication information of the target traffic light at the first moment.

[0062] It can be understood that the method for determining the indication information of the target traffic light at each first moment in different moments is similar, and will not be described in detail. Therefore, based on the above method, the indication information of the target traffic light at different moments can be determined.

[0063] In some embodiments, the above-mentioned method of determining the indication information of the target traffic light at different times may also be referred to as a three-dimensional (information)-two-dimensional (information) association of the target traffic light.

[0064] S102: Acquire driving trajectory information of each vehicle among a plurality of vehicles, wherein the driving trajectory information is used to indicate the position and driving status of each vehicle at different times, and one driving trajectory information corresponds to one lane.

[0065] In an embodiment of the present application, the electronic device may determine the driving trajectory information of each vehicle based on the three-dimensional point cloud data. The driving state indicated by the driving trajectory information may be used to indicate the driving speed of the vehicle. In some embodiments, the driving speed of the vehicle includes the driving rate and driving direction of the vehicle, and the driving direction includes the direction in which the vehicle is traveling toward the target traffic light.

[0066] The driving track information of each vehicle in the three-dimensional point cloud data may be pre-marked. The present application does not limit the marking method of the driving track information of each vehicle, which may be manually marked or automatically marked.

[0067] In the embodiment of the present application, the driving trajectory information of each vehicle corresponds to a lane. It can be understood that the three-dimensional point cloud data acquired by the electronic device includes the driving trajectory information of each vehicle and the pre-marked lanes. However, the driving trajectory information and lanes of each vehicle are not related to each other. Therefore, the electronic device can associate the driving trajectory information and lanes of each vehicle, that is, determine the lane corresponding to the driving trajectory information of each vehicle.

[0068] Among them, the method for determining the lane corresponding to the driving trajectory information of the first vehicle in each vehicle includes: determining the lane centerline information of each lane among multiple lanes; taking the lane corresponding to the lane centerline information with the smallest distance difference between the lane centerline information and the driving trajectory information of the first vehicle among the multiple lane centerline information as the lane corresponding to the driving trajectory information of the first vehicle.

[0069] Figure 4FIG. 1 shows a schematic diagram of a scene including multiple vehicles and multiple target traffic lights. Figure 4 As shown, Figure 4 The first target traffic light 401, the second target traffic light 402, the vehicle A2, the vehicle B2, the vehicle C2 and the vehicle D2 are included. In addition, Figure 4 The lane M2, lane N2 and lane P2 are included. Among them, lane M2 and lane N2 are through lanes, and lane P2 is a right turn lane.

[0070] Below Figure 4 Taking the vehicle C2 in the figure as the first vehicle as an example, the method for determining the lane corresponding to the driving trajectory information of the vehicle C2 is described.

[0071] It can be understood that lanes M2, lanes N2, and lanes P2 in the three-dimensional point cloud data are pre-marked, so the electronic device can determine the three-dimensional position information of lanes M2, lanes N2, and lanes P2 based on the three-dimensional point cloud data, and can further determine the lane centerline information of each lane. For example, the lane centerline indicated by the lane centerline information of lane M2 can be expressed as centerline m2; the lane centerline indicated by the lane centerline information of lane N2 can be expressed as centerline n2; and the lane centerline indicated by the lane centerline information of lane P2 can be expressed as centerline p2.

[0072] It can be understood that the driving trajectory information of the vehicle C2 is used to indicate the position of the vehicle C2 at different times. Therefore, the electronic device can determine the sum of the shortest distances from the position of the vehicle C2 to each center line (such as the center line m2, the center line n2, and the center line p2) at each time. The sum of the shortest distances from the position of the vehicle C2 to each center line at each time can be the distance difference between the lane center line information and the driving trajectory information of the vehicle C2.

[0073] For example, if at each moment, the sum of the shortest distances from the position of vehicle C2 to the center line n2 is the smallest, then it can be determined that the lane N2 corresponding to the center line n2 is the lane corresponding to the driving trajectory of vehicle C2.

[0074] Figure 4 The method for determining the lane corresponding to the driving trajectory information of each vehicle in is similar, and will not be described one by one. Specifically, Figure 4 , the lane corresponding to the driving trajectory information of vehicle A2 is lane M2; the lane corresponding to the driving trajectory information of vehicle B2 is lane M2; the lane corresponding to the driving trajectory information of vehicle C2 is lane N2; and the lane corresponding to the driving trajectory information of vehicle D2 is lane P2.

[0075] In some embodiments, the process of determining the lane corresponding to the driving trajectory information of each vehicle may also be referred to as trajectory information-lane association.

[0076] S103: Based on the driving trajectory information of multiple vehicles and the indication information of the target traffic lights at different times, a target vehicle is determined from the multiple vehicles, wherein when the target vehicle is at the target position, the driving state of the target vehicle matches the indication information of the target traffic light.

[0077] In some embodiments, S103 may further include: determining the driving state to be matched when the vehicle to be matched is at the target position based on the driving trajectory information of the vehicle to be matched among multiple vehicles; determining the indication information to be matched of the target traffic light when the vehicle to be matched is at the target position based on the indication information of the target traffic light at different times; matching the driving speed indicated by the driving state to be matched with the indication information to be matched, and determining the vehicle to be matched as the target vehicle.

[0078] In some embodiments, the target position includes the position of the road marking corresponding to the target traffic light, wherein the road marking may include any one of a stop line and a zebra crossing. The position of the road marking included in the target position may be a pre-set position. Taking the road marking as a stop line as an example, the stop line in the three-dimensional point cloud data is pre-marked. In this way, the electronic device determines information such as the position of the stop line based on the three-dimensional point cloud data.

[0079] The embodiments of the present application do not limit the method for determining that the target vehicle is at the target position. In some embodiments, if the distance between the center point of the target vehicle and the target position is less than or equal to the distance threshold, the target vehicle is determined to be at the target position. The present application also does not limit the size of the distance threshold, which can be set based on experience or flexibly adjusted according to actual application scenarios.

[0080] Wherein, corresponding to the case where the target position is a stop line, a zebra crossing, etc., the distance between the center point of the target vehicle and the target position may be the distance between the center point of the vehicle and a target point in the target position. Wherein, the target point in the target position may be, for example, the midpoint of the stop line, or the center point of the area corresponding to the zebra crossing.

[0081] It is understandable that the number of target vehicles in the embodiment of the present application can be one or more, and the embodiment of the present application does not limit this. The following takes the case where the number of target vehicles is more than one as an example to describe the method for determining the target vehicles.

[0082] In some examples, the method of determining the target vehicle based on matching the driving speed of the driving state indication to be matched with the indication information to be matched in S103 includes the following two methods.

[0083] Method 1: The driving speed corresponding to the driving state indication to be matched is greater than the speed threshold, and the indication information to be matched is green, and the vehicle to be matched is determined as the target vehicle.

[0084] Road markings Figure 4 For example, for the first target traffic light 401, if the driving speed of vehicle A2 at the stop line S1 is greater than the speed threshold, and the indication information of the first target traffic light 401 is green at this time, the electronic device can use vehicle A2 as the target vehicle corresponding to the first target traffic light 401. If the driving speed of vehicle C2 at the stop line S1 is greater than the speed threshold, and the indication information of the first target traffic light 401 is green at this time, the electronic device can use vehicle C2 as the target vehicle corresponding to the first target traffic light 401.

[0085] If the driving speed of vehicle D2 at stop line S1 is less than or equal to the speed threshold, or the indication information of the first target traffic light 401 is not green at this time, the electronic device can determine that vehicle D2 is not the target vehicle corresponding to the first target traffic light 401. If the driving speed of vehicle D2 at stop line S1 is greater than the speed threshold, and the indication information of the second target traffic light 402 is green at this time, the electronic device can regard vehicle D2 as the target vehicle corresponding to the second target traffic light 402.

[0086] In the above example, both vehicle A2 and vehicle C2 can be used as the target vehicle corresponding to the first target traffic light 401 ; vehicle D2 can be used as the target vehicle corresponding to the second target traffic light 402 .

[0087] However, in some scenarios, some lanes may not have corresponding associated traffic lights. Figure 5 FIG. 1 shows a schematic diagram of a scene including multiple vehicles and a target traffic light. Figure 5 As shown, Figure 5 The third target traffic light 501, vehicle A3, vehicle B3, vehicle C3 and vehicle D3 are included. In addition, Figure 5 The lane M3, lane N3 and lane P3 are included. Among them, lane M3 and lane N3 are through lanes, and lane P3 is a right turn lane.

[0088] Figure 5In the example, the right-turn lane P3 has no corresponding associated traffic light, that is, no matter whether the indication information of the third target traffic light 501 is red or green, the driving speed of the vehicle D3 traveling in the lane may be greater than the speed threshold. Therefore, if the target vehicle is still determined in this scenario using the above method 1, the driving speeds of vehicles A3, C3, and D3 at the target position (such as stop line S2) may all be greater than the speed threshold, and the indication information of the third target traffic light 501 at this moment is green, so the electronic device will treat vehicles A3, C3, and D3 as target vehicles corresponding to the third target traffic light 501. This will reduce the accuracy of the target vehicle, and thus reduce the accuracy of the associated lane of the third target traffic light 501.

[0089] Therefore, in order to ensure the accuracy of the associated lane of the target traffic light, in some other embodiments, the electronic device can determine the target vehicle based on the following method 2.

[0090] Method 2: The moment when the vehicle to be matched is at the target position includes a first moment and a second moment, the first moment and the second moment are different, the driving state to be matched includes a first driving state corresponding to the first moment and a second driving state corresponding to the second moment, the indication information to be matched includes first indication information corresponding to the first moment and second indication information corresponding to the second moment, the driving speed corresponding to the driving state indication to be matched matches the indication information to be matched, and the vehicle to be matched is determined as the target vehicle, including: the first driving speed corresponding to the first driving state indication is greater than the speed threshold, the first indication information is green, the second driving speed indicated by the second driving state is less than or equal to the speed threshold, and the second indication information is red, and the vehicle to be matched is determined as the target vehicle.

[0091] Continue as Figure 5 As shown, the road markings are Figure 5 Taking the stop line S2 in the figure as an example, for the third target traffic light 501, if the driving speed of vehicle A3 when it is located at the stop line S2 at the first moment is greater than the speed threshold, and the indication information of the third target traffic light 501 at the first moment is green, and the driving speed of vehicle A3 when it is located at the stop line S2 at the second moment is less than or equal to the speed threshold, and the indication information of the third target traffic light 501 at the second moment is red, it means that vehicle A3 has a driving trend when the third target traffic light 501 displays green, and has a stopping trend when the third target traffic light 501 displays red. At this time, vehicle A3 can be determined as the target vehicle.

[0092] The embodiment of the present application does not limit the order of the first moment and the second moment. The first moment may be a moment before the second moment, or a moment after the second moment.

[0093] For another example, since the lane P3 corresponding to the driving trajectory of vehicle D3 has no corresponding associated traffic light, vehicle D3 has a driving tendency regardless of whether the third target traffic light 501 displays green or red. For example, when the indication information of the third target traffic light 501 is red, the driving speed of vehicle D3 may still be greater than the speed threshold. Therefore, vehicle D3 is not included in the target vehicles determined based on the second method.

[0094] This method determines the target vehicle based on the indication information of the target traffic light for the driving state of the vehicle in red and green conditions, so that the accuracy of the determined target vehicle is higher, and thus the accuracy of the associated lane of the determined target traffic light is higher.

[0095] S104: Determine the lane corresponding to the driving trajectory information of the target vehicle as the associated lane of the target traffic light.

[0096] It can be understood that after the electronic device determines the target vehicle among multiple vehicles, the lane corresponding to the driving trajectory information of the target vehicle can be used as the associated lane of the target traffic light.

[0097] For example, after determining the target vehicles (vehicle A2 and vehicle C2) corresponding to the first target traffic light 401 based on method one, the electronic device can use the lanes (such as lane M2 and lane N2) corresponding to the driving trajectory information of vehicles A2 and C2 as associated lanes of the first target traffic light 401.

[0098] For another example, after determining the target vehicles (vehicle A3 and vehicle C3) corresponding to the third target traffic light 501 based on method 2, the electronic device can use the lanes corresponding to the driving trajectory information of vehicles A3 and C3 (such as lane M3 and lane N3) as associated lanes of the third target traffic light 501.

[0099] In some embodiments, the process of determining the associated lane of a target traffic light may also be referred to as lane-target traffic light association.

[0100] Figure 6 FIG. 2 shows a flow chart of another association method provided by the present application. Figure 6 As shown, the method comprises the following steps: S601: Obtain dynamic feature annotation information.

[0101] In the embodiment of the present application, the dynamic element annotation information may include the driving track information of each vehicle mentioned above. The driving track information of each vehicle may be manually annotated or automatically annotated, and the present application does not limit this.

[0102] S602: Obtain static feature annotation information.

[0103] In the embodiment of the present application, the static element annotation information may include the annotation results of the lane and the target position mentioned above, wherein the lane and the target position may be manually or automatically annotated, and the present application does not limit this.

[0104] S603: Determine a lane centerline representing the lane based on the lane marking result in the static feature marking information.

[0105] It will be appreciated that lane centerlines may be used to represent lanes.

[0106] S604: Based on the driving track information and lane of each vehicle, obtain the association relationship between the driving track information and the lane.

[0107] It can be understood that the method for determining the association between the driving trajectory information and the lane can refer to the content shown in S102 above, and will not be repeated here.

[0108] S605: Acquire the corresponding photographed images of the target traffic light at different times.

[0109] S606: Based on the three-dimensional position of the target traffic light in the static element annotation information and the two-dimensional position of the target traffic light in the captured image, a three-dimensional-two-dimensional association relationship of the target traffic light is obtained.

[0110] It can be understood that the step S606 is to determine the indication information of the target traffic light at the first position at different times. The specific method can be found in the above-mentioned step S101, which will not be described in detail here.

[0111] S607: Based on the association relationship between the driving trajectory information and the lane, and the three-dimensional-two-dimensional association relationship of the target traffic light, the association relationship between the target traffic light and the lane is obtained.

[0112] It can be understood that the method for determining the association between the target traffic light and the lane in S607 is the same as the principle of the content shown in S103 and S104 above, and will not be repeated here.

[0113] In some embodiments, if the dynamic element annotation information and the static element annotation information in the above S601 and S602 are incomplete, the electronic device can perform multiple annotations, and execute the above S601-S607 once after each annotation to continuously update the association between the target traffic light and the lane, thereby improving the accuracy of the determination result of the association relationship.

[0114] The association method provided by the present application can automatically determine the associated lane of the target traffic light by an electronic device. Compared with the manual determination method, the method can improve the determination efficiency of the associated lane of the target traffic light, that is, improve the association efficiency of the target traffic light and the lane. In addition, the method determines the associated lane of the target traffic light based on the matching of the driving state of the vehicle and the indication information of the target traffic light when the vehicle is at the target position. The accuracy of the associated lane determined in this way is higher.

[0115] In some embodiments, the embodiments of the present application further provide a computer-readable storage medium, on which instructions are stored, and when the instructions are executed on an electronic device, the electronic device executes the association method described in the above embodiments.

[0116] In some embodiments, the embodiments of the present application also provide an electronic device, which includes: one or more processors; one or more memories; one or more memories storing one or more programs, and when the one or more programs are executed by one or more processors, the electronic device executes the association method described in the above embodiments.

[0117] In some embodiments, the embodiments of the present application further provide a computer program product, including: computer instructions, when the computer instructions are executed on an electronic device, the electronic device executes the association method described in the above embodiments.

[0118] Figure 7 The schematic diagram of the structure of the electronic device 1400 provided in the embodiment of the present application is shown. In some embodiments, the electronic device 1400 may be a vehicle computer of a vehicle, or a computer, etc. In one embodiment, the electronic device 1400 may include one or more processors 1404, a system control logic 1408 connected to at least one of the one or more processors 1404, a system memory 1412 connected to the system control logic 1408, a non-volatile memory (NVM) 1416 connected to the system control logic 1408, and a network interface 1420 connected to the system control logic 1408.

[0119] In some embodiments, the processor 1404 may include one or more single-core or multi-core processors. In some embodiments, the processor 1404 may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In embodiments where the electronic device 1400 employs an enhanced base station (evolved node B, ENB) or a radio access network (radio access network, RAN) controller, the processor 1404 may be configured to execute various embodiments, such as Figure 1and Figure 6 One or more of the multiple embodiments shown.

[0120] In some embodiments, system control logic 1408 may include any suitable interface controller to provide any suitable interface to at least one of processors 1404 and / or any suitable device or component in communication with system control logic 1408 .

[0121] In some embodiments, the system control logic 1408 may include one or more memory controllers to provide an interface to the system memory 1412. The system memory 1412 may be used to load and store data and / or instructions. In some embodiments, the system memory 1412 of the electronic device 1400 may include any suitable volatile memory, such as a suitable dynamic random access memory.

[0122] The non-volatile memory 1416 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 1416 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of a hard disk drive (HDD), a compact disc (CD) drive, and a digital versatile disc (DVD) drive.

[0123] The non-volatile memory 1416 may include a portion of storage resources on the device on which the electronic device 1400 is installed, or it may be accessible by the device but is not necessarily a portion of the device. For example, the non-volatile memory 1416 may be accessed over a network via the network interface 1420 .

[0124] In particular, the system memory 1412 and the non-volatile memory 1416 may include a temporary copy and a permanent copy of the instructions 1424, respectively. The instructions 1424 may include instructions that, when executed by at least one of the processors 1404, cause the electronic device 1400 to implement the following: Figure 1 and Figure 6 In some embodiments, instructions 1424, hardware, firmware, and / or software components thereof may additionally / alternatively be placed in system control logic 1408, network interface 1420, and / or processor 1404.

[0125] The network interface 1420 may include a transceiver for providing a radio interface for the electronic device 1400, thereby communicating with any other suitable device (such as a front-end module, an antenna, etc.) through one or more networks. In some embodiments, the network interface 1420 may be integrated with other components of the electronic device 1400. For example, the network interface 1420 may be integrated with at least one of the processor 1404, the system memory 1412, the non-volatile memory 1416, and a firmware device (not shown) having instructions. When at least one of the processors 1404 executes the instructions, the electronic device 1400 implements the following. Figure 1 and Figure 6 The method shown.

[0126] The network interface 1420 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 1420 may be a network adapter, a wireless network adapter, a telephone modem and / or a wireless modem.

[0127] In one embodiment, at least one of the processors 1404 may be packaged with logic for one or more controllers of the system control logic 1408 to form a system in package (SiP). In one embodiment, at least one of the processors 1404 may be integrated on the same die with logic for one or more controllers of the system control logic 1408 to form a system on chip (SoC).

[0128] The electronic device 1400 may further include an input / output (I / O) device 1432. The I / O device 1432 may include a user interface to enable a user to interact with the electronic device 1400; the design of the peripheral component interface enables the peripheral components to interact with the electronic device 1400. In some embodiments, the electronic device 1400 further includes a sensor for determining at least one of an environmental condition and location information related to the electronic device 1400.

[0129] In some embodiments, the user interface may include, but is not limited to, a display (e.g., an LCD display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., an LED flash), and a keyboard.

[0130] In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.

[0131] In some embodiments, the sensors may include, but are not limited to, gyroscope sensors, accelerometers, proximity sensors, ambient light sensors, and positioning units. The positioning unit may also be part of or interact with the network interface 1420 to communicate with components of a positioning network (e.g., global positioning system satellites).

[0132] It will be understood that, as used herein, the term "module" may refer to or include an application-specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and / or memory that executes one or more software or firmware programs, a combinational logic circuit, and / or other appropriate hardware components that provide the described functionality, or may be part of these hardware components.

[0133] It is understood that in each embodiment of the present application, the processor may be a microprocessor, a digital signal processor, a microcontroller, etc., and / or any combination thereof. According to another aspect, the processor may be a single-core processor, a multi-core processor, etc., and / or any combination thereof.

[0134] The embodiments disclosed in the present application may be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application may be implemented as a computer program or program code executed on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.

[0135] Program code may be applied to input instructions to perform the functions described herein and to generate output information. The output information may be applied to one or more output devices in a known manner. For purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit, or a microprocessor.

[0136] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0137] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including but not limited to floppy disks, optical disks, optical disks, read-only memories (CD-read-only memories, CD-ROMs), magneto-optical disks, read-only memories (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical cards, flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Accordingly, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0138] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0139] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0140] It should be noted that, in the examples and description of the present application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of further restrictions, the elements defined by the statement "comprise one" do not exclude the presence of other identical elements in the process, method, article or equipment including the elements.

[0141] Although the present application has been illustrated and described with reference to certain embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present application.

Claims

1. A method of association, characterized in that: The method comprises: Obtain the indication information of the target traffic light at different times; Acquire driving trajectory information of each of the multiple vehicles, wherein the driving trajectory information is used to indicate the position and driving state of each of the vehicles at the different times, and one piece of driving trajectory information corresponds to one lane; Based on the driving trajectory information of the multiple vehicles and the indication information of the target traffic lights at different times, determining a target vehicle from the multiple vehicles, wherein when the target vehicle is at a target position, the driving state of the target vehicle matches the indication information of the target traffic light; The lane corresponding to the driving trajectory information of the target vehicle is determined as the associated lane of the target traffic light, wherein the target position includes the position of the road marking corresponding to the target traffic light.

2. The method according to claim 1, characterized in that The determining of a target vehicle from a plurality of vehicles based on the driving trajectory information of the plurality of vehicles and the indication information of the target traffic lights at different times comprises: Based on the driving track information of the to-be-matched vehicle among the multiple vehicles, determining the to-be-matched driving state when the to-be-matched vehicle is at the target position; Based on the indication information of the target traffic light at the different moments, determining the indication information to be matched of the target traffic light when the to-be-matched vehicle is at the target position; The driving speed corresponding to the to-be-matched driving state indication is matched with the to-be-matched indication information, and the to-be-matched vehicle is determined as a target vehicle.

3. The method according to claim 2, characterized in that The driving speed corresponding to the driving state indication to be matched is matched with the indication information to be matched, and the vehicle to be matched is determined as a target vehicle, including: If the driving speed corresponding to the to-be-matched driving state indication is greater than a speed threshold and the to-be-matched indication information is green, the to-be-matched vehicle is determined as a target vehicle.

4. The method according to claim 2, characterized in that: The time when the to-be-matched vehicle is at the target position includes a first time and a second time, the first time and the second time are different, the to-be-matched driving state includes a first driving state corresponding to the first time and a second driving state corresponding to the second time, and the to-be-matched indication information includes first indication information corresponding to the first time and second indication information corresponding to the second time, The driving speed corresponding to the driving state indication to be matched is matched with the indication information to be matched, and the vehicle to be matched is determined as a target vehicle, including: The first driving speed indicated by the first driving state is greater than the speed threshold, the first indication information is green, the second driving speed indicated by the second driving state is less than or equal to the speed threshold, and the second indication information is red, and the vehicle to be matched is determined as the target vehicle.

5. The method according to claim 3 or 4, characterized in that: The driving speed indicated by the driving state to be matched includes the driving rate and driving direction of the vehicle to be matched, and the driving direction includes the direction in which the vehicle to be matched is traveling toward the target traffic light.

6. The method according to claim 1, characterized in that The target traffic light is located at a first position, and obtaining indication information of the target traffic light at different times includes: Acquire a captured image, wherein the captured image is obtained based on the camera in each vehicle captured at the different times, the captured image includes image information of the first traffic light, and the image information includes position information of the first traffic light in the captured image and indication information of the first traffic light; Based on the position of each vehicle at the first moment and the first position of the target traffic light, determining predicted position information of the target traffic light in the captured image at the first moment; Determining that first position information of the first traffic light in first image information matches the predicted position information, wherein the first image information is image information of an image captured at the first moment; Based on the indication information of the first traffic light in the captured image, the indication information of the target traffic light at the different moments is determined.

7. The method according to any one of claims 1 to 4 and 6, characterized in that: The road marking includes any one of a stop line and a zebra crossing.

8. An electronic device, characterized in that: include: one or more processors; One or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the association method described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on an electronic device, enable the electronic device to execute the association method according to any one of claims 1 to 7.

10. A computer program product, characterized in that include: Computer instructions, when the computer instructions are executed on an electronic device, enable the electronic device to execute the association method according to any one of claims 1 to 7.

Images