Correlation method, electronic device, storage medium and program product

Abstract

This application relates to the field of computer technology and discloses an association method, electronic device, storage medium, and program product. In this method, the electronic device, based on the driving trajectory information of each of multiple vehicles and the indication information of target traffic lights at different times, identifies a target vehicle from among the multiple vehicles whose driving status matches the indication information of the target traffic light when the vehicle is at a target location (e.g., a road marking such as a stop line corresponding to the target traffic light). The lane corresponding to the driving trajectory information of the target vehicle is then used as the associated lane for the target traffic light. This method determines the associated lane for the target traffic light based on the driving status of the vehicle at the target location and the indication information of the target traffic light, and the determined associated lane has a high degree of accuracy.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an association method, electronic device, storage medium, and program product. Background Art

[0002] In the field of autonomous driving, when an autonomous vehicle is about to pass through an intersection with multiple traffic lights, it needs to determine the traffic light corresponding to the current lane for safe driving. The autonomous vehicle then determines whether it can pass the intersection based on the corresponding traffic light's indication (e.g., red, yellow, or green). For example, if the traffic light corresponding to the current lane indicates green, the autonomous vehicle determines that it can pass the intersection; if the traffic light corresponding to the current lane indicates red, the autonomous vehicle determines that it is prohibited from passing the intersection.

[0003] Therefore, to enable an autonomous vehicle to determine the traffic light corresponding to its current lane, it can be trained using multiple data pairs consisting of lanes and traffic lights. The lanes and traffic lights in each data pair are associated with each other, meaning the traffic light information in each data pair indicates whether a vehicle traveling in that lane is passable.

[0004] In some solutions, the association between lanes and traffic lights in each data pair is manually annotated within the 3D point cloud data. However, manual annotation can lead to misidentification of lane-traffic light associations, reducing the accuracy of the resulting associations. Furthermore, manual annotation can also result in inefficient association determination. Summary of the Invention

[0005] The present invention provides an association method, electronic device, storage medium, and program product. The method can automatically determine the associated lanes of a target traffic light, thereby improving the efficiency and accuracy of determining the associated lanes.

[0006] In a first aspect, the present application provides an association method, which is applied to an electronic device, and the method includes: obtaining indication information of a target traffic light at different times; obtaining driving trajectory information of each vehicle among multiple vehicles, wherein the driving trajectory information is used to indicate the position and driving status of each vehicle at different times, and one driving trajectory information corresponds to one lane; based on the driving trajectory information of multiple vehicles and the indication information of the target traffic lights at different times, determining a target vehicle from multiple vehicles, wherein when the target vehicle is at a target position, the driving status of the target vehicle matches the indication information of the target traffic light; determining the lane corresponding to the driving trajectory information of the target vehicle as an associated lane of the target traffic light, wherein the target position includes the position of the road marking corresponding to the target traffic light.

[0007] The association method provided in this application enables electronic equipment to automatically determine the associated lane of a target traffic light. Compared to manual determination, this method can improve the efficiency of determining the associated lane of a target traffic light, specifically, the efficiency of associating the target traffic light with the lane. Furthermore, this method determines the associated lane of a target traffic light based on the matching of the vehicle's driving state and the target traffic light's indication information when the vehicle is at the target location, resulting in a higher degree of accuracy in the associated lanes determined using this method.

[0008] In a possible implementation of the first aspect, a target vehicle is determined from multiple vehicles based on driving trajectory information of multiple vehicles and indication information of target traffic lights at different times, including: determining a driving state to be matched when the vehicle to be matched is at a target position based on the driving trajectory information of the vehicle to be matched among the multiple vehicles; determining indication information to be matched of the target traffic light when the vehicle to be matched is at the target position based on the indication information of the target traffic lights at different times; and matching the driving speed corresponding to the driving state indication and the indication information to be matched, and determining the vehicle to be matched as the target vehicle.

[0009] In a possible implementation of the first aspect, the driving speed corresponding to the driving state indication to be matched matches the indication information to be matched, and the vehicle to be matched is determined as the target vehicle, including: the driving speed corresponding to the driving state indication to be matched is greater than a speed threshold, and the indication information to be matched is green, and the vehicle to be matched is determined as the target vehicle.

[0010] It can be understood that if the matching vehicle is at the target location, its speed is greater than the speed threshold, and the target traffic light is green, then the matching vehicle has a tendency to travel when the target traffic light is green. Therefore, the matching vehicle can be used as the target vehicle, and the lane associated with the target traffic light can be further determined based on the target vehicle. This approach ensures the accuracy of the target vehicle, and thus the accuracy of the lane associated with the target traffic light.

[0011] In a possible implementation of the first aspect, the moment when the vehicle to be matched is at the target position includes a first moment and a second moment, the first moment and the second moment are different, the driving state to be matched includes a first driving state corresponding to the first moment and a second driving state corresponding to the second moment, the indication information to be matched includes first indication information corresponding to the first moment and second indication information corresponding to the second moment, the driving speed corresponding to the driving state indication to be matched matches the indication information to be matched, and the vehicle to be matched is determined as the target vehicle, including: the first driving speed corresponding to the first driving state indication is greater than the speed threshold, the first indication information is green, the second driving speed indicated by the second driving state is less than or equal to the speed threshold, and the second indication information is red, and the vehicle to be matched is determined as the target vehicle.

[0012] It's understandable that if a lane doesn't have a corresponding associated traffic light, the speed of vehicles traveling in that lane may exceed the speed threshold regardless of whether the target traffic light indicates red or green. If the target vehicle is determined solely based on the vehicle's speed exceeding the threshold at the target location and the target traffic light indicating green, vehicles traveling in lanes without associated traffic lights may be mistakenly identified as target vehicles. This approach results in lower accuracy in determining the target vehicle, and consequently, in determining the lane associated with the target traffic light.

[0013] Therefore, the electronic device can determine that the vehicle is a target vehicle when it is determined that when the vehicle is at the target position at the first moment, the driving speed is greater than the speed threshold and the indication information of the target traffic light is green, and when the vehicle is at the target position at the second moment, the driving speed is less than or equal to the speed threshold and the indication information of the target traffic light is red.

[0014] This approach can avoid identifying vehicles traveling on lanes without corresponding associated traffic lights as target vehicles, thereby improving the accuracy of identifying target vehicles and further improving the accuracy of identifying associated lanes of target traffic lights.

[0015] In a possible implementation of the first aspect, the driving speed of the driving state indication to be matched includes a driving rate and a driving direction of the vehicle to be matched, and the driving direction includes a direction in which the vehicle to be matched is traveling toward a target traffic light.

[0016] In a possible implementation of the first aspect, the target traffic light is located at a first position, and obtaining indication information of the target traffic light at different times includes: obtaining captured images, the captured images are obtained based on the camera in each vehicle at different times, the captured images include image information of the first traffic light, the image information includes position information of the first traffic light in the captured image and indication information of the first traffic light; based on the position of each vehicle at the first moment and the first position of the target traffic light, determining the predicted position information of the target traffic light in the captured image at the first moment; determining that the first position information and the predicted position information of the first traffic light in the first image information match, wherein the first image information is the image information of the captured image at the first moment; based on the indication information of the first traffic light in the captured image, determining the indication information of the target traffic light at different times.

[0017] It's understandable that the 3D point cloud data acquired by the electronic device includes the first position of the target traffic light. However, the electronic device cannot determine the indication information of the target traffic light at different times based on the 3D point cloud data. Therefore, the electronic device needs to correlate the target traffic light at the first position with the captured image including the first (or target) traffic light to determine the indication information of the target traffic light at different times.

[0018] In a possible implementation of the first aspect, the lane corresponding to the driving trajectory information of the first vehicle in each vehicle is determined based on the following method: determining the lane centerline information of each lane among multiple lanes; and taking the lane corresponding to the lane centerline information with the smallest distance difference between the lane centerline information and the driving trajectory information of the first vehicle among the multiple lane centerline information as the lane corresponding to the driving trajectory information of the first vehicle.

[0019] It will be appreciated that the driving trajectory information of the first vehicle indicates the position of the first vehicle at different times. Therefore, the electronic device can determine the sum of the shortest distances from the position of the first vehicle to the centerline of each lane at each time. The sum of the shortest distances from the position of the first vehicle to the centerline of each lane at each time can be calculated as the distance difference between the lane centerline information and the driving trajectory information of the first vehicle.

[0020] Then, the lane corresponding to the lane centerline information with the smallest sum of the shortest distances among the multiple lane centerline information is used as the lane corresponding to the driving trajectory information of the first vehicle. The lane corresponding to the driving trajectory information of the first vehicle obtained based on this method has a higher accuracy.

[0021] In a possible implementation of the first aspect, the method further includes: determining that the target vehicle is at the target position when a distance between a vehicle center point corresponding to the target vehicle and the target position is less than or equal to a distance threshold.

[0022] The road markings include any of a stop line and a zebra crossing. For situations where the target location is a stop line or a zebra crossing, the distance between the center point of the target vehicle and the target location may be the distance between the center point of the vehicle and a target point in the target location. The target point in the target location may be, for example, the midpoint of the stop line or the center point of an area corresponding to a zebra crossing.

[0023] In a second aspect, the present application provides an electronic device comprising: one or more processors; one or more memories; one or more memories storing one or more programs, which, when the one or more programs are executed by one or more processors, enables the electronic device to execute the first aspect and any associated method of any possible implementation of the first aspect.

[0024] In a third aspect, the present application provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed on an electronic device, the electronic device executes the first aspect and any possible implementation method of the first aspect.

[0025] In a fourth aspect, the present application provides a computer program product, which includes: computer instructions, which, when executed on an electronic device, enable the electronic device to execute the first aspect and any possible implementation method of the first aspect.

[0026] Among them, the beneficial effects of the second to fourth aspects can refer to the beneficial effects of the first aspect and any possible implementation of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 According to some embodiments of the present application, a flowchart of a first association method is shown;

[0028] Figure 2 According to some embodiments of the present application, a flow chart of a method for determining indication information of a target traffic light at different times is shown;

[0029] Figure 3 According to some embodiments of the present application, a schematic diagram of a scene including multiple vehicles and a first traffic light is shown;

[0030] Figure 4 According to some embodiments of the present application, a schematic diagram of a scene including multiple vehicles and multiple target traffic lights is shown;

[0031] Figure 5 According to some embodiments of the present application, a schematic diagram of a scene including multiple vehicles and a target traffic light is shown;

[0032] Figure 6According to some embodiments of the present application, a flow chart of a second association method is shown;

[0033] Figure 7 According to some embodiments of the present application, a schematic structural diagram of an electronic device is shown. DETAILED DESCRIPTION

[0034] Illustrative embodiments of the present application include, but are not limited to, associated methods, electronic devices, storage media, and program products.

[0035] It can be understood that the association method provided in the embodiments of the present application is applicable to electronic devices including, but not limited to, electronic devices with the function of marking association relationships, such as tablet computers, computers with wireless transceiver functions, virtual reality (VR) devices, augmented reality (AR) devices, wireless devices in industrial control, wireless devices in self-driving, wireless devices in remote medical surgery, wireless devices in smart grids, wireless devices in transportation safety, wireless devices in smart cities, wireless devices in smart homes, and the like.

[0036] Based on the above, manual labeling may lead to incorrect determination of the association between lanes and traffic lights, reducing the accuracy of the obtained association. In addition, manual labeling also leads to low efficiency in determining the association.

[0037] Therefore, in order to solve the above technical problems, the present application provides an association method. This method automatically determines the associated lane of the target traffic light based on the driving state of the vehicle at the target position and the indication information of the target traffic light when the vehicle is at the target position. Specifically, the electronic device can determine the target vehicle from multiple vehicles based on the driving trajectory information of multiple vehicles and the indication information of the target traffic lights at different times, and then determine the lane corresponding to the driving trajectory information of the target vehicle as the associated lane of the target traffic light. Among them, when the target vehicle is at the target position, the driving state indicated by the driving trajectory information of the target vehicle matches the indication information of the target traffic light.

[0038] In some embodiments, the target position includes a position of a road marking corresponding to the target traffic light, and the road marking includes any one of a stop line and a zebra crossing.

[0039] The association method provided in this application enables electronic equipment to automatically determine the associated lane of a target traffic light. Compared to manual determination, this method can improve the efficiency of determining the associated lane of a target traffic light, specifically, the efficiency of associating the target traffic light with the lane. Furthermore, this method determines the associated lane of a target traffic light based on the matching of the vehicle's driving state and the target traffic light's indication information when the vehicle is at the target location, resulting in a higher degree of accuracy in the associated lanes determined using this method.

[0040] The association method provided in this application is described in detail below with reference to the accompanying drawings. Figure 1 A schematic flow chart of the first association method provided by the present application is shown. Figure 1 The execution subjects of each step of the process shown are all electronic devices. For the convenience of description, Figure 1 The execution entities of each step will not be described repeatedly in the steps of the process shown. Figure 1 As shown, the method includes but is not limited to the following schemes:

[0041] S101: Obtain indication information of a target traffic light at different times.

[0042] It is understood that the application scenario of the association method provided in this application is to determine the associated lane of a target traffic light in three-dimensional point cloud data. The three-dimensional point cloud data includes the target traffic light at a first position. In other words, the electronic device can obtain the three-dimensional point cloud data and then determine the target traffic light and the first position of the target traffic light in the three-dimensional point cloud data.

[0043] It is understood that the electronic device can determine the target traffic light and its first position based on the three-dimensional point cloud data, but cannot obtain the indication information of the target traffic light. The first position of the target traffic light is the three-dimensional position of the target traffic light, and this first position can be obtained based on the three-dimensional point cloud data. It is understood that three-dimensional point cloud data records objects in a specific area (such as the target traffic light in this application) in the form of points, and each point corresponds to three-dimensional coordinate information. Therefore, based on the three-dimensional point cloud data, the electronic device can obtain the three-dimensional coordinate information corresponding to the target traffic light in the three-dimensional point cloud data, that is, obtain the first position.

[0044] In some embodiments, the target traffic light and the first position in the three-dimensional point cloud data may be pre-labeled. This application does not limit the labeling method of the target traffic light and the first position, and the target traffic light and the first position may be manually labeled or automatically labeled.

[0045] Next, in the present application, the electronic device can determine the indication information of the target traffic light at different times based on the first position of the target traffic light and the captured images of the vehicle at different times. The indication information of the target traffic light may include red, green and yellow.

[0046] In some embodiments, as Figure 2 As shown, the method for determining the indication information of the target traffic light at different times includes:

[0047] S201: Acquire captured images, where the captured images are captured by a camera in each vehicle at different times. The captured images include image information of a first traffic light, and the image information includes position information of the first traffic light in the captured images and indication information of the first traffic light.

[0048] Figure 3 A schematic diagram of a scene including multiple vehicles and a first traffic light is shown. It is understood that the vehicles and the first traffic light in this application are obtained by annotating 3D point cloud data. However, for ease of illustration and understanding, this application does not represent the vehicles and the first traffic light in the form of 3D point cloud data.

[0049] like Figure 3 As shown, Figure 3 The first traffic light 301, vehicles A1, B1, C1, and D1 are included. Vehicles A1 and B1 are located in lane M1, and vehicles C1 and D1 are located in lane N1.

[0050] It can be understood that since vehicles A1, B1, C1 and D1 are traveling toward the first traffic light 301, the captured images obtained by the electronic device can be, for example, images of vehicles A1, B1, C1 and D1 taken at different times based on the cameras in front of the vehicles.

[0051] The captured image includes image information of the first traffic light, for example, two-dimensional position information of the first traffic light in the captured image and indication information of the first traffic light (for example, red, green, yellow).

[0052] It can be understood that after the electronic device obtains the captured images captured by the camera of each vehicle at different times, it needs to further execute subsequent S202 and S203 to determine whether the first traffic light in the captured image is the target traffic light at the first position.

[0053] It can be understood that since the electronic device needs to determine whether the first traffic light is the target traffic light based on the captured image obtained by the vehicle, the vehicle in this application can be a vehicle traveling towards the target traffic light, and the distance between the vehicle and the target traffic light meets the distance condition, so as to avoid the situation where the vehicle is too far away from the target traffic light and the first traffic light that is different from the target traffic light is mistakenly regarded as the target traffic light.

[0054] S202: Based on the position of each vehicle at the first moment and the first position of the target traffic light, determine the predicted position information of the target traffic light in the captured image at the first moment.

[0055] For the first moment among different moments, the electronic device can determine the predicted position information of the target traffic light in the captured image at the first moment based on the position of each vehicle at the first moment, the first position of the target traffic light, and the internal and external parameters of the camera on each vehicle.

[0056] The camera's intrinsic parameters describe its internal geometric properties and typically include focal length, principal point, and distortion parameters. The camera's extrinsic parameters describe its position and orientation within the world coordinate system. In this application, these parameters can be used to describe the camera's position and orientation relative to the corresponding vehicle. It can be understood that the camera's intrinsic and extrinsic parameters determine how the camera maps the target traffic light into a two-dimensional image (e.g., a captured image).

[0057] Therefore, in S202, the electronic device can determine the predicted position information of the target traffic light in the captured image at the first moment based on the position of each vehicle at the first moment, the first position of the target traffic light, and the internal and external parameters of the camera on each vehicle.

[0058] The embodiments of the present application do not limit the method for determining the predicted position information of the target traffic light in the captured image at the first moment. For example, the electronic device can obtain the predicted position information of the target traffic light in the captured image at the first moment based on a pre-trained mapping model.

[0059] S203: Determine whether first position information of the first traffic light in the first image information matches the predicted position information, wherein the first image information is image information of an image captured at a first moment.

[0060] It is understood that after the electronic device determines the predicted position information of the target traffic light in the captured image at the first moment, it matches the predicted position information with the first position information in the first image information of the captured image at the first moment. In some embodiments, if the first position information and the predicted position information are the same, or the difference between the first position information and the predicted position information is less than a difference threshold, it can be determined that the first position information and the predicted position information match.

[0061] The first image information is image information of an image captured by the vehicle at the first moment of the first traffic light, including the first position information of the first traffic light in the captured image. The predicted position information is the predicted position information of the target traffic light in the captured image at the first moment, predicted based on the vehicle's position information at the first moment, the first position of the target traffic light, and the camera's internal and external parameters. Specifically, the predicted position information can be understood as the position information of the target traffic light in the image captured if the vehicle actually captured the target traffic light in the first position at the first moment using a camera with corresponding internal and external parameters.

[0062] Therefore, when an electronic device has the first image information, if it wants to determine whether the target traffic light at a known position (i.e., the first position) is the first traffic light, it can determine whether the first position information in the first image information matches the predicted position information corresponding to the target traffic light at the first moment.

[0063] If the electronic device determines that the predicted location information matches the first location information, it can be determined that the first traffic light corresponding to the first location information and the target traffic light corresponding to the predicted location information are the same traffic light, that is, the first traffic light is the target traffic light.

[0064] It is understandable that Figure 3 As shown, at the first moment, there can be multiple vehicles (such as vehicle A1, vehicle B1, vehicle C1, vehicle D1) taking pictures based on the camera, and each vehicle can be equipped with multiple cameras, so the electronic device can obtain multiple captured images corresponding to the first moment, that is, obtain multiple first position information.

[0065] In some embodiments, the electronic device may determine that the first traffic light is the target traffic light when the number of first location information that matches the predicted location information in multiple first location information is greater than a number threshold.

[0066] For example, if the number of vehicles photographed at a first moment is a, and each vehicle has b cameras, the electronic device can obtain a×b images corresponding to the first moment, that is, a×b pieces of first position information. The electronic device then determines the number S of first position information that matches the predicted position information among the a×b pieces of first position information. If the number S is greater than a threshold value S', the first traffic light can be determined to be the target traffic light.

[0067] The embodiment of the present application does not limit the method for determining the quantity threshold, which can be set based on experience or flexibly adjusted according to actual application scenarios.

[0068] S204: Determine the indication information of the target traffic light at different times based on the indication information of the first traffic light in the captured image.

[0069] After the electronic device determines that the first position information and the predicted position information match, it can determine that the first traffic light in the captured image is the target traffic light at the first position, and then can use the indication information of the first traffic light in the first image information as the indication information of the target traffic light at the first moment.

[0070] It is understandable that the method for determining the indication information of the target traffic light at each first moment in different moments is similar, so no further description is given. Therefore, based on the above method, the indication information of the target traffic light at different moments can be determined.

[0071] In some embodiments, the above-mentioned method of determining the indication information of the target traffic light at different times may also be referred to as a three-dimensional (information)-two-dimensional (information) association of the target traffic light.

[0072] S102: Acquire driving trajectory information of each vehicle among a plurality of vehicles, wherein the driving trajectory information is used to indicate the position and driving status of each vehicle at different times, and one driving trajectory information corresponds to one lane.

[0073] In an embodiment of the present application, the electronic device can determine the driving trajectory information of each vehicle based on the three-dimensional point cloud data. The driving state indicated by the driving trajectory information can be used to indicate the driving speed of the vehicle. In some embodiments, the driving speed of the vehicle includes the vehicle's driving rate and driving direction, and the driving direction includes the direction of the vehicle's travel toward the target traffic light.

[0074] The driving track information of each vehicle in the three-dimensional point cloud data may be pre-labeled. This application does not limit the labeling method of the driving track information of each vehicle, which may be manually labeled or automatically labeled.

[0075] In the embodiment of the present application, each vehicle's driving trajectory information corresponds to a lane. It will be understood that the three-dimensional point cloud data acquired by the electronic device includes each vehicle's driving trajectory information and pre-labeled lanes. However, each vehicle's driving trajectory information and lanes are not mutually associated. Therefore, the electronic device can associate each vehicle's driving trajectory information with the lane, that is, determine the lane corresponding to each vehicle's driving trajectory information.

[0076] Among them, the method for determining the lane corresponding to the driving trajectory information of the first vehicle in each vehicle includes: determining the lane centerline information of each lane among multiple lanes; and taking the lane corresponding to the lane centerline information with the smallest distance difference between the lane centerline information and the driving trajectory information of the first vehicle among the multiple lane centerline information as the lane corresponding to the driving trajectory information of the first vehicle.

[0077] Figure 4FIG. 1 shows a schematic diagram of a scene including multiple vehicles and multiple target traffic lights. Figure 4 As shown, Figure 4 The first target traffic light 401, the second target traffic light 402, vehicles A2, B2, C2 and D2 are included. Figure 4 The lane M2, lane N2 and lane P2 are included. Among them, lane M2 and lane N2 are through lanes, and lane P2 is a right turn lane.

[0078] Below is Figure 4 Taking vehicle C2 as the first vehicle as an example, a method for determining the lane corresponding to the driving trajectory information of vehicle C2 is described.

[0079] It will be appreciated that lanes M2, N2, and P2 are pre-labeled in the 3D point cloud data. Therefore, the electronic device can determine the 3D position information of lanes M2, N2, and P2 based on the 3D point cloud data, and further determine the lane centerline information for each lane. For example, the lane centerline information for lane M2 may indicate centerline m2; the lane centerline information for lane N2 may indicate centerline n2; and the lane centerline information for lane P2 may indicate centerline p2.

[0080] It will be appreciated that the driving trajectory information of vehicle C2 indicates the location of vehicle C2 at different times. Therefore, the electronic device can determine the sum of the shortest distances from the location of vehicle C2 to each centerline (e.g., centerline m2, centerline n2, and centerline p2) at each time. The sum of the shortest distances from the location of vehicle C2 to each centerline at each time can be calculated as the distance difference between the lane centerline information and the driving trajectory information of vehicle C2.

[0081] For example, if the sum of the shortest distances from the position of vehicle C2 to the center line n2 is the smallest at each moment, then the lane N2 corresponding to the center line n2 can be determined to be the lane corresponding to the driving trajectory of vehicle C2.

[0082] Figure 4 The method for determining the lane corresponding to the driving trajectory information of each vehicle in is similar, so it will not be described in detail. Figure 4 In the figure, the lane corresponding to the driving trajectory information of vehicle A2 is lane M2; the lane corresponding to the driving trajectory information of vehicle B2 is lane M2; the lane corresponding to the driving trajectory information of vehicle C2 is lane N2; and the lane corresponding to the driving trajectory information of vehicle D2 is lane P2.

[0083] In some embodiments, the process of determining the lane corresponding to the driving trajectory information of each vehicle may also be referred to as trajectory information-lane association.

[0084] S103: Based on the driving trajectory information of multiple vehicles and the indication information of the target traffic lights at different times, a target vehicle is determined from the multiple vehicles, wherein when the target vehicle is at the target position, the driving state of the target vehicle matches the indication information of the target traffic light.

[0085] In some embodiments, S103 may further include: determining the driving state to be matched when the vehicle to be matched is at the target position based on the driving trajectory information of the vehicle to be matched among multiple vehicles; determining the indication information to be matched of the target traffic light when the vehicle to be matched is at the target position based on the indication information of the target traffic light at different times; matching the driving speed corresponding to the driving state indication to be matched with the indication information to be matched, and determining the vehicle to be matched as the target vehicle.

[0086] In some embodiments, the target location includes the location of a road marking corresponding to the target traffic light, where the road marking may include a stop line or a zebra crossing. The location of the road marking included in the target location may be a pre-set location. For example, if the road marking is a stop line, the stop line is pre-marked in the three-dimensional point cloud data. Thus, the electronic device determines information such as the location of the stop line based on the three-dimensional point cloud data.

[0087] The embodiments of this application do not limit the method for determining whether the target vehicle is at the target location. In some embodiments, the target vehicle is determined to be at the target location if the distance between the center point of the target vehicle and the target location is less than or equal to a distance threshold. This application also does not limit the size of the distance threshold; it can be set based on experience or flexibly adjusted according to actual application scenarios.

[0088] Wherein, for a case where the target location is a stop line, a zebra crossing, etc., the distance between the center point of the target vehicle and the target location may be the distance between the center point of the target vehicle and a target point in the target location. The target point in the target location may be, for example, the midpoint of the stop line or the center point of an area corresponding to a zebra crossing.

[0089] It is understood that the number of target vehicles in the embodiment of the present application can be one or more, and the embodiment of the present application does not limit this. The following takes the number of target vehicles as an example to describe the method of determining the target vehicles.

[0090] In some examples, the method of determining the target vehicle based on matching the driving speed of the driving state indication to be matched with the indication information to be matched in S103 includes the following two methods.

[0091] Method 1:

[0092] If the driving speed corresponding to the driving state indication to be matched is greater than the speed threshold and the indication information to be matched is green, the vehicle to be matched is determined as the target vehicle.

[0093] Road markings Figure 4 For example, for the first target traffic light 401, if the driving speed of vehicle A2 at stop line S1 is greater than the speed threshold, and the indication information of the first target traffic light 401 is green at this time, the electronic device may identify vehicle A2 as the target vehicle corresponding to the first target traffic light 401. If the driving speed of vehicle C2 at stop line S1 is greater than the speed threshold, and the indication information of the first target traffic light 401 is green at this time, the electronic device may identify vehicle C2 as the target vehicle corresponding to the first target traffic light 401.

[0094] If the speed of vehicle D2 at stop line S1 is less than or equal to the speed threshold, or the indication information of the first target traffic light 401 is not green at this time, the electronic device can determine that vehicle D2 is not the target vehicle corresponding to the first target traffic light 401. If the speed of vehicle D2 at stop line S1 is greater than the speed threshold, and the indication information of the second target traffic light 402 is green at this time, the electronic device can determine that vehicle D2 is the target vehicle corresponding to the second target traffic light 402.

[0095] In the above example, both vehicle A2 and vehicle C2 can serve as target vehicles corresponding to the first target traffic light 401 ; and vehicle D2 can serve as the target vehicle corresponding to the second target traffic light 402 .

[0096] However, in some scenarios, some lanes may not have corresponding associated traffic lights. Figure 5 A schematic diagram of a scene including multiple vehicles and a target traffic light is shown. Figure 5 As shown, Figure 5 The third target traffic light 501, vehicle A3, vehicle B3, vehicle C3 and vehicle D3 are included. Figure 5 The lanes M3, N3 and P3 are included. Among them, lanes M3 and N3 are through lanes, and lane P3 is a right-turn lane.

[0097] Figure 5In this scenario, right-turn lane P3 has no associated traffic light. This means that regardless of whether the third target traffic light 501 indicates red or green, the speed of vehicle D3 traveling in this lane may exceed the speed threshold. Therefore, if method 1 is still used to determine the target vehicles in this scenario, the speeds of vehicles A3, C3, and D3 at the target location (e.g., stop line S2) may all exceed the speed threshold. Furthermore, at this moment, the third target traffic light 501 indicates green. In this case, the electronic device will identify vehicles A3, C3, and D3 as the target vehicles corresponding to the third target traffic light 501. This reduces the accuracy of the target vehicles and, in turn, the accuracy of the lanes associated with the third target traffic light 501.

[0098] Therefore, in order to ensure the accuracy of the associated lane of the target traffic light, in some other embodiments, the electronic device can determine the target vehicle based on the following method 2.

[0099] Method 2:

[0100] The moments when the vehicle to be matched is at the target position include a first moment and a second moment, the first moment and the second moment are different, the driving state to be matched includes a first driving state corresponding to the first moment and a second driving state corresponding to the second moment, the indication information to be matched includes first indication information corresponding to the first moment and second indication information corresponding to the second moment, the driving speed corresponding to the driving state indication to be matched matches the indication information to be matched, and the vehicle to be matched is determined as the target vehicle, including: the first driving speed corresponding to the first driving state indication is greater than the speed threshold, the first indication information is green, the second driving speed indicated by the second driving state is less than or equal to the speed threshold, and the second indication information is red, and the vehicle to be matched is determined as the target vehicle.

[0101] Continue as Figure 5 As shown, the road markings are Figure 5 Taking the stop line S2 in the figure as an example, for the third target traffic light 501, if the driving speed of vehicle A3 when it is located at the stop line S2 at the first moment is greater than the speed threshold, and the indication information of the third target traffic light 501 at the first moment is green, and the driving speed of vehicle A3 when it is located at the stop line S2 at the second moment is less than or equal to the speed threshold, and the indication information of the third target traffic light 501 at the second moment is red, it means that vehicle A3 has a moving trend when the third target traffic light 501 displays green, and has a stopping trend when the third target traffic light 501 displays red. At this time, vehicle A3 can be determined as the target vehicle.

[0102] The embodiment of the present application does not limit the order of the first moment and the second moment. The first moment can be a moment before the second moment or a moment after the second moment.

[0103] For another example, since lane P3, corresponding to vehicle D3's trajectory, has no associated traffic light, vehicle D3 maintains a driving trend regardless of whether the third target traffic light 501 is green or red. For example, when the third target traffic light 501 is red, vehicle D3's speed may still exceed the speed threshold. Therefore, vehicle D3 is not included in the target vehicles determined using method 2.

[0104] This method determines the target vehicle based on the indication information of the target traffic light for the driving status of the vehicle in red and green conditions, so that the accuracy of the determined target vehicle is higher, and thus the accuracy of the associated lane of the determined target traffic light is higher.

[0105] S104: Determine the lane corresponding to the driving trajectory information of the target vehicle as the associated lane of the target traffic light.

[0106] It can be understood that after the electronic device determines the target vehicle among multiple vehicles, it can use the lane corresponding to the driving trajectory information of the target vehicle as the associated lane of the target traffic light.

[0107] For example, after determining the target vehicles (vehicle A2 and vehicle C2) corresponding to the first target traffic light 401 based on method one, the electronic device can use the lanes corresponding to the driving trajectory information of vehicles A2 and C2 (such as lane M2 and lane N2) as the associated lanes of the first target traffic light 401.

[0108] For another example, after determining the target vehicles (vehicle A3 and vehicle C3) corresponding to the third target traffic light 501 based on method 2, the electronic device can use the lanes corresponding to the driving trajectory information of vehicles A3 and C3 (such as lane M3 and lane N3) as the associated lanes of the third target traffic light 501.

[0109] In some embodiments, the process of determining the associated lane of a target traffic light may also be referred to as lane-target traffic light association.

[0110] Figure 6 FIG. 1 shows a flow chart of another association method provided by the present application. Figure 6 As shown, the method includes the following steps:

[0111] S601: Obtain dynamic feature annotation information.

[0112] In the embodiment of the present application, the dynamic element annotation information may include the driving trajectory information of each vehicle mentioned above. The driving trajectory information of each vehicle may be manually annotated or automatically annotated, and the present application does not limit this.

[0113] S602: Obtain static feature annotation information.

[0114] In the embodiment of the present application, the static element annotation information may include the annotation results of the lane and target position mentioned above. The lane and target position may be manually or automatically annotated, and the present application does not limit this.

[0115] S603: Determine a lane centerline representing the lane based on the lane annotation result in the static feature annotation information.

[0116] It will be appreciated that lane centerlines may be used to represent lanes.

[0117] S604: Based on the driving trajectory information and lane of each vehicle, obtain the association relationship between the driving trajectory information and the lane.

[0118] It can be understood that the method for determining the association between the driving trajectory information and the lane can be referred to the content shown in S102 above, and will not be repeated here.

[0119] S605: Acquire the corresponding captured images of the target traffic light at different times.

[0120] S606: Based on the three-dimensional position of the target traffic light in the static element annotation information and the two-dimensional position of the target traffic light in the captured image, a three-dimensional-two-dimensional association relationship of the target traffic light is obtained.

[0121] It can be understood that the step S606 is to determine the indication information of the target traffic light at the first position at different times. The specific method can be found in the above step S101 and will not be described in detail here.

[0122] S607: Based on the association between the driving trajectory information and the lanes, and the three-dimensional-two-dimensional association between the target traffic lights, an association between the target traffic lights and the lanes is obtained.

[0123] It can be understood that the method for determining the association between the target traffic light and the lane in S607 is the same as the principle of the content shown in S103 and S104 above, and will not be repeated here.

[0124] In some embodiments, if the dynamic element annotation information and the static element annotation information in the above S601 and S602 are incomplete, the electronic device can perform multiple annotations, and execute the above S601-S607 once after each annotation to continuously update the association between the target traffic light and the lane, thereby improving the accuracy of the determination result of the association relationship.

[0125] The association method provided in this application enables electronic equipment to automatically determine the associated lane of a target traffic light. Compared to manual determination, this method can improve the efficiency of determining the associated lane of a target traffic light, specifically, the efficiency of associating the target traffic light with the lane. Furthermore, this method determines the associated lane of a target traffic light based on the matching of the vehicle's driving state and the target traffic light's indication information when the vehicle is at the target location, resulting in a higher degree of accuracy in the associated lanes determined using this method.

[0126] In some embodiments, the embodiments of the present application further provide a computer-readable storage medium, on which instructions are stored. When the instructions are executed on an electronic device, the electronic device executes the association method described in the above embodiments.

[0127] In some embodiments, an embodiment of the present application also provides an electronic device, which includes: one or more processors; one or more memories; one or more memories storing one or more programs, and when the one or more programs are executed by one or more processors, the electronic device executes the association method described in the above embodiment.

[0128] In some embodiments, the embodiments of the present application further provide a computer program product, including: computer instructions, which, when executed on an electronic device, enable the electronic device to execute the association method described in the above embodiments.

[0129] Figure 7 The figure shows a schematic diagram of the structure of an electronic device 1400 provided in an embodiment of the present application. In some embodiments, the electronic device 1400 can be a vehicle computer, a computer, etc. In one embodiment, the electronic device 1400 can include one or more processors 1404, system control logic 1408 connected to at least one of the one or more processors 1404, system memory 1412 connected to the system control logic 1408, non-volatile memory (NVM) 1416 connected to the system control logic 1408, and a network interface 1420 connected to the system control logic 1408.

[0130] In some embodiments, the processor 1404 may include one or more single-core or multi-core processors. In some embodiments, the processor 1404 may include any combination of general-purpose processors and specialized processors (e.g., graphics processors, application processors, baseband processors, etc.). In embodiments where the electronic device 1400 employs an enhanced base station (ENB) or a radio access network (RAN) controller, the processor 1404 may be configured to execute various embodiments, such as Figure 1and Figure 6 One or more of the various embodiments shown.

[0131] In some embodiments, system control logic 1408 may include any suitable interface controller to provide any suitable interface to at least one of processors 1404 and / or any suitable device or component in communication with system control logic 1408 .

[0132] In some embodiments, the system control logic 1408 may include one or more memory controllers to provide an interface to the system memory 1412. The system memory 1412 may be used to load and store data and / or instructions. In some embodiments, the system memory 1412 of the electronic device 1400 may include any suitable volatile memory, such as a suitable dynamic random access memory.

[0133] The non-volatile memory 1416 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 1416 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of a hard disk drive (HDD), a compact disc (CD) drive, and a digital versatile disc (DVD) drive.

[0134] The non-volatile memory 1416 may include a portion of storage resources on the device on which the electronic device 1400 is installed, or it may be accessible to the device but not necessarily a portion of the device. For example, the non-volatile memory 1416 may be accessed via the network interface 1420 over a network.

[0135] In particular, the system memory 1412 and the non-volatile memory 1416 may each include a temporary copy and a permanent copy of the instructions 1424. The instructions 1424 may include instructions that, when executed by at least one of the processors 1404, cause the electronic device 1400 to perform the following operations: Figure 1 and Figure 6 In some embodiments, instructions 1424 , hardware, firmware, and / or software components thereof may additionally or alternatively reside in system control logic 1408 , network interface 1420 , and / or processor 1404 .

[0136] The network interface 1420 may include a transceiver for providing a radio interface for the electronic device 1400, thereby communicating with any other suitable devices (such as a front-end module, an antenna, etc.) via one or more networks. In some embodiments, the network interface 1420 may be integrated with other components of the electronic device 1400. For example, the network interface 1420 may be integrated with at least one of the processor 1404, the system memory 1412, the non-volatile memory 1416, and a firmware device (not shown) having instructions. When at least one of the processors 1404 executes the instructions, the electronic device 1400 implements the following. Figure 1 and Figure 6 The method shown.

[0137] The network interface 1420 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 1420 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.

[0138] In one embodiment, at least one of the processors 1404 may be packaged together with logic for one or more controllers of the system control logic 1408 to form a system in package (SiP). In one embodiment, at least one of the processors 1404 may be integrated on the same die with logic for one or more controllers of the system control logic 1408 to form a system on chip (SoC).

[0139] Electronic device 1400 may further include input / output (I / O) devices 1432. I / O devices 1432 may include a user interface to enable a user to interact with electronic device 1400; peripheral component interfaces may also be designed to enable peripheral components to interact with electronic device 1400. In some embodiments, electronic device 1400 may also include a sensor for determining at least one of environmental conditions and location information related to electronic device 1400.

[0140] In some embodiments, the user interface may include, but is not limited to, a display (e.g., an LCD display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., an LED flash), and a keyboard.

[0141] In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.

[0142] In some embodiments, the sensors may include, but are not limited to, a gyroscope sensor, an accelerometer, a proximity sensor, an ambient light sensor, and a positioning unit. The positioning unit may also be part of or interact with the network interface 1420 to communicate with components of a positioning network (e.g., global positioning system satellites).

[0143] It will be understood that, as used herein, the term "module" may refer to or include an application-specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and / or memory that executes one or more software or firmware programs, a combinational logic circuit, and / or other appropriate hardware components that provide the described functionality, or may be part of these hardware components.

[0144] It is understood that in each embodiment of the present application, the processor can be a microprocessor, a digital signal processor, a microcontroller, etc., and / or any combination thereof. According to another aspect, the processor can be a single-core processor, a multi-core processor, etc., and / or any combination thereof.

[0145] The various embodiments disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of this application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0146] Program code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor, such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit, or a microprocessor.

[0147] Program code can be implemented with a high-level programming language or an object-oriented programming language to communicate with the processing system. Where necessary, program code can also be implemented in assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0148] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried by or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which can be read and executed by one or more processors. For example, the instructions may be distributed over a network or via other computer-readable media. Thus, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to floppy disks, optical disks, optical discs, read-only memories (CD-ROMs), magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical cards, flash memory, or tangible machine-readable storage for transmitting information via the Internet using electrical, optical, acoustic, or other propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.). Accordingly, machine-readable media includes any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0149] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order may not be required. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of a structural or method feature in a particular figure does not imply that such feature is required in all embodiments, and in some embodiments, such features may not be included or may be combined with other features.

[0150] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems raised by this application. In addition, in order to highlight the innovative part of this application, the above-mentioned device embodiments of this application do not introduce units / modules that are not closely related to solving the technical problems raised by this application. This does not mean that other units / modules do not exist in the above-mentioned device embodiments.

[0151] It should be noted that in the examples and description of the present application, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a" does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0152] While the present application has been shown and described with reference to certain embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present application.

Claims

1. A correlation method, characterized in that: The method comprises: Obtain the indication information of the target traffic light at different times; Acquire driving trajectory information of each of the multiple vehicles, wherein the driving trajectory information is used to indicate the position and driving status of each vehicle at the different moments, and one piece of driving trajectory information corresponds to one lane; Determining a target vehicle from the plurality of vehicles based on the driving trajectory information of the plurality of vehicles and the indication information of the target traffic lights at different moments, wherein when the target vehicle is at a target position, the driving state of the target vehicle matches the indication information of the target traffic light, and the target vehicle is determined to be at the target position when the distance between the vehicle center point and the target position is less than or equal to a distance threshold; Determining the lane corresponding to the driving trajectory information of the target vehicle as the associated lane of the target traffic light, wherein the target position includes the position of the road marking corresponding to the target traffic light; The determining of a target vehicle from the plurality of vehicles based on the driving trajectory information of the plurality of vehicles and the indication information of the target traffic lights at different times includes: Determining a driving state of the to-be-matched vehicle when the to-be-matched vehicle is at the target position based on driving trajectory information of the to-be-matched vehicle among the multiple vehicles; Based on the indication information of the target traffic light at the different moments, determining the indication information to be matched of the target traffic light when the vehicle to be matched is at the target position; matching the driving speed corresponding to the to-be-matched driving state indication with the to-be-matched indication information, and determining the to-be-matched vehicle as a target vehicle; The time when the to-be-matched vehicle is at the target position includes a first time and a second time, the first time and the second time are different, the to-be-matched driving state includes a first driving state corresponding to the first time and a second driving state corresponding to the second time, and the to-be-matched indication information includes first indication information corresponding to the first time and second indication information corresponding to the second time. The matching of the driving speed corresponding to the to-be-matched driving state indication with the to-be-matched indication information, and determining the to-be-matched vehicle as a target vehicle, includes: The first driving speed indicated by the first driving state is greater than the speed threshold, the first indication information is green, the second driving speed indicated by the second driving state is less than or equal to the speed threshold, and the second indication information is red, and the vehicle to be matched is determined as the target vehicle.

2. The method according to claim 1, characterized in that The driving speed indicated by the driving state to be matched includes the driving rate and driving direction of the vehicle to be matched, and the driving direction includes the direction in which the vehicle to be matched is traveling toward the target traffic light.

3. The method according to claim 1, characterized in that The target traffic light is located at a first position, and obtaining indication information of the target traffic light at different times includes: Acquire captured images, where the captured images are captured by the camera in each vehicle at the different times, the captured images including image information of the first traffic light, the image information including position information of the first traffic light in the captured images and indication information of the first traffic light; Determining predicted position information of the target traffic light in the captured image at the first moment based on the position of each vehicle at the first moment and the first position of the target traffic light; Determining that first position information of the first traffic light in first image information matches the predicted position information, wherein the first image information is image information of an image captured at the first moment; Based on the indication information of the first traffic light in the captured image, the indication information of the target traffic light at the different moments is determined.

4. The method according to claim 1 or 3, characterized in that The road marking includes any one of a stop line and a zebra crossing.

5. An electronic device, characterized in that: include: one or more processors; One or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the association method according to any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, which, when executed on an electronic device, enable the electronic device to execute the association method according to any one of claims 1 to 4.

7. A computer program product, characterized in that include: Computer instructions, when the computer instructions are executed on an electronic device, enable the electronic device to execute the association method according to any one of claims 1 to 4.

Images