Traffic signal light switching point prediction method, device, equipment, medium and vehicle

Abstract

This disclosure provides a method, apparatus, device, medium, and vehicle for predicting traffic light switching points, relating to the field of data processing technology, specifically big data, intelligent transportation, and cloud computing technologies. It can be applied to navigation, vehicle-to-everything (V2X) technology, smart cockpit, and autonomous driving technology. The method includes: acquiring target intersection image data; performing image recognition on the target intersection image data to obtain a target traffic light identification result; and sending the target traffic light identification result to a server. The server calculates the switching point information of the target traffic light based on the target traffic light identification result and the target traffic light's switching pattern information. This disclosure reduces the time and cost of traffic light switching point prediction, as well as the computational power consumption of the server, thereby improving the accuracy and efficiency of traffic light switching point prediction.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, specifically to technologies such as big data, intelligent transportation, and cloud computing, and can be applied to the fields of navigation, vehicle networking, smart cockpit, and autonomous driving technology. Background Technology

[0002] Traffic lights are crucial infrastructure for traffic management departments to control vehicle traffic order and adjust road traffic flow. They also represent the most complex and user-centric traffic scenario for navigation. Utilizing big data and cloud computing technologies to mine traffic light switching point information is beneficial for intelligent transportation construction. It allows for precise calculation of intersection costs, serving as auxiliary features to predict dynamic traffic conditions such as congestion and congestion dissipation. This has significant application value in various related fields, including navigation, vehicle-to-everything (V2X) technology, smart cockpits, and autonomous driving. Furthermore, displaying traffic light switching point information to users can alleviate anxiety caused by the unknown duration of red lights. Therefore, accurately identifying traffic light switching points is of paramount importance. Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, medium, and vehicle for predicting traffic light switching points, which can reduce the time, value cost, and server-side computing power consumption of traffic light switching point prediction, and improve the accuracy and efficiency of traffic light switching point prediction.

[0004] In a first aspect, embodiments of this disclosure provide a traffic light switching point prediction method, applied to an in-vehicle terminal, comprising:

[0005] Acquire the target intersection image data;

[0006] Image recognition is performed on the target intersection image data to obtain the target traffic light recognition result of the target traffic light in the target intersection;

[0007] The target traffic light identification result is sent to the server.

[0008] The server is used to calculate the switching point information of the target traffic light based on the target traffic light identification result and the traffic light switching pattern information of the target traffic light.

[0009] Secondly, embodiments of this disclosure provide a traffic light switching point prediction method, applied to a server, including:

[0010] Obtain the target traffic light recognition results of the target traffic lights at the target intersection sent by the vehicle terminal;

[0011] The switching point information of the target traffic light is calculated based on the target traffic light identification result and the traffic light switching pattern information of the target traffic light.

[0012] Thirdly, embodiments of this disclosure provide a traffic signal switching point prediction device, configured in an in-vehicle terminal, comprising:

[0013] The target intersection image data acquisition module is used to acquire the target intersection image data.

[0014] The target intersection image data recognition module is used to perform image recognition on the target intersection image data to obtain the target traffic light recognition result in the target intersection;

[0015] The target traffic light recognition result sending module is used to send the target traffic light recognition result to the server.

[0016] The server is used to calculate the switching point information of the target traffic light based on the target traffic light recognition result and the traffic light switching pattern information of the target traffic light.

[0017] Fourthly, embodiments of this disclosure provide a traffic signal switching point prediction device, configured on a server, comprising:

[0018] Obtain the target traffic light recognition results of the target traffic lights at the target intersection sent by the vehicle terminal;

[0019] The switching point information of the target traffic light is calculated based on the target traffic light identification result and the traffic light switching pattern information of the target traffic light.

[0020] Fifthly, embodiments of this disclosure provide an electronic device, including:

[0021] At least one processor; and

[0022] A memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the traffic light switching point prediction method provided in the first aspect embodiment.

[0024] Sixthly, embodiments of this disclosure provide an electronic device, including:

[0025] At least one processor; and

[0026] A memory communicatively connected to the at least one processor; wherein,

[0027] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the traffic light switching point prediction method provided in the second aspect embodiment.

[0028] In a seventh aspect, embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the traffic light switching point prediction method provided in the first or second aspect embodiments.

[0029] Eighthly, this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the traffic light switching point prediction method provided in the first or second aspect of the embodiments.

[0030] Ninthly, embodiments of this disclosure provide a vehicle, including a vehicle body, and further including electronic equipment and at least one image acquisition device disposed on the vehicle body;

[0031] The electronic device includes:

[0032] At least one processor; and

[0033] A memory communicatively connected to the at least one processor; wherein,

[0034] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the traffic light switching point prediction method described in the first aspect.

[0035] This embodiment of the disclosure acquires target intersection image data through a vehicle-mounted terminal, performs image recognition on the target intersection image data to obtain the target traffic light recognition result, and sends the target traffic light recognition result to the server. After receiving the target traffic light recognition result, the server calculates the switching point information of the target traffic light based on the target traffic light recognition result and the traffic light switching pattern information. This solves the problems of high time, value cost, and server computing power consumption in existing traffic light switching point prediction, which affect the accuracy and efficiency of prediction. It can reduce the time, value cost, and server computing power consumption of traffic light switching point prediction, and improve the accuracy and efficiency of traffic light switching point prediction.

[0036] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0037] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0038] Figure 1 This is a flowchart of a traffic light switching point prediction method provided in an embodiment of this disclosure;

[0039] Figure 2 This is a flowchart of a traffic light switching point prediction method provided in an embodiment of this disclosure;

[0040] Figure 3 This is a flowchart illustrating a traffic light switching point prediction method provided in an embodiment of this disclosure;

[0041] Figure 4 This is a flowchart of a traffic light switching point prediction method provided in an embodiment of this disclosure;

[0042] Figure 5 This is a flowchart of a traffic light switching point prediction method provided in an embodiment of this disclosure;

[0043] Figure 6 This is a structural diagram of a traffic signal switching point prediction device provided in an embodiment of this disclosure;

[0044] Figure 7 This is a structural diagram of a traffic signal switching point prediction device provided in an embodiment of this disclosure;

[0045] Figure 8 This is a schematic diagram of the structure of an electronic device used to implement the traffic signal switching point prediction method of the present disclosure. Detailed Implementation

[0046] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0047] During driving navigation, there is a more than 90% chance that users will not see the countdown timer for traffic lights when waiting at a traffic light. Furthermore, users are mostly unaware of the remaining time of the traffic light as they approach the intersection. This lack of information about the countdown timer at the intersection creates uncertainty and anxiety for the user, and the inability to determine whether to proceed quickly or slowly when approaching the intersection affects their psychological expectation.

[0048] By predicting the countdown timer information of traffic lights and sending this prediction to users, users gain complete knowledge of the countdown status of traffic lights, which helps improve overall road traffic efficiency and reduce traffic accidents. Furthermore, the predicted traffic light information can be used to accurately calculate the cost of passing through intersections, serving as an auxiliary feature to predict dynamic traffic conditions such as road congestion and congestion dissipation. Simultaneously, the predicted traffic light information has a positive impact on the user's driving experience, driving safety, and the surrounding environment. For example, knowing the remaining time of a red light while waiting prevents users from becoming distracted and starting late; knowing that the green light is about to end when crossing prevents users from braking suddenly or even rear-ending others; and knowing that the red light has a long time left allows users to turn off their engines while waiting at a red light, reducing air pollution and urban noise.

[0049] Currently, the main methods for predicting traffic light switching points are as follows:

[0050] (1) The platform cooperates with the traffic signal management department to obtain the traffic signal switching time and timing rules of certain intersections or areas through which the other party passes, and then predicts the switching time of the traffic signal at the corresponding intersection based on the official traffic signal switching time and timing rules.

[0051] (2) Obtain the trajectory generated by the user during driving and extract its features. Analyze the acceleration, deceleration and stopping / starting time information in the trajectory, and combine it with road attributes to mine the traffic light switching time.

[0052] (3) Identify traffic light switching points based on sensor information transmitted back from the user's mobile phone or vehicle equipment, as well as location density and traffic flow information.

[0053] (4) Using the image information transmitted back in real time by the user's vehicle camera and other equipment, image recognition and switching point calculation are performed on the server side to mine and infer the switching information of traffic lights.

[0054] However, all four methods have different problems in application:

[0055] Regarding scheme (1), when the platform and relevant departments cooperate, the number of intersections and lights that can be covered is limited, resulting in significant communication and information costs, and the ability to identify erroneous information is weak. Predicting traffic light switching information through scheme (1) largely depends on mutual assistance and cooperation between the two parties, has a long cooperation cycle, involves many dependent parties, and is prone to instability and uncertainty, and is also severely affected by policy.

[0056] Regarding scheme (2), user driving behaviors such as stopping and starting times are often subject to subjective factors, which may result in significant errors compared to traffic light switching times. Furthermore, user trajectories contain a large number of dirty trajectories, severely limiting prediction accuracy due to the density of trajectory points. Accurate prediction is impossible at intersections with sparse trajectories, and modeling slow-moving and congested scenarios is extremely challenging at intersections with particularly dense trajectories. Therefore, this scheme has many inherent limitations.

[0057] Regarding scheme (3), the uncertainty of the user's in-vehicle mobile phone posture will generate a lot of noise in the sensor data. The accuracy and recall of the motion detection method on the client side cannot be achieved with second-level precision. The user's location traffic also contains a lot of low-precision data such as network positioning, resulting in a low accuracy of traffic light prediction. At the same time, traffic light switching is only one of the factors affecting the migration of crowd density in front of the intersection. Therefore, this scheme also has serious limitations.

[0058] Regarding scheme (4), the image information data returned by the user's vehicle camera and other devices is large, which will cause a large network transmission load and traffic cost to the acquisition equipment. At the same time, due to the amount of data, the network transmission and server calculation will also have a large delay, affecting the real-time performance of the calculation. Moreover, the above-mentioned drawbacks may affect the accuracy of traffic light prediction. Therefore, this scheme also has significant limitations.

[0059] In one example Figure 1 This is a flowchart of a traffic light switching point prediction method provided in this embodiment. This embodiment is applicable to situations where an in-vehicle terminal collects and identifies intersection images, and sends the identification results to a server, allowing the server to directly use the identification results to predict traffic light switching points. This method can be executed by a traffic light switching point prediction device, which can be implemented in software and / or hardware, and is generally integrated into the in-vehicle terminal for use in conjunction with the server. Accordingly, as... Figure 1 As shown, the method includes the following operations:

[0060] S110. Obtain the target intersection image data.

[0061] The target intersection can be the intersection that the vehicle is about to reach. The type of target intersection can be cross-shaped, T-shaped, etc., as long as it has traffic lights; this embodiment does not limit the type of target intersection. The target intersection image data can be image data obtained by taking pictures of the target intersection by the vehicle.

[0062] The acquisition of target intersection image data can be achieved using vehicle-mounted image acquisition devices, such as vehicle-mounted cameras, vehicle-mounted mobile phones, or dashcams, as long as they can acquire target intersection image data. This disclosure does not limit the method by which vehicles acquire target intersection image data.

[0063] It is understandable that different vehicles with camera functions can activate their own onboard image acquisition devices to collect images of the target intersection as target intersection image data when they pass through the target intersection.

[0064] S120. Perform image recognition on the target intersection image data to obtain the target traffic light recognition result of the target traffic light in the target intersection.

[0065] The target traffic light can be all traffic lights involved in the target intersection, or it can be a subset of the traffic lights included in the target intersection, such as the traffic light suitable for the lane where the vehicle is located. This embodiment does not limit the type of target traffic light. The target traffic light recognition result is data such as the current status information of the target traffic light obtained by the vehicle through image recognition, including but not limited to the light status, current duration, and location of the target traffic light. The target traffic light recognition result can be sent in the form of an array, rather than image data, to reduce network transmission load and traffic costs, as well as server-side computation latency.

[0066] Correspondingly, after the vehicle acquires the target intersection image data through the vehicle-mounted image acquisition device, it can call the vehicle's own image recognition function. For example, it can call the vehicle's image recognition terminal model to perform real-time image recognition on the target intersection image data on the vehicle side, and obtain the target traffic light recognition result of the target intersection.

[0067] Understandably, the models or algorithms used by vehicle-mounted terminals to identify target intersection image data need to be trained in advance using samples to ensure high recognition accuracy and efficiency.

[0068] S130. The target traffic light identification result is sent to the server, wherein the server is used to calculate the switching point information of the target traffic light based on the target traffic light identification result and the traffic light switching pattern information of the target traffic light.

[0069] The information regarding traffic light switching patterns can include, for example, the switching cycle of the traffic lights. Switching point information can be the timing of the switching between traffic lights, such as the time when green turns red, red turns yellow, and yellow turns green. To ensure driving safety, the switching information can optionally be summarized into two types: red-to-green and green-to-red. The duration of green-to-red switching can include the duration of green-to-yellow and yellow-to-red switching. Correspondingly, the switching point information can include the switching times of red-to-green and green-to-red switching.

[0070] Understandably, since vehicles can identify only a small amount of target intersection image data, real-time and accurate image recognition of target intersections can be achieved. Simultaneously, different vehicles with camera capabilities can activate their onboard image acquisition devices to capture and identify images of the target intersection as they pass through, obtaining highly real-time and accurate target traffic light recognition results. Therefore, the server can receive these highly accurate and real-time target traffic light recognition results from different vehicles, and then combine them with the target traffic light switching pattern information to calculate the target traffic light switching point information.

[0071] For example, suppose vehicle A sends the following target traffic light identification result: the current light state of light a at intersection A is green, and the current countdown information is 10 seconds. Based on the traffic light switching pattern information of light a: the green light duration is 50 seconds, the red light duration is 60 seconds (including the yellow light duration), the server can determine that light a will switch to red in 10 seconds, switch to green in 70 seconds, and so on, and can calculate the switching point information of light a within a future period of time.

[0072] Therefore, by using an onboard terminal to acquire image data of the target intersection for image recognition, obtaining the target traffic light identification result, and sending the target traffic light identification result to the server, the server can predict the traffic light switching point information based on the target traffic light identification result. This not only avoids the surge in network transmission load and traffic costs caused by the onboard terminal transmitting large amounts of image data, but also reduces the server's computation latency due to the real-time data transmission. Furthermore, the server receives a highly accurate and real-time target traffic light identification result, which can be directly used to calculate the traffic light switching point information without further image recognition processing. Therefore, the accuracy and efficiency of the server's calculation of traffic light switching point information are significantly improved. In addition, compared to trajectory data processing, traffic light image data can directly reflect the characteristics of traffic lights. Because image features are easier to extract, the information reflected by the traffic lights is more accurate. Moreover, the above processing does not require the intervention of any additional third party, reducing the dependence on traffic light information acquisition and resulting in higher information processing efficiency.

[0073] This embodiment of the disclosure acquires target intersection image data through a vehicle-mounted terminal, performs image recognition on the target intersection image data to obtain the target traffic light recognition result, and sends the target traffic light recognition result to the server. After receiving the target traffic light recognition result, the server calculates the switching point information of the target traffic light based on the target traffic light recognition result and the traffic light switching pattern information. This solves the problems of high time, value cost, and server computing power consumption in existing traffic light switching point prediction, which affect the accuracy and efficiency of prediction. It can reduce the time, value cost, and server computing power consumption of traffic light switching point prediction, and improve the accuracy and efficiency of traffic light switching point prediction.

[0074] In one example Figure 2 This is a flowchart of a traffic light switching point prediction method provided in an embodiment of this disclosure. Figure 3 This is a flowchart illustrating a traffic light switching point prediction method provided in this embodiment. Based on the technical solutions of the above embodiments, this embodiment has been optimized and improved, and provides various specific optional implementation methods for acquiring target intersection image data, performing image recognition on the target intersection image data, and sending the target traffic light recognition results to the server.

[0075] like Figure 2 A traffic signal switching point prediction method is shown, comprising:

[0076] S210. Obtain the current real-time geographical location of the current vehicle.

[0077] The current vehicle is also the target intersection image data that will be acquired soon.

[0078] S220. Determine whether the distance between the current real-time geographical location and the target geographical location of the target intersection is less than or equal to a preset distance threshold. If yes, execute S230; otherwise, return to execute S210.

[0079] The preset distance threshold is generated based on the intersection profile features of the target intersection.

[0080] The target geographic location can be a reference position for determining whether the current vehicle has entered the target intersection area. For example, the target geographic location can be determined based on the center point of the target intersection. The preset distance threshold can be set according to actual needs, such as 50 meters, and this embodiment does not limit the specific value of the preset distance threshold.

[0081] Understandably, the appropriateness of the preset distance threshold setting affects the clarity and usability of the acquired image data. For example, if the preset distance threshold is too high, the distance between the vehicle and the target intersection may be too great, making it impossible to capture a clear and usable image of the target traffic light. If the preset distance threshold is too low, vehicles ahead may obstruct the view, preventing the acquisition of an image that includes the target traffic light.

[0082] To this end, the intersection profile of each target intersection can be analyzed to extract the profile features of different target intersections. Based on these profile features, a preset distance threshold can be configured for each target intersection. That is, the preset distance thresholds configured for different target intersections can be different to ensure that clear and usable target intersection image data can be obtained before vehicles reach the target intersection.

[0083] S230. Obtain the intersection queue shape point data of the target intersection.

[0084] The intersection queue shape point data may include, but is not limited to, the location deployment of traffic lights at the target intersection and the visual distance information between the target intersection and the current vehicle.

[0085] S240. Based on the intersection queue shape point data of the target intersection, take a picture of the target intersection to obtain the image data of the target intersection.

[0086] For example, the vehicle can obtain real-time GPS signals through hardware devices such as a GPS (Global Positioning System) chip, and then use positioning software to process the signals to obtain positioning data to determine its current real-time geographical location. Simultaneously, the server can determine the preset range area where the vehicle has entered the target intersection based on the current real-time geographical location, and then send the intersection queue shape point data to the vehicle for display. After obtaining the intersection queue shape point data, the vehicle can activate its camera function to take a picture of the target intersection.

[0087] The above technical solution, by utilizing preset distance thresholds configured individually for different intersections, sets the optimal shooting distance between the vehicle and the target intersection, thereby improving the clarity and usability of the target intersection image data acquired by the vehicle. Simultaneously, by using the intersection queue shape point data as a basis for shooting the target intersection, the success rate of acquiring target intersection image data can be improved.

[0088] In an optional embodiment of this disclosure, the step of photographing the target intersection based on the intersection queue shape point data may include: determining the type of traffic lights included in the target intersection based on the intersection queue shape point data; and photographing the target intersection if the type of traffic lights included in the target intersection is determined to be a target hot traffic light.

[0089] Among them, the target popularity traffic lights can be traffic lights that have been pre-selected by the server and require prediction of switching point information.

[0090] Understandably, some traffic lights, due to low traffic volume or other objective factors, do not need to display switching point information in real time, thus further reducing the computational load on the server. Therefore, once the intersection queue shape point data for the target intersection is available, the types of traffic lights included in the target intersection can be determined based on this data. If the traffic light type included in the target intersection is determined to be a high-traffic traffic light, then the target intersection can be photographed. If the traffic light type included in the target intersection is determined to be a low-traffic traffic light, then photographing the target intersection is prohibited. Alternatively, the server can also directly identify the target intersection. Correspondingly, once the intersection queue shape point data for the target intersection is available, the type of the target intersection can be determined based on this data. If the type of the target intersection is determined to be a high-traffic intersection requiring prediction of traffic light switching point information, then the target intersection can be photographed. If the type of the target intersection is determined to be a non-high-traffic intersection requiring prediction of traffic light switching point information, then photographing the target intersection is prohibited.

[0091] The above technical solution, by screening key traffic lights to obtain target traffic lights with high traffic volume and instructing vehicles to acquire image data of the target traffic lights with high traffic volume, can achieve the prediction of valuable traffic light switching point information, so as to cover as many users as possible with fewer lights, and to ensure the stability and accuracy of the predicted switching points by using a set of lights with certain specific patterns.

[0092] S250: Obtain the current lane-level positioning information of the current vehicle.

[0093] Among them, the current lane-level positioning information can represent the lane information where the vehicle is currently located.

[0094] S260. Based on the current lane-level positioning information, the target intersection image data is located and identified to obtain the identification result of the lane-level traffic light that matches the current lane where the current vehicle is located, which is then used as the target traffic light identification result.

[0095] Among them, lane-level traffic lights can be traffic lights that match the current lane, that is, traffic lights that can guide vehicles in the current lane.

[0096] In this embodiment, the vehicle can locate and identify the target intersection image data based on its current lane-level positioning information. Location and identification can be understood as acquiring only the image data of the traffic light corresponding to each lane, in a scenario where each lane corresponds to one traffic light. If the vehicle acquires video-type target intersection image data, before performing image recognition, the vehicle can also perform viewpoint correction and frame extraction on the video, and then perform location and identification on the extracted frames to obtain the identification result of the lane-level traffic light matching the current lane of the vehicle as the target traffic light identification result.

[0097] Thus, when a target intersection includes vehicles in different lanes guided by multiple different traffic lights, vehicles equipped with image acquisition and edge recognition can locate and identify the target intersection image data to obtain the recognition result of the target traffic light in their lane. That is, a vehicle can upload only the recognition result of the traffic light corresponding to its lane to the server. The advantage of this setup is that vehicles can accurately report real-time lane-level traffic light recognition results to the server, allowing the server to quickly calculate the predicted switching point of the traffic light corresponding to the lane. Simultaneously, it facilitates the server in locating lane-level trajectory data based on the real-time lane-level traffic light recognition results to verify the accuracy of the real-time recognition results.

[0098] It can be understood that if multiple lanes share traffic lights, then it is not necessary to locate and identify the target intersection image data. Instead, the complete target intersection image data can be identified directly, and the complete identification result of the target intersection image data can be sent to the server.

[0099] In an optional embodiment of this disclosure, the step of locating and identifying the target intersection image data based on the current lane-level positioning information may include: extracting features from the target intersection image data to obtain signal light extraction features of the target traffic light; wherein, the signal light extraction features include the position of the target traffic light, the type of the target traffic light, the light status of the target traffic light, and the timing information of the target traffic light; classifying and identifying the target traffic light based on the signal light extraction features to obtain a target traffic light classification and identification result; if the number of target traffic lights is determined to be multiple based on the target traffic light classification and identification result, determining the lane-level traffic light that matches the current lane of the current vehicle from each of the target traffic lights based on the current lane-level positioning information; and filtering the identification results that match the lane-level traffic light from the target traffic light classification and identification results to obtain the target traffic light identification result.

[0100] The traffic light extraction features can be features obtained by extracting features from the target intersection image data. The target traffic light classification and recognition result is the specific type of the target traffic light, such as a straight-ahead red light. The current lane-level positioning information can be information that locates the current lane, such as lane markings or the lane's relative position on the road.

[0101] Specifically, the vehicle-mounted terminal can use an end-to-end recognition model to extract features from the target intersection image data, obtaining signal light extraction features including the location, type, status, and timing information of the target traffic lights. Further, the end-to-end recognition model uses these extracted features to classify and identify the target traffic lights, obtaining classification results. Simultaneously, the vehicle-mounted terminal can detect the presence of multiple target traffic lights at the intersection based on the classification results. If multiple target traffic lights are identified, to further improve the real-time prediction of traffic light switching points, the vehicle-mounted terminal can determine the lane-level traffic light matching the current lane of the vehicle based on the current lane-level positioning information, and then select the matching lane-level traffic light from the classification results as the final target traffic light identification result.

[0102] For example, the edge recognition model can consist of a subject detection model and a classification model. The subject detection model can be used to locate the position of the subject (i.e., the traffic light), and could be, for example, YOLOv5 (YouOnly Look Once version 5, a subject detection model). The classification model can be used to identify the type of traffic light, and could be, for example, MobileNetv3 (a lightweight mobile terminal neural network). If the target intersection image data is a continuous video, metric learning can be used to extract frames from the captured video and further extract features from each frame. For example, the type and state of the traffic light signal ultimately identified by the edge recognition model can include, but are not limited to, the following: circular lights (red, yellow, green); pedestrian lights (red, yellow, green); straight arrow lights (red, yellow, green); left turn lights (red, yellow, green); right turn lights (red, yellow, green); and U-turn lights (red, yellow, green), etc.

[0103] Understandably, the edge recognition model needs to be pre-trained before it can be applied to vehicle terminals. During the training process, data from key intersection image sets can be used to fine-tune the model parameters to improve the accuracy of the lightweight image recognition model on the edge in key road scenes and ensure recognition performance.

[0104] S270. Determine if the target traffic light recognition result is empty. If yes, execute S280; otherwise, execute S290.

[0105] S280. Delete the target traffic light identification result.

[0106] It is understandable that, while a vehicle is approaching a target intersection, it may be unable to acquire image data of the target intersection, including the target traffic lights, at any given time. For example, if the vehicle is obstructed by other vehicles, it may be unable to successfully acquire image data of the target intersection, including the target traffic lights. Therefore, the onboard terminal can make a judgment on the target traffic light recognition result. If it is determined that the target traffic light recognition result is empty, it can be deleted to avoid sending invalid data to the server and wasting resources.

[0107] S290. Generate multi-dimensional vector data of the target traffic light based on the target traffic light recognition result, and send the multi-dimensional vector data of the target traffic light to the server.

[0108] The multidimensional vector data of the target traffic light can be, for example, various data types that can reflect the associated attributes of the target traffic light, such as the target traffic light's location, type, light status, and current timing duration.

[0109] Correspondingly, if the vehicle terminal determines that the target traffic light recognition result is not empty, it can process or filter the target traffic light recognition result to generate the final required multi-dimensional vector data of the target traffic light and feed it back to the server.

[0110] The aforementioned technical solution utilizes the user's GPS trajectory information and offline-mined intersection shape point set information through the on-board terminal's end-recognition model to perform real-time calculations and scheduling of traffic lights the user is currently passing. It also processes image data acquired at the target intersection, performing image perspective correction, video frame extraction, and single-image road element recognition (including traffic light type and status). The identified road element information is then transmitted back to the server in real-time. The server receives the traffic light change information as the vehicle passes the target intersection and uses this information to predict the target traffic light switching point. This reduces the time and cost of traffic light switching point prediction, as well as the server's computing power consumption, thereby improving the accuracy and efficiency of traffic light switching point prediction.

[0111] In one example Figure 4 This is a flowchart of a traffic light switching point prediction method provided in this embodiment. This embodiment is applicable to situations where the server directly uses the intersection recognition results from the vehicle-mounted terminal to predict traffic light switching points. This method can be executed by a traffic light switching point prediction device, which can be implemented in software and / or hardware, and is generally integrated into the server for use with the vehicle-mounted terminal. Correspondingly, as... Figure 1 As shown, the method includes the following operations:

[0112] S410: Obtain the target traffic light identification result of the target traffic light in the target intersection sent by the vehicle terminal.

[0113] S420. Calculate the switching point information of the target traffic light based on the target traffic light identification result and the traffic light switching pattern information of the target traffic light.

[0114] Optionally, before calculating the switching point information of the target traffic light, the server can use all the target traffic light recognition results for prediction, or it can filter the target traffic light recognition results, such as filtering the target traffic light recognition results related to red-to-green switching, and use the filtered target traffic light recognition results in combination with the traffic light switching pattern information of the target traffic light to calculate the switching point information of the target traffic light, thereby further improving the accuracy and efficiency of the prediction of the switching point information of the target traffic light.

[0115] This embodiment of the disclosure acquires target intersection image data through a vehicle-mounted terminal, performs image recognition on the target intersection image data to obtain the target traffic light recognition result, and sends the target traffic light recognition result to the server. After receiving the target traffic light recognition result, the server calculates the switching point information of the target traffic light based on the target traffic light recognition result and the traffic light switching pattern information. This solves the problems of high time, value cost, and server computing power consumption in existing traffic light switching point prediction, which affect the accuracy and efficiency of prediction. It can reduce the time, value cost, and server computing power consumption of traffic light switching point prediction, and improve the accuracy and efficiency of traffic light switching point prediction.

[0116] In one example Figure 5 This is a flowchart of a traffic signal switching point prediction method provided in this embodiment. Based on the technical solutions of the above embodiments, this embodiment has been optimized and improved, and provides various specific optional implementation methods for the server to verify the target traffic signal identification results, generate traffic signal switching pattern information and target hot intersection set, and send switching point information.

[0117] like Figure 5 A traffic signal switching point prediction method is shown, comprising:

[0118] S510: Obtain the target traffic light identification result of the target traffic light in the target intersection sent by the vehicle terminal.

[0119] In an optional embodiment of this disclosure, before obtaining the target traffic light identification result of the target traffic light at the target intersection sent by the vehicle terminal, the method may further include: obtaining full vehicle trajectory data and traffic light image data; generating a target popularity traffic light set based on the full vehicle trajectory data and the traffic light image data; generating a target popularity intersection set based on the target popularity traffic light set; wherein the target popularity intersection set includes the target intersection.

[0120] Among these, the full vehicle trajectory data can be all vehicle trajectory data that the server can statistically collect and analyze. Traffic light image data can be offline images of traffic lights. The target popularity traffic light set is a collection of traffic lights with the target popularity metric. The target popularity intersection set is a combination of intersections with the target popularity metric. The target popularity intersection set is a collection of intersections with the target popularity metric.

[0121] In this embodiment, the server can offline determine the target hot traffic light set and the target hot intersection set. Specifically, the server can utilize massive amounts of vehicle trajectory data and offline-scheduled traffic light image data to identify valuable traffic lights as target hot traffic lights and organize them into a target hot traffic light set. Simultaneously, based on the intersections where the target hot traffic lights are located, the server determines the target hot intersections and organizes them into a target hot intersection set.

[0122] By organizing and determining the target high-traffic traffic light set and the target high-traffic intersection set, the image acquisition equipment and terminal recognition function can be triggered in time before the vehicle reaches the target high-traffic intersection to collect image data of the target high-traffic traffic lights and identify them, thereby improving the real-time performance of image data acquisition and recognition of the vehicle terminal.

[0123] S520. Obtain the data return time matching the target traffic light recognition result.

[0124] The data transmission time can be the time it takes for the vehicle terminal to send the target traffic light recognition result back to the server.

[0125] Optionally, the vehicle terminal can mark the generated target traffic light recognition result with a timestamp. To ensure the accuracy of the prediction, the timestamp can represent the time when the target intersection image data was acquired.

[0126] S530. Obtain vehicle trajectory data of the target intersection based on the data return time.

[0127] S540. Verify the data transmission time based on the vehicle trajectory data of the target intersection.

[0128] Considering that a significant delay in the transmission of target traffic light recognition results could affect the accuracy of switching point information prediction, the server needs to verify the validity of the received target traffic light recognition results. During the verification process, the server first obtains the data transmission time matching the target traffic light recognition result. Based on this data transmission time, it retrieves the vehicle trajectory data of the target intersection at that time point and then verifies the data transmission time against the retrieved vehicle trajectory data of the target intersection at that time point, thereby ensuring the availability and reliability of the target traffic light recognition results.

[0129] It should be noted that if the recognition results returned by the vehicle-mounted terminal include the time information of the recognition result generation, considering that there is a certain delay in the output of the recognition results by the vehicle-mounted terminal, the vehicle-mounted terminal can record the time point of the captured image and send the captured image time point to the server at the same time. The server can calibrate the timing information in the recognition results based on the captured image time point to accurately determine the actual timing of the current traffic light, and then predict the future traffic light switching point based on the calibrated time and traffic light switching pattern information.

[0130] In one optional embodiment of this disclosure, the step of verifying the data transmission time based on the vehicle trajectory data of the target intersection may include: filtering the vehicle trajectory data of the target intersection to obtain target vehicle trajectory data matching the target traffic light recognition result; determining the vehicle driving status of the reference vehicle based on the target vehicle trajectory data; and verifying the data transmission time based on the vehicle driving status of the reference vehicle.

[0131] The reference vehicle can be a vehicle whose trajectory can be verified by the data transmission time.

[0132] Optionally, the vehicle trajectory data at the target intersection can be filtered. This could involve filtering the vehicle trajectory data of the lanes corresponding to the target traffic lights that can guide vehicles at the data feedback time. In other words, the target vehicle trajectory data can reflect the specific driving status of the target traffic lights at the data feedback time. Correspondingly, the server can determine the driving status of reference vehicles based on the target vehicle trajectory data. For example, if the target vehicle trajectory changes in real time at the data feedback time, the driving status of the corresponding reference vehicle can be determined to be "passing". If the target vehicle trajectory does not change within a certain time range corresponding to the data feedback time, the driving status of the corresponding reference vehicle can be determined to be "parked". Furthermore, the server can verify the data feedback time based on the driving status of the reference vehicles.

[0133] For example, suppose the recognition result of traffic light a at intersection A is red (countdown display shows 20 seconds), and the data transmission time is 10:00 AM on October 10, 2022, indicating that the image data of intersection A was captured by the vehicle terminal at 10:00 AM on October 10, 2022. Correspondingly, the server can obtain the vehicle trajectory data of intersection A at 10:00 AM on October 10, 2022, and analyze the trajectory direction and vehicle driving status (starting and stopping) of reference vehicles at that time to determine whether the vehicle driving status of the reference vehicles at that time matches the traffic light status at 10:00 AM on October 10, 2022. If the vehicle driving status of the reference vehicles at 10:00 AM on October 10, 2022 is straight through intersection A, then the target traffic light recognition result match fails the verification, and the target traffic light recognition result match is unusable.

[0134] Therefore, it can be seen that by referencing the vehicle's driving status to verify the data transmission time, the accuracy and rationality of the data transmission time verification can be improved.

[0135] S550. Obtain the associated historical vehicle trajectory data of the target intersection.

[0136] Among them, the associated historical vehicle trajectory data can be the trajectory data that matches the target intersection from the offline historical vehicle trajectory data.

[0137] S560. Determine the historical trajectory movement pattern of vehicles at the target intersection based on the associated historical vehicle trajectory data.

[0138] Among them, the historical trajectory movement pattern of vehicles can be the movement pattern of each vehicle at the target intersection in historical moments.

[0139] For example, the historical vehicle trajectory movement patterns of a target intersection determined based on associated historical vehicle trajectory data could be as follows: From 10:00 AM to 10:01 AM on January 1, 2020, vehicles passed through intersection A in both the left-turn and straight-ahead directions; from 10:01 AM to 10:02 AM on January 1, 2020, no vehicles passed through intersection A in either the left-turn or straight-ahead directions; from 10:02 AM to 10:03 AM on January 1, 2020, vehicles passed through intersection A in both the left-turn and straight-ahead directions; from 10:03 AM to 10:04 AM on January 1, 2020, no vehicles passed through intersection A in either the left-turn or straight-ahead directions. This pattern can be repeated to obtain the historical vehicle trajectory movement patterns of the target intersection.

[0140] S570. Generate the traffic light switching pattern information based on the historical trajectory movement patterns of vehicles at the target intersection.

[0141] Correspondingly, the historical vehicle trajectory patterns at the target intersection can effectively reflect the traffic light switching patterns at that intersection. Therefore, traffic light switching pattern information matching the target intersection can be generated based on the historical vehicle trajectory patterns at the target intersection.

[0142] Continuing with the example above, based on the historical vehicle trajectory patterns of the target intersection, the preliminary traffic light switching patterns can be generated as follows: From 10:00 AM to 10:01 AM on January 1, 2020, the traffic lights corresponding to the left-turn and straight-ahead directions at intersection A are green; from 10:01 AM to 10:02 AM on January 1, 2020, the traffic lights corresponding to the left-turn and straight-ahead directions at intersection A are red; from 10:02 AM to 10:03 AM on January 1, 2020, the traffic lights corresponding to the left-turn and straight-ahead directions at intersection A are green; from 10:03 AM to 10:04 AM on January 1, 2020, the traffic lights corresponding to the left-turn and straight-ahead directions at intersection A are green. And so on, the traffic light switching patterns of the target intersection over a future period can be obtained.

[0143] By using historical vehicle trajectory data associated with the target intersection to extrapolate and calculate the traffic light switching patterns at the target intersection, the accuracy of the traffic light switching patterns can be improved, thereby ensuring the accuracy of the server's prediction of traffic light switching points.

[0144] It should be noted that, Figure 5 This is merely a schematic diagram of one implementation method; there is no specific order between steps S510-S540 and steps S550-S570. That is, steps S510-S540 can be executed first, followed by steps S550-S570. Alternatively, steps S550-S570 can be executed first, followed by steps S510-S540. Or, both can be executed in parallel.

[0145] S580. Calculate the switching point information of the target traffic light based on the target traffic light identification result and the traffic light switching pattern information of the target traffic light.

[0146] It is understandable that when the server calculates the switching point information of a target traffic light based on the target traffic light recognition results sent by different vehicles and the traffic light switching pattern information, the differences in the target traffic light recognition results may lead to different switching point information being calculated for the same target traffic light. To solve this problem, the server can determine the confidence level of each target traffic light recognition result and use a certain algorithm to comprehensively calculate the different switching point information calculated for the same target traffic light. This allows the server to calculate the target switching point information with higher accuracy and confidence based on the different switching point information calculated for the same target traffic light, which will then be used as the switching point information for that target traffic light.

[0147] S590. Determine the target vehicle that requests to obtain the switching point information of the target traffic signal.

[0148] S5A0: Send the switching point information of the target traffic light to the target vehicle for display.

[0149] In this embodiment of the disclosure, when the server predicts the switching point information of the target traffic light at the target intersection, it can send the switching point information of the target traffic light to the target vehicle requesting the switching point information in real time. For example, when the target vehicle's navigation route involves the target traffic light, the switching point information of the target traffic light can be displayed on the navigation side so that the user can grasp the real-time light status of each target traffic light in the route and use it as a reference for the user's driving.

[0150] The above technical solution utilizes vehicle-mounted terminals to acquire image information of intersections. After on-device image processing and recognition, the recognition results are transmitted back to the server. The server uses historical intersection information and real-time traffic light recognition information to predict future traffic light switching points. This significantly reduces network transmission time and costs while ensuring the accuracy of future traffic light switching points. Furthermore, disseminating traffic light switching point prediction information to users provides a more comfortable user experience and improves their overall evaluation of the map.

[0151] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information (such as user vehicle trajectory data) in this technical solution comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0152] It should be noted that any arrangement or combination of the technical features in the above embodiments also falls within the protection scope of this disclosure.

[0153] In one example Figure 6This is a structural diagram of a traffic signal switching point prediction device provided in an embodiment of this disclosure. This embodiment of the disclosure is applicable to situations where an in-vehicle terminal collects and identifies intersection images and sends the identification results to a server so that the server can directly use the identification results to predict traffic signal switching points. The device is implemented through software and / or hardware and is specifically configured in the in-vehicle terminal for use in conjunction with the server.

[0154] like Figure 6 A traffic signal switching point prediction device 600, as shown, includes: a target intersection image data acquisition module 610, a target intersection image data recognition module 620, and a target traffic signal recognition result transmission module 630. Among them,

[0155] The target intersection image data acquisition module 610 is used to acquire the target intersection image data.

[0156] The target intersection image data recognition module 620 is used to perform image recognition on the target intersection image data to obtain the target traffic light recognition result of the target traffic light in the target intersection;

[0157] The target traffic light recognition result sending module 630 is used to send the target traffic light recognition result to the server.

[0158] The server is used to calculate the switching point information of the target traffic light based on the target traffic light identification result and the traffic light switching pattern information of the target traffic light.

[0159] This embodiment of the disclosure acquires target intersection image data through a vehicle-mounted terminal, performs image recognition on the target intersection image data to obtain the target traffic light recognition result, and sends the target traffic light recognition result to the server. After receiving the target traffic light recognition result, the server calculates the switching point information of the target traffic light based on the target traffic light recognition result and the traffic light switching pattern information. This solves the problems of high time, value cost, and server computing power consumption in existing traffic light switching point prediction, which affect the accuracy and efficiency of prediction. It can reduce the time, value cost, and server computing power consumption of traffic light switching point prediction, and improve the accuracy and efficiency of traffic light switching point prediction.

[0160] Optionally, the target intersection image data acquisition module 610 is further configured to: acquire the current real-time geographical location of the current vehicle; if the distance between the current real-time geographical location and the target geographical location of the target intersection is less than or equal to a preset distance threshold, acquire the intersection queue shape point data of the target intersection; and capture images of the target intersection based on the intersection queue shape point data of the target intersection to acquire the target intersection image data; wherein the preset distance threshold is generated based on the intersection profile features of the target intersection.

[0161] Optionally, the target intersection image data acquisition module 610 is further configured to: determine the type of traffic lights included in the target intersection based on the intersection queue shape point data of the target intersection; and, if the type of traffic lights included in the target intersection is determined to be a target hot traffic light, take a picture of the target intersection.

[0162] Optionally, the target intersection image data recognition module 620 is further configured to: acquire the current lane-level positioning information of the current vehicle; perform positioning recognition on the target intersection image data according to the current lane-level positioning information, and obtain the recognition result of the lane-level traffic light that matches the current lane where the current vehicle is located as the target traffic light recognition result.

[0163] Optionally, the target intersection image data recognition module 620 is further configured to: extract features from the target intersection image data to obtain the signal light extraction features of the target traffic light; wherein, the signal light extraction features include the location of the target traffic light, the type of the target traffic light, the light status of the target traffic light, and the timing information of the target traffic light; classify and recognize the target traffic light according to the signal light extraction features of the target traffic light to obtain the target traffic light classification and recognition result; if the number of target traffic lights is determined to be multiple according to the target traffic light classification and recognition result, determine the lane-level traffic light that matches the current lane of the current vehicle from each of the target traffic lights according to the current lane-level positioning information; filter the recognition results of the lane-level traffic light matching from the target traffic light classification and recognition results to obtain the target traffic light recognition result.

[0164] Optionally, the target traffic light recognition result sending module 630 is further configured to: generate target traffic light multi-dimensional vector data based on the target traffic light recognition result when it is determined that the target traffic light recognition result is not empty; and send the target traffic light multi-dimensional vector data to the server; the device further includes a target traffic light recognition result deletion module, configured to: delete the target traffic light recognition result when it is determined that the target traffic light recognition result is empty.

[0165] The traffic light switching point prediction device described above can execute the traffic light switching point prediction method executed by the vehicle-mounted terminal provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the execution method. Technical details not described in detail in this embodiment can be found in the traffic light switching point prediction method executed by the vehicle-mounted terminal provided in any embodiment of this disclosure.

[0166] In one example Figure 7 This is a structural diagram of a traffic signal switching point prediction device provided in an embodiment of this disclosure. This embodiment of the disclosure is applicable to situations where the server directly uses the recognition results of the intersection by the vehicle terminal to predict the traffic signal switching point. The device is implemented by software and / or hardware and is specifically configured in the server terminal for use in conjunction with the vehicle terminal.

[0167] like Figure 7 A traffic signal switching point prediction device 700, as shown, includes: a target traffic signal recognition result acquisition module 710 and a switching point information calculation module 720. Among them,

[0168] The target traffic light recognition result acquisition module is used to acquire the target traffic light recognition result of the target traffic light in the target intersection sent by the vehicle terminal;

[0169] The switching point information calculation module is used to calculate the switching point information of the target traffic light based on the target traffic light identification result and the traffic light switching pattern information of the target traffic light.

[0170] This embodiment of the disclosure acquires target intersection image data through a vehicle-mounted terminal, performs image recognition on the target intersection image data to obtain the target traffic light recognition result, and sends the target traffic light recognition result to the server. After receiving the target traffic light recognition result, the server calculates the switching point information of the target traffic light based on the target traffic light recognition result and the traffic light switching pattern information. This solves the problems of high time, value cost, and server computing power consumption in existing traffic light switching point prediction, which affect the accuracy and efficiency of prediction. It can reduce the time, value cost, and server computing power consumption of traffic light switching point prediction, and improve the accuracy and efficiency of traffic light switching point prediction.

[0171] Optionally, the above device further includes a data return time verification module, used for: obtaining the data return time matching the target traffic light recognition result; obtaining vehicle trajectory data of the target intersection based on the data return time; and verifying the data return time based on the vehicle trajectory data of the target intersection.

[0172] Optionally, the data return time verification module is further configured to: filter the vehicle trajectory data of the target intersection to obtain target vehicle trajectory data matching the target traffic light recognition result; determine the vehicle driving status of the reference vehicle based on the target vehicle trajectory data; and verify the data return time based on the vehicle driving status of the reference vehicle.

[0173] Optionally, the above device further includes a traffic light switching pattern information generation module, used to: acquire associated historical vehicle trajectory data of the target intersection; determine the historical vehicle trajectory movement pattern of the target intersection based on the associated historical vehicle trajectory data; and generate the traffic light switching pattern information based on the historical vehicle trajectory movement pattern of the target intersection.

[0174] Optionally, the above-mentioned device further includes a heat set generation module, used for: acquiring full vehicle trajectory data and traffic light image data; generating a target heat traffic light set based on the full vehicle trajectory data and the traffic light image data; generating a target heat intersection set based on the target heat traffic light set; wherein the target heat intersection set includes the target intersection.

[0175] Optionally, the above device further includes a switching point information sending module, used to: determine a target vehicle requesting to obtain the switching point information of the target traffic light; and send the switching point information of the target traffic light to the target vehicle for display.

[0176] The traffic light switching point prediction device described above can execute the traffic light switching point prediction method executed by the server provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the execution method. Technical details not described in detail in this embodiment can be found in the traffic light switching point prediction method executed by the server provided in any embodiment of this disclosure.

[0177] In one example, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0178] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0179] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0180] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0181] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the traffic light switching point prediction method. For example, in some embodiments, the traffic light switching point prediction method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the traffic light switching point prediction method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform a traffic light switching point prediction method by any other suitable means (e.g., by means of firmware).

[0182] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0183] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0184] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0185] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0186] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0187] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, also known as cloud computing servers or cloud hosts, which are hosting products within the cloud computing service ecosystem to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers integrated with blockchain technology.

[0188] This embodiment of the disclosure acquires target intersection image data through a vehicle-mounted terminal, performs image recognition on the target intersection image data to obtain the target traffic light recognition result, and sends the target traffic light recognition result to the server. After receiving the target traffic light recognition result, the server calculates the switching point information of the target traffic light based on the target traffic light recognition result and the traffic light switching pattern information. This solves the problems of high time, value cost, and server computing power consumption in existing traffic light switching point prediction, which affect the accuracy and efficiency of prediction. It can reduce the time, value cost, and server computing power consumption of traffic light switching point prediction, and improve the accuracy and efficiency of traffic light switching point prediction.

[0189] In one example, based on the above embodiments, this disclosure also provides a vehicle, including a vehicle body, and further including the electronic equipment described in the above embodiments and at least one image acquisition device disposed on the vehicle body;

[0190] The electronic device includes:

[0191] At least one processor; and

[0192] A memory communicatively connected to the at least one processor; wherein,

[0193] The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to perform a traffic light switching point prediction method executed by the vehicle terminal.

[0194] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0195] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for predicting traffic signal switching points, applied to an on-board terminal, comprising: Acquire the target intersection image data; Image recognition is performed on the target intersection image data to obtain the target traffic light recognition result of the target traffic light in the target intersection; The target traffic light identification result is sent to the server. The server is used to calculate the switching point information of the target traffic light based on the target traffic light identification result and the traffic light switching pattern information of the target traffic light; The server is also used to: obtain the data return time matching the target traffic light recognition result; obtain vehicle trajectory data of the target intersection based on the data return time; and verify the data return time based on the vehicle trajectory data of the target intersection. The server is also used to: filter the vehicle trajectory data of the target intersection to obtain the target vehicle trajectory data that matches the target traffic light recognition result; determine the vehicle driving status of the reference vehicle based on the target vehicle trajectory data; and verify the data transmission time based on the vehicle driving status of the reference vehicle.

2. The method according to claim 1, wherein, The acquisition of the target intersection image data includes: Obtain the current real-time geographical location of the vehicle; If the distance between the current real-time geographic location and the target geographic location of the target intersection is less than or equal to a preset distance threshold, the intersection queue shape point data of the target intersection is obtained. The target intersection is photographed based on the intersection queue shape point data to obtain the target intersection image data; The preset distance threshold is generated based on the intersection profile features of the target intersection.

3. The method according to claim 2, wherein, The step of capturing images of the target intersection based on the intersection queue shape point data includes: The types of traffic lights included in the target intersection are determined based on the intersection queue shape point data of the target intersection; If the target intersection is determined to be a target-heat traffic light, then the target intersection is photographed.

4. The method according to claim 1, wherein, The step of performing image recognition on the target intersection image data to obtain the target traffic light recognition result in the target intersection includes: Obtain the current lane-level location information of the current vehicle; Based on the current lane-level positioning information, the target intersection image data is located and identified, and the identification result of the lane-level traffic light that matches the current lane where the current vehicle is located is obtained as the target traffic light identification result.

5. The method according to claim 4, wherein, The step of locating and identifying the target intersection image data based on the current lane-level positioning information includes: Feature extraction is performed on the target intersection image data to obtain the signal light extraction features of the target traffic light; wherein, the signal light extraction features include the location of the target traffic light, the type of the target traffic light, the light status of the target traffic light, and the timing information of the target traffic light; Based on the extracted features of the target traffic lights, the target traffic lights are classified and identified to obtain the classification and identification results of the target traffic lights. If the number of target traffic lights is determined to be multiple based on the target traffic light classification and identification results, the lane-level traffic light that matches the current lane of the current vehicle is determined from each of the target traffic lights based on the current lane-level positioning information; The target traffic light identification result is obtained by filtering the lane-level traffic light matching identification results from the target traffic light classification and identification results.

6. The method according to any one of claims 1-5, wherein, Sending the target traffic light identification result to the server includes: If the target traffic light identification result is determined to be non-empty, multi-dimensional vector data of the target traffic light is generated based on the target traffic light identification result. The multi-dimensional vector data of the target traffic light is sent to the server. The method further includes: If the target traffic light identification result is determined to be empty, the target traffic light identification result is deleted.

7. A method for predicting traffic light switching points, applied to a server, comprising: Obtain the target traffic light recognition results of the target traffic lights at the target intersection sent by the vehicle terminal; The switching point information of the target traffic light is calculated based on the target traffic light identification result and the traffic light switching pattern information of the target traffic light; The method further includes: The data transmission time matching the target traffic light identification result is obtained; The vehicle trajectory data of the target intersection is obtained based on the data return time. The data transmission time is verified based on the vehicle trajectory data at the target intersection; The step of verifying the data transmission time based on the vehicle trajectory data at the target intersection includes: The vehicle trajectory data of the target intersection is filtered to obtain the target vehicle trajectory data that matches the target traffic light recognition result; The driving status of the reference vehicle is determined based on the target vehicle trajectory data; The data transmission time is verified based on the vehicle driving status of the reference vehicle.

8. The method according to claim 7, further comprising: Obtain the associated historical vehicle trajectory data of the target intersection; The historical vehicle trajectory movement patterns at the target intersection are determined based on the associated historical vehicle trajectory data. The traffic light switching pattern information is generated based on the historical trajectory movement patterns of vehicles at the target intersection.

9. The method according to claim 7, further comprising: Acquire full vehicle trajectory data and traffic light image data; A target heat map set of traffic lights is generated based on the full vehicle trajectory data and the traffic light image data; Generate a target-heat intersection set based on the target-heat traffic light set; The target hot intersection set includes the target intersections.

10. The method according to any one of claims 7-9, further comprising: Identify the target vehicle requesting information on the switching point of the target traffic light; The switching point information of the target traffic light is sent to the target vehicle for display.

11. A traffic signal switching point prediction device, configured in an on-board terminal, comprising: The target intersection image data acquisition module is used to acquire the target intersection image data. The target intersection image data recognition module is used to perform image recognition on the target intersection image data to obtain the target traffic light recognition result in the target intersection; The target traffic light recognition result sending module is used to send the target traffic light recognition result to the server. The server is used to calculate the switching point information of the target traffic light based on the target traffic light identification result and the traffic light switching pattern information of the target traffic light; The server is also used to: obtain the data return time matching the target traffic light recognition result; obtain vehicle trajectory data of the target intersection based on the data return time; and verify the data return time based on the vehicle trajectory data of the target intersection. The server is also used to: filter the vehicle trajectory data of the target intersection to obtain the target vehicle trajectory data that matches the target traffic light recognition result; determine the vehicle driving status of the reference vehicle based on the target vehicle trajectory data; and verify the data transmission time based on the vehicle driving status of the reference vehicle.

12. The apparatus according to claim 11, wherein, The target intersection image data acquisition module is also used for: Obtain the current real-time geographical location of the vehicle; If the distance between the current real-time geographic location and the target geographic location of the target intersection is less than or equal to a preset distance threshold, the intersection queue shape point data of the target intersection is obtained. The target intersection is photographed based on the intersection queue shape point data to obtain the target intersection image data; The preset distance threshold is generated based on the intersection profile features of the target intersection.

13. The apparatus according to claim 12, wherein, The target intersection image data acquisition module is also used for: The types of traffic lights included in the target intersection are determined based on the intersection queue shape point data of the target intersection; If the target intersection is determined to be a target-heat traffic light, then the target intersection is photographed.

14. The apparatus according to claim 11, wherein, The target intersection image data recognition module is also used for: Obtain the current lane-level location information of the current vehicle; Based on the current lane-level positioning information, the target intersection image data is located and identified, and the identification result of the lane-level traffic light that matches the current lane where the current vehicle is located is obtained as the target traffic light identification result.

15. The apparatus according to claim 14, wherein, The target intersection image data recognition module is also used for: Feature extraction is performed on the target intersection image data to obtain the signal light extraction features of the target traffic light; wherein, the signal light extraction features include the location of the target traffic light, the type of the target traffic light, the light status of the target traffic light, and the timing information of the target traffic light; Based on the extracted features of the target traffic lights, the target traffic lights are classified and identified to obtain the classification and identification results of the target traffic lights. If the number of target traffic lights is determined to be multiple based on the target traffic light classification and identification results, the lane-level traffic light that matches the current lane of the current vehicle is determined from each of the target traffic lights based on the current lane-level positioning information; The target traffic light identification result is obtained by filtering the lane-level traffic light matching identification results from the target traffic light classification and identification results.

16. The apparatus according to any one of claims 11-15, wherein, The target traffic light identification result sending module is also used for: If the target traffic light identification result is determined to be non-empty, multi-dimensional vector data of the target traffic light is generated based on the target traffic light identification result. The multi-dimensional vector data of the target traffic light is sent to the server. The device further includes a target traffic light recognition result deletion module, used to delete the target traffic light recognition result when it is determined that the target traffic light recognition result is empty.

17. A traffic signal switching point prediction device, configured on a server, comprising: The target traffic light recognition result acquisition module is used to acquire the target traffic light recognition result of the target traffic light in the target intersection sent by the vehicle terminal; The switching point information calculation module is used to calculate the switching point information of the target traffic light based on the target traffic light identification result and the traffic light switching pattern information of the target traffic light; The device further includes a data return time verification module, used for: The data transmission time matching the target traffic light identification result is obtained; The vehicle trajectory data of the target intersection is obtained based on the data return time. The data transmission time is verified based on the vehicle trajectory data at the target intersection; The data return time verification module is also used for: The vehicle trajectory data of the target intersection is filtered to obtain the target vehicle trajectory data that matches the target traffic light recognition result; The driving status of the reference vehicle is determined based on the target vehicle trajectory data; The data transmission time is verified based on the vehicle driving status of the reference vehicle.

18. The apparatus according to claim 17, further comprising a traffic light switching pattern information generation module, used for: Obtain the associated historical vehicle trajectory data of the target intersection; The historical vehicle trajectory movement patterns at the target intersection are determined based on the associated historical vehicle trajectory data. The traffic light switching pattern information is generated based on the historical trajectory movement patterns of vehicles at the target intersection.

19. The apparatus of claim 17, further comprising a heat set generation module, used for: Acquire full vehicle trajectory data and traffic light image data; A target heat map set of traffic lights is generated based on the full vehicle trajectory data and the traffic light image data; Generate a target-heat intersection set based on the target-heat traffic light set; in, The target heat intersection set includes the target intersections.

20. The apparatus according to any one of claims 17-19, further comprising a switching point information sending module, used for: Identify the target vehicle requesting information on the switching point of the target traffic light; The switching point information of the target traffic light is sent to the target vehicle for display.

21. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the traffic light switching point prediction method according to any one of claims 1-6, or to perform the traffic light switching point prediction method according to any one of claims 7-10.

22. A non-transitory computer-readable storage medium storing computer instructions, said computer instructions being configured to cause a computer to perform the traffic signal switching point prediction method of any one of claims 1-6, or to perform the traffic signal switching point prediction method of any one of claims 7-10.

23. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the traffic light switching point prediction method according to any one of claims 1-6, or performs the traffic light switching point prediction method according to any one of claims 7-10.

24. A vehicle, comprising a vehicle body, and further comprising electronic equipment and at least one image acquisition device disposed on the vehicle body; in, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the traffic signal switching point prediction method according to any one of claims 1-6.

Images