Traffic condition prediction method and device and computer program product

Abstract

The embodiment of the invention discloses a traffic condition prediction method and device and a computer program product. According to the main technical scheme, the method comprises the steps of obtaining trajectory data of vehicles in a target area; performing visualization processing on the trajectory data to obtain visual data; the traffic condition of the target area can be obtained by further combining effective traffic information extracted from at least one of traffic intelligence data, user report data and environmental data based on visual data by utilizing a visual language model. The traffic condition comprises at least one of the congestion degree, the occurrence position of the traffic event, the occurrence reason and the congestion trend information. According to the mode, the comprehensive and accurate traffic condition of the target area can be obtained by utilizing the visual language model based on the visual data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular, to a traffic condition prediction method and device and computer program product. BACKGROUND

[0002] With the acceleration of urbanization, the contradiction between supply and demand of the transportation system is increasingly prominent, and more and more cities are trapped in "normal congestion", resulting in higher and higher time cost for users to travel. Users want to know the relevant traffic conditions in time before or during travel, so there is an urgent need for a way to accurately predict traffic conditions. SUMMARY

[0003] The present application provides a traffic condition prediction method, device and computer program product to obtain accurate traffic conditions.

[0004] The present application provides the following solutions:

[0005] According to a first aspect, a traffic condition prediction method is provided, the method comprising:

[0006] obtaining trajectory data of vehicles in a target area;

[0007] visualizing the trajectory data to obtain visualized data;

[0008] using a visual language model to obtain traffic conditions of the target area based on the visualized data, the traffic conditions comprising at least one of congestion degree, occurrence location of a traffic event, occurrence cause and congestion trend information.

[0009] According to a second aspect, a traffic condition prediction method is provided, the device comprising:

[0010] a data acquisition unit configured to obtain trajectory data of vehicles in a target area;

[0011] a data processing unit configured to visualize the trajectory data to obtain visualized data;

[0012] a model output unit configured to use a visual language model to obtain traffic conditions of the target area based on the visualized data, the traffic conditions comprising at least one of congestion degree, occurrence location of a traffic event, occurrence cause and congestion trend information.

[0013] According to a third aspect, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method of any one of the above first aspect.

[0014] According to the specific embodiments provided by the present application, the following technical effects are disclosed:

[0015] The application first visualizes the trajectory data of the vehicle, and then processes and analyzes the information based on the visual data by using a visual language model to obtain the traffic condition of the target area. In this way, the numerical information of the trajectory data is converted into visual semantic symbols understandable by the visual language model through the visual processing of the trajectory data, which upgrades the paradigm from "data understanding" to "scene understanding", fully utilizes the visual scene understanding capability of the visual language model, and thus improves the accuracy of traffic condition prediction.

[0016] Of course, implementing any product of the application does not necessarily require all the advantages described above to be achieved at the same time. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings in the following description only constitute some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0018] Figure 1 The system architecture diagram applicable to the embodiments of the application;

[0019] Figure 2 The flowchart of the traffic condition prediction method provided by the embodiments of the application;

[0020] Figure 3 The schematic diagram of the picture corresponding to a time frame in the video provided by the embodiments of the application;

[0021] Figure 4 The schematic diagram of a speed diagram provided by the embodiments of the application;

[0022] Figure 5 The schematic diagram of a density diagram provided by the embodiments of the application;

[0023] Figure 6 The process schematic diagram of training based on each training task provided by the embodiments of the application;

[0024] Figure 7 The schematic block diagram of the traffic condition prediction device provided by the embodiments of the application;

[0025] Figure 8 The schematic block diagram of the electronic device provided by the embodiments of the application. DETAILED DESCRIPTION

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0027] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0028] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0029] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0030] There are already some methods for determining traffic conditions, such as deploying dedicated equipment (e.g., cameras) at intersections to collect vehicle images and then identifying traffic conditions through image recognition. However, this method is difficult to cover a large area of ​​roads, has high deployment costs, and is easily affected by weather, lighting, and other factors, so its accuracy still needs to be improved.

[0031] Figure 1 The system architecture diagram applicable to the embodiments of this application is as follows: Figure 1 As shown, the system architecture may include: user equipment and a traffic condition prediction device located on the server side.

[0032] The user equipment and the server can communicate with each other. The user equipment and the server can be connected directly or indirectly through wired or wireless communication, which is not limited herein.

[0033] User devices can include, but are not limited to, smart mobile terminals, smart home devices, wearable devices, and PCs (Personal Computers). Smart mobile devices can include mobile phones, tablets, laptops, PDAs (Personal Digital Assistants), and connected cars. Smart home devices can include smart TVs, smart refrigerators, and so on. Wearable devices can include smartwatches, smart glasses, virtual reality devices, augmented reality devices, and mixed reality devices (devices that support both virtual and augmented reality).

[0034] A server can be a standalone server, a server cluster, or a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a hosting product within the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPS) services, such as high management difficulty and weak service scalability.

[0035] Users can send requests through their user devices. Specifically, the user device sends the request via the network to a traffic condition prediction device on the server side. The traffic condition prediction device uses a visual language model and visualized data to obtain the traffic conditions of the target area. In response to the user's request, it returns the traffic conditions to the user device via the network. The user device then displays the received traffic conditions to the user. Alternatively, the server can proactively push traffic conditions to the user device. Or, the server can perform downstream tasks based on the predicted traffic conditions, such as determining recommended routes, recommended departure times, and generating risk warning information, and then send the results of these downstream tasks to the user device.

[0036] Apart from Figure 1 In addition to the architecture shown, the traffic conditions of the target area can also be obtained by a computer terminal device with strong computing power using the method provided in the embodiments of this application.

[0037] It should be understood that Figure 1 The number of user equipment and traffic prediction devices shown is merely illustrative. Any number of user equipment and traffic prediction devices can be included depending on implementation needs.

[0038] In view of this, this application provides a new approach to traffic condition prediction. Figure 2 This is a flowchart of a traffic condition prediction method provided in an embodiment of this application. The method can be... Figure 1 The traffic condition prediction device in the system shown is executed. For example... Figure 2 As shown, the method may include the following steps:

[0039] Step 201: Obtain the trajectory data of vehicles within the target area.

[0040] Step 203: Perform visualization processing on the trajectory data to obtain visualized data.

[0041] Step 205: Using a visual language model, obtain the traffic conditions of the target area based on the visualization data. The traffic conditions include at least one of the following: congestion level, location of traffic incidents, cause of incidents, and congestion trend information.

[0042] As can be seen from the above process, this application first visualizes the vehicle trajectory data, and then, based on the visualized data, uses a visual language model to process and analyze the information to obtain the traffic conditions of the target area. This approach, by visualizing the trajectory data, transforms the numerical information of the trajectory data into visual semantic symbols that the visual language model can understand, representing a paradigm shift from "data understanding" to "scene understanding." It fully utilizes the visual scene understanding capabilities of the visual language model, thereby improving the accuracy of traffic condition prediction.

[0043] The following describes in detail each step of the above process and the effects that can be further produced, with reference to the embodiments.

[0044] First, the above step 201, namely "acquiring the trajectory data of vehicles within the target area", will be described in detail with reference to the embodiments.

[0045] In this embodiment, vehicle trajectory data within a target area is acquired. The target area refers to a region on the road where traffic conditions need to be monitored. The target area can be a pre-defined region, such as selecting areas with a significant impact on the overall road network traffic based on their influence, or selecting areas prone to traffic incidents based on historical data. Alternatively, the target area can be selected based on traffic flow. For example, traffic flow analysis can be performed on various regions, and areas with traffic flow exceeding a threshold can be selected as target areas.

[0046] Trajectory data is a collection of spatial location information recording the changes of moving objects (such as cars, two-wheeled vehicles, pedestrians, etc.) over time. It is widely used in traffic management, urban planning, logistics optimization, environmental monitoring, and other fields. In this embodiment, the vehicles involved mainly include automobiles, but in some scenarios, they may also include two-wheeled vehicles. Trajectory data consists of a series of spatial coordinate points arranged in chronological order. Each coordinate point (i.e., trajectory point) contains location information (such as latitude and longitude, address) and a timestamp.

[0047] The aforementioned trajectory data can be obtained by the positioning device on the vehicle reporting location information, or by the navigation data used by the user terminal on the vehicle, or by the target tracking and image fusion after the vehicle data is collected by cameras at roadside or intersection locations.

[0048] The following describes step 203, namely "performing visualization processing on trajectory data to obtain visualized data", in detail with reference to the embodiments.

[0049] After acquiring the trajectory data of vehicles within the target area, the trajectory data is visualized to obtain visualized data. Specifically, trajectory data over a period of time can be displayed on a map base map (including ordinary maps, lane-level maps, satellite images, etc.) to obtain video or images, which facilitates the subsequent use of visual language models to obtain the traffic conditions of the target area.

[0050] When the visualization data is video, the video can represent the movement trajectory of vehicles within the target area. This application can obtain the position of vehicles within the target area in each time frame based on the trajectory data, and a continuous movement trajectory can be generated based on the position of vehicles within the target area in each time frame. Figure 3 As shown, Figure 3 This shows the image corresponding to a specific time frame in the video, at which point the target area is... Figure 3 The intersection shown Figure 3 The red or yellow elements in the image indicate the location of the vehicle within the target area in that time frame.

[0051] When the visualization data is an image, the image may include attribute feature maps of vehicles within the target area, including at least one of speed maps, acceleration maps, density maps, traffic flow maps, road condition maps, and emergency braking heat maps.

[0052] The speed map uses different colors to represent the speed distribution of each vehicle within the target area, such as... Figure 4 As shown, Figure 4 In the diagram, red indicates lower vehicle speeds, and blue indicates higher vehicle speeds. This application can calculate the speed between two adjacent trajectory points based on the trajectory data of vehicles within a target area (i.e., obtain the speed between two adjacent trajectory points based on their location information and timestamps), and generate a speed map based on the speeds between multiple adjacent trajectory points.

[0053] The acceleration map uses different colors to represent the acceleration distribution of each vehicle in the target area. This application can calculate the acceleration between two adjacent trajectory points based on the trajectory data of vehicles in the target area (that is, obtain the acceleration between two adjacent trajectory points based on the position information and timestamp of two adjacent trajectory points), and generate an acceleration map based on the acceleration between multiple adjacent trajectory points.

[0054] Density maps use different colors to represent the number of vehicles per unit area, reflecting the degree of vehicle clustering. For example... Figure 5 As shown, Figure 5 In this application, green represents low density and red represents high density. The target area can be divided into fixed-size grids, the number of vehicles in each grid can be counted, and the grids can be colored according to the number of vehicles.

[0055] The emergency braking heatmap uses different colors to represent the frequency of emergency braking events per unit area. This application can extract acceleration information (i.e., calculate the rate of change of speed) from the trajectory data of vehicles within the target area. When the acceleration between two adjacent trajectory points is less than the speed drop threshold, an emergency braking event is considered to have occurred. The emergency braking events are statistically analyzed according to the frequency of preset road segments or preset grids, and then colored according to the frequency.

[0056] Traffic flow maps depict the spatial distribution of traffic flow, using different colors to represent the traffic volume at corresponding locations. Traffic volume refers to the number of vehicles passing through per unit of time, reflecting the level of traffic congestion. This application can first divide the target area into grids of fixed size, count the traffic volume within each grid, and then color the grids according to the volume of traffic flow.

[0057] Traffic maps show the spatial distribution of traffic information. Different colors can be used to represent the degree of traffic congestion, usually based on vehicle speed.

[0058] Of course, the images in this application may also include other forms of images such as trajectory line diagrams (directly drawing the original trajectory line of the vehicle to show the movement path) and stop point heat maps (using different colors to indicate the length of time the vehicle stays in a certain area), and this application does not make specific limitations.

[0059] This approach transforms discrete trajectory data into structured video or images, enabling better utilization of the visual language model's ability to understand visual information. This results in more accurate traffic conditions within the target area, improving the real-time nature of the obtained traffic information and enhancing the analytical efficiency of the visual language model.

[0060] The following describes in detail step 205, namely, "using a visual language model to obtain the traffic conditions of the target area based on visualization data, wherein the traffic conditions include at least one of the following: congestion level, location of traffic incidents, cause of incidents, and congestion trend information," with reference to an embodiment.

[0061] First, let's explain visual-language models. A visual-language model (VLM) is a type of artificial intelligence model capable of simultaneously understanding and processing visual modal inputs (such as images / videos) and text modal inputs. It achieves cross-modal tasks by learning the association between visual and textual information.

[0062] In this embodiment, a visual language model can be used to obtain the traffic conditions of the target area based on the visualization data obtained in step 203. The traffic conditions include at least one of the following: congestion level (e.g., very congested, moderately congested, slightly congested, no congestion), location of traffic events (represented by geographic coordinates), cause of occurrence (e.g., sudden accidents, road construction, weather changes), and congestion trend information (e.g., congestion worsens within half an hour, congestion dissipates after ten minutes). The traffic events involved in this embodiment refer to various abnormal events occurring within the target area, caused by vehicles, pedestrians, road facilities, or external environmental factors, leading to traffic flow interruption, congestion, disorder, or increased safety risks. These events may include vehicle collisions, vehicle fires, road damage, temporary obstacles, traffic facility malfunctions, road construction, and emergency rescue. These traffic events may or may not have already caused congestion.

[0063] When inputting visualization data into the visual language model, you can also input prompts at the same time. Prompts are used to control the output direction and content of the visual language model.

[0064] While traffic conditions can be obtained based solely on visualized data, the prediction of causes and congestion trends may not be accurate enough. As a preferred embodiment, this application can utilize a visual language model to obtain the traffic conditions of a target area based on visualized data and effective traffic information. The content of the effective traffic information can be in the form of text, images, or videos, and is extracted from at least one of traffic intelligence data, user-reported data, and environmental data.

[0065] Traffic intelligence data refers to data on traffic operation status released by transportation departments through websites or broadcasts, such as traffic flow, congestion index, bus punctuality rate, and traffic control information. User-reported data refers to data on traffic operation status published by users online, such as real-time photos or videos of a target area uploaded by users, or textual descriptions of traffic conditions in a target area published by users. Environmental data refers to weather and geological data. Weather data can include data such as rainfall, visibility, snow depth, and water accumulation, while geological data can include data such as road subsidence and landslide risk.

[0066] For example, the target area is a bridge. Visualized trajectory data shows that initially, there are few vehicles on the bridge with relatively even speed distribution. However, as time progresses, the number of vehicles on the bridge gradually increases, and the distribution becomes uneven, with a significant increase in slow-moving vehicles and a decrease in fast-moving vehicles. Vehicles begin to congregate on the bridge, forming a distinct low-speed convoy. After some time, vehicles near location A on the bridge come to a complete stop. Further analysis using traffic intelligence data reveals that a fire occurred on the bridge, causing partial damage. After multi-source data analysis and prediction using a visual language model, the traffic situation can be determined as follows: severe congestion, location of the traffic incident (A), and cause: bridge damage due to a fire.

[0067] For another example, the target area is a highway. In the first time period, the trajectory points are evenly distributed without obvious dense areas, indicating normal traffic flow and no abnormalities were found. In the second time period, a slight trajectory point shift occurred at location B, but it had little impact on the overall traffic flow. In the third time period, a trajectory point shift occurred at location B, and the range expanded to some extent. Combining user-reported real-scene videos, the visual language model analyzes and predicts multi-source data, resulting in the following traffic conditions: no congestion, traffic occurred at location B, and the cause was temporary lane change.

[0068] Therefore, by inputting additional effective traffic information into the visual language model, the model can conduct a more in-depth analysis of the causes of traffic events, providing a more accurate picture of traffic conditions. Furthermore, this approach, which essentially combines data from different sources to predict traffic conditions, can effectively correct for biases and anomalies in single data points, thus improving both the accuracy and real-time performance of predictions.

[0069] In addition to the three types of data mentioned above, the effective traffic information in this application can also be extracted from data obtained from other channels, such as images or videos of the target area captured by vehicle cameras or road cameras. Another example is extraction from satellite images of the target area. Furthermore, effective traffic information can also be extracted from time-related data, such as holiday data and morning / evening rush hour data. All of this data can help the visual language model predict more accurate traffic conditions. Generally, the visual language model can first obtain information on congestion level, location of traffic events, and congestion trends based on visualization data. Then, it uses the effective traffic information to perform in-depth prediction of the causes of the events, and supplements and verifies the obtained information on congestion level, location of traffic events, and congestion trends, thereby obtaining accurate traffic conditions.

[0070] As one feasible approach, this application can utilize a visual language model to obtain the traffic conditions of a target area based on visualized data and valid traffic information within a currently preset time period. For example, if it is necessary to predict the traffic conditions of a target area between 8:00 AM and 9:00 AM on a certain day, the traffic conditions of the target area can be obtained based on the visualized data and valid traffic information released during that time period. As another example, if it is necessary to predict the traffic conditions of a target area at 8:00 AM on a certain day, the traffic conditions of the target area can be obtained based on the visualized data and valid traffic information released during the 30 minutes prior to 8:00 AM on that day.

[0071] As another possible approach, this application can utilize a visual language model to obtain the traffic conditions of the target area based on visualized data and valid traffic information within the current preset time period, as well as visualized data and valid traffic information from the same historical time period (such as the same day last week or the same time period yesterday). For example, if it is necessary to predict the traffic conditions of the target area at 9:00 AM on Monday this week, the traffic conditions of the target area can be obtained based on visualized data from 8:00 AM to 9:00 AM on Monday this week, visualized data from 8:00 AM to 9:00 AM on historical Mondays, valid traffic information released from 8:00 AM to 9:00 AM on Monday this week, and valid traffic information released from 8:00 AM to 9:00 AM on historical Mondays.

[0072] In this way, the visual language model can combine data from the preset time period and the same historical time period to conduct a more comprehensive and accurate analysis of traffic conditions, further uncovering the root causes of congestion. In particular, combining data from the same historical time period can also help the visual language model better analyze changes in congestion trends.

[0073] Based on this, a visual language model can be used to obtain relatively accurate traffic conditions. These conditions are generally expressed in text form. As a preferred embodiment, this application can also visualize the obtained traffic conditions. Specifically, a visual generation model is used to generate traffic scene images or videos within a target area based on the traffic conditions. The visual generation model can refer to an image generation model, which, based on the textual traffic conditions, marks the corresponding map of the target area to generate a traffic scene map of the target area. For example, it can generate a single-frame image highlighting key information such as the location of traffic events and the degree of congestion. Alternatively, the visual generation model can refer to a video generation model, which, based on the textual traffic conditions, generates descriptive animations, such as synthesizing a continuous video that emphasizes congestion trends.

[0074] By visualizing traffic conditions, the traffic situation can be presented to users in a more intuitive way, enabling them to understand the current situation more efficiently and improving their decision-making efficiency.

[0075] In addition, this application can also generate driving suggestions based on the obtained traffic conditions, and the steps for generating driving suggestions include, but are not limited to, at least one of the following.

[0076] The first aspect involves determining recommended routes for users based on traffic conditions and their personalized information. Personalized information includes frequently used routes, commuting times, and typical driving speeds. For example, if abnormal congestion is predicted for a target area along a frequently used route, this application can determine several alternative routes based on the user's personalized information, including the input start and end points. Each alternative route will be provided with estimated travel time and road complexity information based on the predicted traffic conditions in the target area, helping users make efficient decisions.

[0077] The second aspect involves determining the recommended departure time for the target route based on traffic conditions and target route information. The target route can refer to a route planned by the user, a route selected by the user from the planned routes, or a route the user is navigating to, with the target area located on the target route. As one possible approach, this application can provide the user with a recommended departure time based on traffic conditions, including congestion trend information. Alternatively, this application can determine the recommended departure time for the target route based on a combination of traffic conditions and target route information. The target route information can include weather information for the area where the target route is located and information on special events affecting the target route (such as large-scale events). When special events occur or weather conditions are severe, the recommended departure time determined by this application can avoid factors that may lead to extreme congestion on the target route.

[0078] The third aspect: Generating risk warning information based on traffic conditions. Specifically, traffic conditions include causes such as road construction and traffic accidents. This application can determine whether the congestion level and cause of the incident affect the user's safe travel, and then generate risk warning information, which is displayed to the user through their device. When the travel risk is low, the generated risk warning information can be sent to users within a smaller area surrounding the target area; when the travel risk is high, the generated risk warning information can be sent to users within a larger area surrounding the target area.

[0079] Based on the above driving suggestions, routes that are more in line with users' own habits can be provided, and users can be reminded to avoid extremely congested areas to avoid delays caused by unforeseen factors. This also helps to improve the overall driving safety level and thus enhance the user's driving experience.

[0080] Since general-purpose visual language models already possess a strong ability to understand visual patterns, the visual language model used in the above process can be a general-purpose visual language model. However, general-purpose visual language models are not strong in understanding traffic scenarios; that is, they may not be adept at understanding traffic information within visual patterns. Therefore, as a preferred approach, a general-purpose visual language model can be trained specifically for traffic scenarios, and the trained visual language model can be used to perform the aforementioned traffic condition predictions.

[0081] One feasible approach is to train the visual language model using supervised fine-tuning (SFT), which involves further fine-tuning the model parameters on labeled data for a specific task to adapt it to downstream tasks. Another feasible approach is to apply reinforcement learning (RL) to the visual language model. Yet another feasible approach is to first perform supervised fine-tuning on the initial visual language model to obtain an intermediate model, and then apply reinforcement learning to this intermediate model to obtain the final visual language model. Reinforcement learning is a machine learning method that autonomously optimizes its policy based on reward values.

[0082] Furthermore, the visual language model can be pre-trained using at least one of the following training tasks (e.g.) Figure 6 As shown, for each training task, supervised fine-tuning and reinforcement learning steps can be included:

[0083] The first type: map element recognition task. Map data samples are used as input to the visual language model, and the map elements (such as roads, buildings, etc.) marked in the map data samples are used as the target output of the visual language model to train the visual language model.

[0084] Specifically, during supervised fine-tuning of the initial visual language model, map data samples are input into the initial visual language model to obtain predicted map elements corresponding to the map data samples output by the initial visual language model. Based on the difference between the predicted map elements and the labeled map elements in the map data samples, a first loss value is obtained. Minimizing the first loss value is used as the training objective to fine-tune the model parameters of the initial visual language model, resulting in an intermediate model.

[0085] When performing reinforcement learning on the intermediate model, a reward mechanism is set up. Based on the predicted map elements output by the intermediate model, it is determined whether the intermediate model has correctly identified a specific map element. If it has, a positive reward is given to the intermediate model; otherwise, a negative reward is given. This results in the reward value of the intermediate model. The model parameters of the intermediate model are updated to maximize the reward value, so that the intermediate model can learn to locate key feature points more efficiently and gradually improve its performance until it reaches a better state.

[0086] The goal is to enable visual language models to automatically parse map elements on a map and understand unstructured information on the map (such as text labels).

[0087] The second type is traffic flow prediction. Historical traffic visualization samples (such as historical traffic videos) are used as input to the visual language model, and future traffic flow information (such as vehicle flow information) labeled for historical periods is used as the target output of the visual language model to train the model.

[0088] Specifically, during supervised fine-tuning of the initial visual language model, traffic visualization samples from historical time periods are input into the initial visual language model to obtain predicted traffic flow information for future time periods output by the initial visual language model. Based on the difference between the predicted traffic flow information for future time periods and the traffic flow information for future time periods labeled for historical time periods, a second loss value is obtained. Minimizing the second loss value is used as the training objective to fine-tune the model parameters of the initial visual language model, resulting in an intermediate model.

[0089] When performing reinforcement learning on the intermediate model, a reward mechanism is set up. Based on the accuracy of the traffic flow prediction information for future time periods output by the intermediate model, rewards and penalties are allocated. If the traffic flow information predicted by the intermediate model is accurate, a positive reward is given; if the traffic flow information predicted by the intermediate model is incorrect, a negative reward is given. This results in the reward value of the intermediate model. The model parameters of the intermediate model are updated to maximize the reward value, so that the intermediate model can gradually learn how to better understand and predict complex traffic patterns.

[0090] The goal is to train a visual language model to predict traffic flow trends over a future period based on current road conditions. This is significant for alleviating congestion and optimizing route selection.

[0091] The third type: intelligence extraction task. Traffic intelligence visualization samples (such as images or videos released by the traffic department) are used as input to the visual language model, and event information (such as keywords, information points, basic elements, etc.) labeled on the traffic intelligence visualization samples are used as the target output of the visual language model to train the visual language model.

[0092] Specifically, during supervised fine-tuning of the initial visual language model, traffic intelligence visualization samples are input into the initial visual language model to obtain the predicted event information output by the initial visual language model. Based on the difference between the predicted event information and the event information labeled for the traffic intelligence visualization samples, a third loss value is obtained. Minimizing the third loss value is used as the training objective to fine-tune the model parameters of the initial visual language model, resulting in an intermediate model.

[0093] When performing reinforcement learning on the intermediate model, a reward mechanism is set up. Based on the accuracy of the predicted event information output by the intermediate model, rewards and penalties are allocated. If the predicted event information output by the intermediate model is accurate, a positive reward is given; if the predicted event information output by the intermediate model is incorrect, a negative reward is given. This results in the reward value of the intermediate model, which is used to update the model parameters of the intermediate model to maximize the reward value and improve the intermediate model's ability to process complex intelligence.

[0094] The goal is to train a visual language model to transform this raw traffic information into useful knowledge, and to extract information about traffic accidents or special events from the network.

[0095] The fourth type: User-reported extraction task. The visual language model is trained by using the user-reported visualization samples (such as user-uploaded images or videos) as input and the event information (such as keywords, information points, basic elements, etc.) labeled on the user-reported visualization samples as the target output.

[0096] Specifically, during supervised fine-tuning of the initial visual language model, user-reported visualization samples are input into the initial visual language model to obtain predicted event information output by the initial visual language model. Based on the difference between the predicted event information and the event information labeled for the user-reported visualization samples, a fourth loss value is obtained. Minimizing the fourth loss value is used as the training objective to fine-tune the model parameters of the initial visual language model, resulting in an intermediate model.

[0097] When performing reinforcement learning on the intermediate model, a reward mechanism is set up. Based on the accuracy, efficiency, and quality of the predicted event information output by the intermediate model, rewards and penalties are allocated. The faster, more accurate, and better the quality of the output predicted event information, the higher the reward value. The slower, less accurate, and worse the quality of the output predicted event information, the lower the reward value. This maximizes the reward value to update the model parameters of the intermediate model, thereby prompting the intermediate model to continuously optimize its own parameters.

[0098] The goal is to train a visual language model to efficiently and accurately identify user-reported data in order to extract important information from it.

[0099] The fifth type: Multi-source information analysis task. Multiple traffic segment samples (such as trajectory data, social media posts, news reports, etc.) within the same time period and targeting the same area are used as input to the visual language model. Traffic conditions (such as at least one of the following: congestion level, location of traffic incident, cause of incident, and congestion trend information) labeled for multiple traffic segment samples are used as the target output of the visual language model to train the visual language model.

[0100] Specifically, during supervised fine-tuning of the initial visual language model, multiple traffic segment samples from the same time period and targeting the same area are input into the initial visual language model to obtain the predicted traffic conditions output by the initial visual language model. Based on the difference between the predicted traffic conditions and the traffic conditions labeled for the multiple traffic segment samples, a fifth loss value is obtained. Minimizing the fifth loss value is used as the training objective to fine-tune the model parameters of the initial visual language model, resulting in an intermediate model.

[0101] When performing reinforcement learning on the intermediate model, a reward mechanism is set up. Based on the accuracy of the traffic condition prediction output of the intermediate model, rewards and penalties are allocated. If the traffic condition prediction output of the intermediate model is accurate, a positive reward is given; if the traffic condition prediction output of the intermediate model is incorrect, a negative reward is given. This results in the reward value of the intermediate model. The model parameters of the intermediate model are updated to maximize the reward value, thereby prompting the intermediate model to improve its understanding ability.

[0102] In simple terms, in practical applications, it is often necessary to integrate data from different sources to gain a comprehensive understanding of the traffic conditions in a given area. However, because the data may use different time bases or spatial coordinate systems, they must first be processed uniformly so that the visual language model can integrate the traffic conditions of the target area based on data from different sources—a process known as "spatiotemporal alignment." This training task is crucial for the visual language model to achieve cross-platform and cross-domain comprehensive analysis, enabling it to obtain a more complete and accurate view of traffic conditions.

[0103] Training the visual language model using multiple tasks allows for a deeper analysis of various information, enhancing its understanding of traffic conditions and leading to more accurate and comprehensive traffic information. Furthermore, training the model using supervised fine-tuning or reinforcement learning effectively combines the advantages of both methods, further improving the visual language model's performance across these tasks and enabling it to output even more accurate traffic information.

[0104] Furthermore, the reinforcement learning process mentioned above is only one feasible approach. In practical applications, rewards and penalties can also be assigned based on indicators such as whether the format of the visual language model's output is correct and whether its length meets the requirements. This application does not impose any specific limitations on this.

[0105] It should be noted that supervised fine-tuning primarily focuses on improving the basic cognitive level of the visual language model, while reinforcement learning focuses on further optimizing the behavioral strategies of the visual language model through interactive learning. This application can iteratively train the visual language model using these two training methods. For example, after every N rounds of reinforcement learning, the visual language model is re-trained using new sample data.

[0106] It should also be noted that this application can use at least one of the above training tasks to train the visual language model. As a preferred embodiment, the visual language model can be trained first based on the first four training tasks so that the visual language model learns basic traffic understanding ability, and then trained based on the multi-source information analysis task so that the visual language model learns the spatiotemporal alignment ability of multi-source information, that is, the ability to integrate information in spatiotemporal space.

[0107] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0108] According to another embodiment, a traffic condition prediction device is provided. Figure 7 A schematic block diagram of a traffic condition prediction device according to one embodiment is shown, the device being disposed in... Figure 1 The server side in the illustrated architecture. For example... Figure 7As shown, the device 700 includes a data acquisition unit 701, a data processing unit 702, and a model output unit 703, and further includes a vision generation unit 704, a suggestion generation unit 705, and a model training unit 706. The main functions of each component are as follows:

[0109] The data acquisition unit 701 is configured to acquire trajectory data of vehicles within the target area.

[0110] The data processing unit 702 is configured to perform visualization processing on the trajectory data to obtain visualized data.

[0111] The model output unit 703 is configured to use a visual language model to obtain the traffic conditions of a target area based on visualization data. The traffic conditions include at least one of the following: congestion level, location of traffic incidents, cause of incidents, and congestion trend information.

[0112] As one possible embodiment, when the model output unit 703 obtains the traffic conditions of the target area based on the visual data using the visual language model, it is specifically configured to: obtain the traffic conditions of the target area based on the visual data and effective traffic information using the visual language model, wherein the effective traffic information is extracted from at least one of traffic intelligence data, user-reported data and environmental data.

[0113] As one possible embodiment, when the model output unit 703 obtains the traffic conditions of the target area using a visual language model based on visual data and effective traffic information, it is specifically configured to: obtain the traffic conditions of the target area using a visual language model based on visual data and effective traffic information within the current preset time period, as well as visual data and effective traffic information within the same historical time period.

[0114] As one possible embodiment, the visualization data includes video and / or images; the video represents the movement trajectory of vehicles within the target area; the images include attribute feature maps of vehicles within the target area, the attribute feature maps including at least one of velocity maps, acceleration maps, density maps, flow maps, and emergency braking heat maps.

[0115] Furthermore, the visual generation unit 704 can be configured to generate traffic scene images or traffic scene videos within a target area based on traffic conditions using a visual generation model.

[0116] Furthermore, it is suggested that the generation unit 705 can be configured to: determine a recommended route for a user based on traffic conditions and the user's personalized information; determine a recommended departure time for the target route based on traffic conditions and the target route information; and generate risk warning information based on traffic conditions.

[0117] Furthermore, the model training unit 706 can be configured to pre-train the visual language model using at least one of the following training tasks: Map element recognition task: using map data samples as input to the visual language model, and map elements labeled in the map data samples as the target output of the visual language model, to train the visual language model; Traffic flow prediction task: using traffic visualization samples from historical periods as input to the visual language model, and traffic flow information for future periods labeled for historical periods as the target output of the visual language model, to train the visual language model; Intelligence extraction task: using traffic intelligence visualization samples as input to the visual language model, and event information labeled for traffic intelligence visualization samples as the target output of the visual language model, to train the visual language model; User report extraction task: using visualization samples reported by users as input to the visual language model, and event information labeled for user-reported visualization samples as the target output of the visual language model, to train the visual language model; Multi-source information analysis task: using multiple traffic segment samples within the same time period and for the same area as input to the visual language model, and traffic conditions labeled for multiple traffic segment samples as the target output of the visual language model, to train the visual language model.

[0118] As one possible implementation, the model training unit 706 can be specifically configured during training to: first perform supervised fine-tuning on the initial visual language model using at least one of the training tasks to obtain an intermediate model, and then perform reinforcement learning on the intermediate model to obtain a visual language model.

[0119] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0120] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0121] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0122] And an electronic device, comprising:

[0123] One or more processors; and

[0124] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0125] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0126] in, Figure 8 An exemplary architecture of an electronic device is shown, which may include a processor 810, a video display adapter 811, a disk drive 812, an input / output interface 813, a network interface 814, and a memory 820. The processor 810, video display adapter 811, disk drive 812, input / output interface 813, network interface 814, and memory 820 can communicate with each other via a communication bus 830.

[0127] The processor 810 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0128] The memory 820 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 820 can store the operating system 821 for controlling the operation of the electronic device 800, and the basic input / output system (BIOS) 822 for controlling the low-level operations of the electronic device 800. Additionally, it can store a web browser 823, a data storage management system 824, and a traffic condition prediction device 700, etc. The aforementioned traffic condition prediction device 700 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 820 and is called and executed by the processor 810.

[0129] The input / output interface 813 is used to connect input / output modules to enable information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0130] Network interface 814 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0131] Bus 830 includes a pathway for transmitting information between various components of the device, such as processor 810, video display adapter 811, disk drive 812, input / output interface 813, network interface 814, and memory 820.

[0132] It should be noted that although the above-described device only shows the processor 810, video display adapter 811, disk drive 812, input / output interface 813, network interface 814, memory 820, bus 830, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0133] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0134] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A traffic condition prediction method, characterized in that, The method includes: Acquire vehicle trajectory data within the target area; The trajectory data is then visualized to obtain visualized data; Using a visual language model, the traffic conditions of the target area are obtained based on the visualization data. The traffic conditions include at least one of the following: congestion level, location of traffic incidents, cause of incidents, and congestion trend information.

2. The method according to claim 1, characterized in that, The process of using a visual language model to obtain the traffic conditions of the target area based on the visualization data includes: Using a visual language model, the traffic conditions of the target area are obtained based on the visualization data and effective traffic information, wherein the effective traffic information is extracted from at least one of traffic intelligence data, user-reported data, and environmental data.

3. The method according to claim 2, characterized in that, The process of using a visual language model to obtain the traffic conditions of the target area based on the visualized data and effective traffic information includes: Using a visual language model, based on the visualized data and valid traffic information within the current preset time period, as well as the visualized data and valid traffic information within the same historical time period, the traffic conditions of the target area are obtained.

4. The method according to any one of claims 1 to 3, characterized in that, The visualization data includes videos and / or images; The video shows the movement trajectory of the vehicle within the target area; The image includes attribute feature maps of vehicles within the target area, the attribute feature maps including at least one of speed map, acceleration map, density map, flow rate map, road condition map, and emergency braking heat map.

5. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Using a visual generation model, traffic scene images or videos are generated within the target area based on the traffic conditions.

6. The method according to any one of claims 1 to 3, characterized in that, The method further includes at least one of the following: Based on the traffic conditions and the user's personalized information, a recommended route is determined for the user; Based on the traffic conditions and the target route information, a recommended departure time for the target route is determined; Based on the traffic conditions, risk warning information is generated.

7. The method according to any one of claims 1 to 3, characterized in that, The visual language model is pre-trained using at least one of the following training tasks: Map element recognition task: Use map data samples as input to the visual language model, and use the map elements marked in the map data samples as the target output of the visual language model to train the visual language model; Traffic flow prediction task: Use traffic visualization samples from historical time periods as input to the visual language model, and use traffic flow information for future time periods labeled for the historical time periods as the target output of the visual language model to train the visual language model; Intelligence extraction task: Use traffic intelligence visualization samples as input to the visual language model, and use the event information labeled for the traffic intelligence visualization samples as the target output of the visual language model to train the visual language model; User-reported extraction task: The visual language model is trained by taking the user-reported visual samples as input and the event information labeled for the user-reported visual samples as the target output of the visual language model. Multi-source information analysis task: Take multiple traffic segment samples within the same time period and targeting the same area as input to the visual language model, and take the traffic conditions labeled for the multiple traffic segment samples as the target output of the visual language model to train the visual language model.

8. The method according to claim 7, characterized in that, The training includes: Using at least one of the training tasks, the initial visual language model is first subjected to supervised fine-tuning to obtain an intermediate model, and then the intermediate model is subjected to reinforcement learning to obtain the visual language model.

9. A traffic condition prediction device, characterized in that, The device includes: The data acquisition unit is configured to acquire trajectory data of vehicles within the target area; The data processing unit is configured to perform visualization processing on the trajectory data to obtain visualized data; The model output unit is configured to use a visual language model to obtain the traffic conditions of the target area based on the visualization data. The traffic conditions include at least one of the following: congestion level, location of traffic incidents, cause of incidents, and congestion trend information.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 8.

Images