Operation vehicle travel risk management and control system and method

Abstract

The invention relates to the field of travel risk management and control, and particularly discloses a commercial vehicle travel risk management and control system and method. The method comprises the following steps: firstly, acquiring traffic monitoring videos of each region of a service truck route range acquired by a camera, traffic road information of the service truck route range acquired by a database and travel environment information of the service truck route range, and then performing feature extraction and correlation analysis on the three information by using a deep learning technology; and finally, a classification result is obtained through a classifier so as to judge whether to send out an early warning of the travel congestion risk of the operating truck, thereby helping a driver and operation management personnel to take actions, reducing the uncertainty in the travel, and improving the transportation efficiency and punctuality.

Description

Technical Field

[0001] The present application relates to the field of travel risk management and control, and more specifically, to a travel risk management system and method for commercial vehicles. Background Art

[0002] Commercial vehicles are vehicles used for commercial transportation and services, including trucks, buses, taxis, etc. These vehicles are usually involved in a large number of daily operations and undertake the task of transporting goods or traveling passengers.

[0003] With the increase in the number of vehicles, especially during holidays and peak hours, traffic congestion is becoming more and more serious, which poses a challenge to the punctuality and efficiency of operating trucks. Although the existing radio broadcast system and navigation system can provide traffic information on congested roads, they are often based on manual feedback from drivers, which is lagging.

[0004] Therefore, a system and method for managing and controlling the travel risks of commercial vehicles is desired. Summary of the invention

[0005] In order to solve the above technical problems, this application is proposed. The embodiment of this application provides a travel risk management system and method for operating vehicles, which first obtains traffic monitoring videos of various areas within the operating truck route range collected by the camera, traffic road information within the operating truck route range collected by the database, and travel environment information within the operating truck route range, and then uses deep learning technology to perform feature extraction and correlation analysis on the three, and finally obtains classification results through a classifier to determine whether to issue a congestion risk warning for operating truck travel, thereby helping drivers and operation managers to take action, reduce uncertainty in the journey, and improve transportation efficiency and punctuality.

[0006] According to one aspect of the present application, a travel risk management system for commercial vehicles is provided, which includes:

[0007] The operating vehicle travel data acquisition module is used to obtain the traffic monitoring video of each area of ​​the operating truck route range collected by the camera, the traffic road information of the operating truck route range collected by the database, and the travel environment information of the operating truck route range;

[0008] An operating vehicle travel data extraction module is used to extract a multimodal feature vector of traffic congestion information and a semantic association feature vector of travel environment information from the traffic monitoring videos of each area of ​​the operating truck route range collected by the camera, the traffic road information of the operating truck route range collected by the database, and the travel environment information of the operating truck route range;

[0009] The travel congestion risk warning module is used to determine whether to issue a travel congestion risk warning for operating trucks based on the multimodal feature vector of the traffic congestion information and the semantic association feature vector of the travel environment information.

[0010] According to another aspect of the present application, a method for managing and controlling travel risks of an operating vehicle is provided, comprising:

[0011] Obtaining traffic monitoring videos of various areas within the operating truck route range collected by cameras, traffic road information within the operating truck route range collected by a database, and travel environment information within the operating truck route range;

[0012] Extracting a multimodal feature vector of traffic congestion information and a semantic association feature vector of travel environment information from the traffic monitoring videos of each area of ​​the operating truck route range collected by the camera, the traffic road information of the operating truck route range collected by the database, and the travel environment information of the operating truck route range;

[0013] Based on the multimodal feature vector of the traffic congestion information and the semantic association feature vector of the travel environment information, it is determined whether to issue a travel congestion risk warning for operating trucks.

[0014] Compared with the prior art, the present application provides a travel risk management system and method for operating vehicles, which first obtains traffic monitoring videos of various areas within the operating truck route range collected by cameras, traffic road information within the operating truck route range collected by a database, and travel environment information within the operating truck route range, and then uses deep learning technology to perform feature extraction and correlation analysis on the three. Finally, a classifier is used to obtain classification results to determine whether to issue a travel congestion risk warning for operating trucks, thereby helping drivers and operation managers to take action, reduce uncertainty in the journey, and improve transportation efficiency and punctuality. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 It is a block diagram of a travel risk management system for commercial vehicles according to an embodiment of the present application.

[0017] Figure 2 It is a block diagram of a commercial vehicle travel data extraction module in the commercial vehicle travel risk management and control system according to an embodiment of the present application.

[0018] Figure 3 It is a block diagram of a traffic congestion information feature extraction unit in a travel risk management and control system for commercial vehicles according to an embodiment of the present application.

[0019] Figure 4 It is a block diagram of a travel congestion risk warning module in a travel risk management and control system for commercial vehicles according to an embodiment of the present application.

[0020] Figure 5 It is a flow chart of a method for controlling travel risks of commercial vehicles according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0022] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0023] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0024] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0025] Figure 1 Schematic diagram of the operating vehicle travel risk management system of the embodiment of the present application. Figure 1As shown, according to the embodiment of the present application, the operating vehicle travel risk management system 100 includes: an operating vehicle travel data acquisition module 110, which is used to obtain the traffic monitoring video of each area of ​​the operating truck route range collected by the camera, the traffic road information of the operating truck route range collected by the database, and the travel environment information of the operating truck route range; an operating vehicle travel data extraction module 120, which is used to extract the traffic congestion information multimodal feature vector and the travel environment information semantic association feature vector from the traffic monitoring video of each area of ​​the operating truck route range collected by the camera, the traffic road information of the operating truck route range collected by the database, and the travel environment information of the operating truck route range; a travel congestion risk warning module 130, which is used to determine whether to issue an operating truck travel congestion risk warning based on the traffic congestion information multimodal feature vector and the travel environment information semantic association feature vector.

[0026] In the above-mentioned operating vehicle travel risk management system 100, the operating vehicle travel data acquisition module 110 is used to obtain traffic monitoring videos of various areas within the operating truck route range collected by the camera, traffic road information within the operating truck route range collected by the database, and travel environment information within the operating truck route range. It should be understood that operating vehicles, such as trucks, buses and taxis, are mainly used for commercial transportation and services, involving a large amount of daily operations to transport goods or passengers. However, with the increase in the number of vehicles, especially during holidays and peak hours, traffic congestion has become more and more serious, which has brought challenges to the punctuality and efficiency of operating trucks. Although existing radio broadcasts and navigation systems can provide information on congested sections, they usually rely on feedback from drivers, resulting in slow information updates. Therefore, in the technical solution of the present application, by obtaining traffic monitoring videos of various areas within the operating truck route range collected by cameras, traffic road information within the operating truck route range collected by the database, and travel environment information within the operating truck route range, and combining deep learning technology, it is determined whether to issue a congestion risk warning to the operating truck. This process can help drivers and operations managers take appropriate measures to reduce uncertainty in the journey, thereby improving the efficiency and punctuality of transportation.

[0027] Specifically, obtaining traffic monitoring videos of various areas within the operating truck route range collected by cameras, traffic road information within the operating truck route range collected by the database, and travel environment information within the operating truck route range can help to fully understand and analyze the traffic conditions and environmental characteristics of the truck's driving route, thereby providing data support for risk prediction and management. First, the traffic monitoring videos collected by the camera provide real-time road conditions, including traffic flow, traffic light status, and emergencies on the road. These video data can help identify traffic congestion, accidents, or other factors that may affect truck driving. Secondly, the traffic road information collected by the database covers detailed road data within the truck route range. This information includes the structure of the road, the number of lanes, speed limit regulations, traffic signs, and historical traffic flow data. Finally, the travel environment information within the operating truck route range includes meteorological conditions, road maintenance conditions, construction information, etc. By integrating these multi-source data, the system can more comprehensively analyze the risk factors that may be encountered during truck driving, thereby improving the accuracy and reliability of early warnings, helping freight managers to better plan and adjust driving strategies, reduce traffic delays, and improve transportation efficiency.

[0028] In the above-mentioned operating vehicle travel risk management and control system 100, the operating vehicle travel data extraction module 120 is used to extract the multimodal feature vector of traffic congestion information and the semantic association feature vector of travel environment information from the traffic monitoring video of each area of ​​the operating truck route range collected by the camera, the traffic road information of the operating truck route range collected by the database, and the travel environment information of the operating truck route range. In this way, the extracted multimodal feature vector and semantic association feature vector provide a comprehensive perspective that combines video surveillance, road data and environmental factors. This comprehensive analysis not only improves the ability to identify traffic congestion risks, but also can effectively predict and manage potential traffic problems, thereby optimizing the driving routes and times of trucks, reducing transportation delays, and improving transportation efficiency.

[0029] Figure 2 FIG. 1 is a block diagram of a commercial vehicle travel data extraction module in a commercial vehicle travel risk management system according to an embodiment of the present application. Figure 2 As shown, in a specific embodiment of the present application, the operating vehicle travel data extraction module 120 includes: a traffic congestion information feature extraction unit 121, which is used to extract features from the traffic monitoring videos of each area of ​​the operating truck route range collected by the camera and the traffic road information of the operating truck route range collected by the database to obtain the multimodal feature vector of the traffic congestion information; a travel environment information feature extraction unit 122, which is used to extract features from the travel environment information of the operating truck route range to obtain the travel environment information semantic association feature vector.

[0030] It should be understood that the traffic monitoring video collected by the camera contains real-time road conditions, including traffic flow, vehicle speed, traffic signals and emergencies. After feature extraction, these video data can be converted into specific feature vectors, which describe the density of traffic flow, changes in vehicle speed and the degree of traffic congestion. This feature extraction can convert complex visual information into structured data, thereby providing real-time traffic congestion conditions, identifying traffic bottlenecks and potential congestion points, and making forward-looking predictions of possible traffic congestion conditions.

[0031] Furthermore, feature extraction of travel environment information within the operating truck route range is performed to obtain semantically associated feature vectors of travel environment information in order to fully understand and predict the driving risk and efficiency of trucks under different environmental conditions. This feature extraction can not only enhance the ability to analyze environmental factors, but also effectively integrate this information into the traffic management and optimization system to improve the scientificity and accuracy of decision-making. Among them, travel environment information includes multiple aspects, such as weather conditions (such as rain, snow, fog), road maintenance conditions (such as construction, closure) and environmental factors (such as air quality, temperature). These factors can significantly affect the driving safety and efficiency of trucks. Changes in these environmental factors are directly related to traffic smoothness and safety. The feature vector of road maintenance conditions can provide detailed information about the construction area, helping truck drivers or transportation companies to plan detour routes in advance. By integrating the features of these environmental information, the semantically associated feature vector can reveal the relationship between different environmental factors and their comprehensive impact on traffic. This comprehensive analysis not only helps to understand the impact of a single environmental factor, but also reveals the joint effects of multiple environmental factors, thereby providing more comprehensive risk assessment and optimization suggestions.

[0032] Figure 3 FIG. 1 is a block diagram of a traffic congestion information feature extraction unit in a travel risk management system for commercial vehicles according to an embodiment of the present application. Figure 3 As shown, in a specific embodiment of the present application, the traffic congestion information feature extraction unit 121 includes: a traffic monitoring feature encoding subunit 1211, which is used to feature encode the traffic monitoring video of each area of ​​the operating truck route range collected by the camera to obtain a regional traffic monitoring understanding feature vector; a traffic road information feature encoding subunit 1212, which is used to feature encode the traffic road information of the operating truck route range collected by the database to obtain multiple traffic road information understanding feature vectors; a traffic congestion information feature association subunit 1213, which is used to associate the regional traffic monitoring understanding feature vector and the multiple traffic road information understanding feature vectors to obtain the traffic congestion information multimodal feature vector.

[0033] It should be understood that traffic surveillance videos contain a large amount of real-time and dynamic data, such as traffic flow, vehicle speed, traffic light status, and traffic events (such as accidents, congestion, etc.). Although this information is intuitive in the video, in order to achieve effective analysis and application, it is necessary to convert these complex data into a form that can be processed by computers. Feature encoding is the process of extracting key information from video data and converting it into feature vectors. These feature vectors can accurately represent various traffic conditions in the video. The feature encoding process usually involves extracting several key features from the video, such as the movement trajectory of the vehicle, the density of traffic flow, the change of traffic signals, and the frequency of events. These feature-encoded vectors can reflect the traffic status in the area, such as whether a certain section of the road is congested, whether the traffic flow is increasing, or whether there are emergencies. In this way, the system can quickly understand the current and historical traffic conditions and make better decisions.

[0034] Furthermore, feature encoding is performed on the traffic road information of the operating truck route range collected by the database to obtain multiple traffic road information understanding feature vectors, in order to convert static and structured road data into a data form that can be used for in-depth analysis and decision-making. Among them, the traffic road information in the database usually includes road type, number of lanes, speed limit regulations, traffic signs, historical road condition data, etc. Although this information is structured, it is usually raw data, and direct analysis may not be intuitive or difficult to integrate. Feature encoding extracts key information from road information and converts it into concise and representative vectors, which can effectively describe various characteristics of roads in multiple dimensions. For example, feature encoding can convert basic attributes of roads (such as road types: highways, urban roads, rural roads, etc.), traffic control information (such as speed limits, no-drive areas), and historical traffic flow of roads into vectors. These feature vectors can include information such as road capacity, traffic density, and past accident frequency, thereby helping to form a comprehensive understanding of road conditions. In this way, the system can evaluate the condition of roads in multiple dimensions, thereby optimizing transportation strategies and improving transportation efficiency.

[0035] Furthermore, the regional traffic monitoring understanding feature vector is associated with multiple traffic road information understanding feature vectors to obtain a multimodal feature vector of traffic congestion information in order to achieve a comprehensive analysis of traffic conditions and provide more accurate and comprehensive traffic congestion prediction and management. This process can improve the understanding of the traffic system by integrating different types of data, thereby optimizing decision-making and improving transportation efficiency. Among them, the regional traffic monitoring understanding feature vectors are usually derived from real-time video data collected by cameras, which can reflect dynamic information such as traffic flow, vehicle speed, and traffic events. These feature vectors provide an instant view of the current traffic conditions, such as whether a certain area is congested and whether vehicles move smoothly. On the other hand, the traffic road information understanding feature vector is derived from structured data in the database, including road types, speed limits, traffic signs, and historical traffic data. This information provides static properties and long-term usage of roads, helping to understand the basic conditions and long-term traffic trends of roads. By combining this information, the cause of congestion can be identified, such as whether it is due to unreasonable road design, traffic light configuration problems, or emergencies. This comprehensive analysis capability can not only provide more accurate congestion forecasts, but also provide a basis for optimizing traffic management measures, such as adjusting traffic signal timing, improving road design or suggesting detour routes.

[0036] In a specific embodiment of the present application, the traffic monitoring feature encoding subunit 1211 includes: extracting multiple regional traffic monitoring key frames from the traffic monitoring videos of each area of ​​the operating truck route range collected by the camera; arranging the multiple regional traffic monitoring key frames into a regional traffic monitoring input tensor; and passing the regional traffic monitoring input tensor through a regional traffic monitoring feature encoder based on a convolutional neural network to obtain the regional traffic monitoring understanding feature vector.

[0037] It should be understood that traffic surveillance videos usually cover long time periods and wide areas. Direct analysis of the entire video data is not only computationally intensive, but may also contain redundant information. By extracting key frames, the amount of data can be effectively reduced, and attention can be focused on important moments and areas, so that traffic conditions can be analyzed more efficiently. The main purpose of key frame extraction is to select those frames that best represent the traffic status from continuous video frames. These key frames can capture critical moments of traffic flow, such as peak traffic, congestion, accidents, etc., so they provide important perspectives for analysis. Among them, the key frames select the most representative scenes, which allows traffic analysts to quickly evaluate traffic conditions at critical moments, avoid analyzing all video frames one by one, and improve work efficiency.

[0038] Furthermore, multiple regional traffic monitoring keyframes are arranged as regional traffic monitoring input tensors in order to effectively integrate the information of time series and spatial regions, thereby providing structured input for further data analysis and model training. The structured processing of regional traffic monitoring input tensors can effectively integrate keyframes from different regions. Each keyframe represents the traffic status at a specific time point and region. After these keyframes are arranged in chronological order, a continuous time series can be constructed, reflecting the dynamic changes of traffic flow and status. By organizing these keyframes into tensors, time series data can be combined with spatial information to create a multi-dimensional data structure for further analysis and processing.

[0039] Furthermore, considering that the input tensor of regional traffic monitoring usually consists of multiple key frames, these frames represent the traffic conditions at different time points. Each frame contains rich spatial information, such as the location of vehicles, traffic flow, road signs, etc. Directly processing these image data may face the challenges of high dimensionality and complexity, so it is necessary to convert them into more concise and useful feature representations. Convolutional neural network (CNN) is a very effective image processing technology, which can extract and abstract features in images layer by layer through multiple convolution layers, pooling layers and activation functions. In CNN, the convolution layer extracts local features of the input image by applying multiple convolution kernels (filters). Each convolution kernel can identify different patterns in the image, such as edges, textures or specific shapes, which are crucial for understanding traffic conditions. After multiple layers of convolution and pooling operations, the network can capture high-level features in the image, such as traffic flow patterns, congested areas or abnormal events. These high-level features will be integrated and compressed in the last few layers of the network to form a regional traffic monitoring understanding feature vector, which is a high-level abstract representation of the input tensor. Specifically, each layer of the regional traffic monitoring feature encoder based on the convolutional neural network is used to perform convolution processing, mean pooling processing based on the local feature matrix and nonlinear activation processing on the input data in the forward pass of the layer so that the last layer of the regional traffic monitoring feature encoder based on the convolutional neural network outputs the regional traffic monitoring understanding feature vector, wherein the input of the regional traffic monitoring feature encoder based on the convolutional neural network is the regional traffic monitoring input tensor.

[0040] In a specific embodiment of the present application, the traffic road information feature encoding subunit 1212 includes: segmenting the traffic road information of the operating truck route range collected by the database to obtain a traffic road information word sequence; passing the traffic road information word sequence through a traffic road information text context semantic encoder to obtain the multiple traffic road information understanding feature vectors.

[0041] It should be understood that the traffic road information in the database usually includes road name, type, length, speed limit, traffic signs, road facilities, etc. Although this information is very useful, it is usually stored in the database in the form of structured data, which is not necessarily directly applicable to natural language processing and many types of analysis. Word segmentation is a process of dividing text or data fragments into smaller units, which can be words, phrases or other units of meaning. In traffic road information, the purpose of word segmentation is to break down various information into separate, meaningful parts for further analysis and processing. This processing method makes it possible to use natural language processing techniques and text analysis methods, such as word frequency analysis, topic modeling and information retrieval, to identify patterns and trends in road data.

[0042] Furthermore, the traffic road information word sequence is a text sequence obtained by segmenting the original road data in the database. These word sequences include various attributes about the road, such as road type, speed limit, sign information, etc. However, these words are isolated in the original text sequence and lack a deep understanding of their context and mutual relationship. Therefore, relying solely on the words themselves may not fully capture the complex semantics and actual meaning of road information. The main purpose of the traffic road information text context semantic encoder is to generate a feature vector that can express the semantics of the word by analyzing the contextual relationship of the word. Such encoders are usually based on advanced natural language processing technologies, such as word embedding, context-aware models (such as BERT), etc. These technologies can convert each word or phrase into a high-dimensional vector, which not only contains the basic meaning of the word, but also includes the semantic information of the word in a specific context. Specifically, the context semantic encoder generates corresponding feature vectors by encoding each word in the word sequence, and these feature vectors can capture the semantic relationship and context information between words. Specifically, the embedding layer of the traffic road information text context semantic encoder is used to map each traffic road information word in the traffic road information word sequence into a word embedding vector to obtain a sequence of traffic road information word embedding vectors; and the converter-based Bert model of the traffic road information text context semantic encoder is used to perform global context semantic encoding on the sequence of traffic road information word embedding vectors to obtain multiple traffic road information understanding feature vectors.

[0043] In a specific embodiment of the present application, the travel environment information feature extraction unit 122 includes: passing the travel environment information of the operating truck route range through a travel environment information semantic encoder to obtain a plurality of travel environment information semantic feature vectors; and cascading the plurality of travel environment information semantic feature vectors into the travel environment information semantic association feature vector.

[0044] It should be understood that travel environment information usually includes various factors that affect truck routes, such as weather conditions, road conditions, traffic signals, road section characteristics, etc. This information often exists in an unstructured or semi-structured form, and may include text descriptions, sensor data, or other forms of data. The complexity and diversity of this information makes it difficult to directly analyze and process it. Therefore, a method is needed to convert this data into a structured feature representation that can be further analyzed. The core role of the travel environment information semantic encoder is to convert each element in the environmental information into a semantic feature vector by analyzing the context and semantic relationship of the text. Semantic encoders usually use advanced natural language processing technologies, such as deep learning models (such as BERT, GPT) and word embedding technologies, which can understand and capture complex semantics and contextual information between words. Specifically, the embedding layer of the travel environment information semantic encoder is used to convert the travel environment information of the operating truck route range into an embedding vector to obtain a sequence of travel environment information embedding vectors; the transformer-based Bert model of the travel environment information semantic encoder is used to perform global context semantic encoding on the sequence of travel environment information embedding vectors to obtain multiple travel environment information semantic feature vectors.

[0045] Furthermore, travel environment information usually includes multiple different dimensions, such as weather conditions, road conditions, traffic flow, etc. Each dimension is converted into a feature vector through a semantic encoder, and each of these vectors captures the semantic information of a specific type of environmental data. However, a single feature vector can often only provide a partial perspective on the environment, and cannot fully reflect the overall impact of the environment. The cascading operation is to splice these multiple feature vectors together to form a longer and more comprehensive feature vector that can comprehensively consider the combined effects of various environmental factors. Through cascading, the semantic features from different sources are sequentially integrated into a unified feature vector, which can ensure that the contribution of each feature is taken into account. This method can make full use of the information provided by each feature vector, bringing together their advantages in a richer representation, thereby obtaining a more comprehensive perspective of the environment.

[0046] In the above-mentioned operating vehicle travel risk management and control system 100, the travel congestion risk warning module 130 is used to determine whether to issue a travel congestion risk warning for operating trucks based on the multimodal feature vector of traffic congestion information and the semantic association feature vector of travel environment information. It should be understood that the multimodal feature vector of traffic congestion information usually contains information from different data sources, such as real-time traffic flow data, historical congestion records, sensor data, etc. These data integrate multiple aspects of traffic conditions, such as traffic density, traffic signal status, road usage, etc. through multimodal feature extraction technology. The information provided by each data source has its unique perspective and value. Through this comprehensive analysis, we can fully understand how traffic flow and environmental factors jointly affect traffic congestion risks. Based on this comprehensive information, the model can use machine learning algorithms or statistical methods to perform pattern recognition and risk assessment to determine whether there is a congestion risk. Such an early warning mechanism can help operating trucks adjust routes in advance and choose to avoid congested areas, thereby reducing the risk of transportation delays and improving operational efficiency.

[0047] Figure 4 FIG. 1 is a block diagram of a travel congestion risk warning module in a travel risk management system for commercial vehicles according to an embodiment of the present application. Figure 4 As shown, in a specific embodiment of the present application, the travel congestion risk warning module 130 includes: an operating vehicle travel feature fusion unit 131, which is used to fuse the multimodal feature vector of the traffic congestion information and the semantic association feature vector of the travel environment information to obtain a congestion risk warning judgment feature vector; an operating vehicle travel feature optimization unit 132, which is used to perform nonlinear transformation compensation based on local disturbance on the congestion risk warning judgment feature vector to obtain an optimized congestion risk warning judgment feature vector; a travel congestion risk warning classification judgment unit 133, which is used to pass the optimized congestion risk warning judgment feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether to issue a travel congestion risk warning for operating trucks.

[0048] It should be understood that using traffic flow data or environmental data alone may not fully capture the complexity of traffic congestion. Traffic flow data can reflect the current traffic situation, but it may not take into account the interference of environmental factors; while environmental data can show possible external influences, but lacks a dynamic grasp of real-time traffic flow. By integrating these two features, real-time data of traffic flow can be combined with environmental influencing factors to provide a more comprehensive congestion risk assessment. Integrating information from multiple data sources into a unified feature vector can comprehensively consider the combined impact of traffic flow and environmental conditions on congestion, thereby providing a more accurate congestion risk prediction.

[0049] In particular, in the technical solution of the present application, in the process of integrating traffic monitoring features, road information features and travel environment information features to obtain a congestion risk warning judgment feature vector, if there are contradictions or inconsistencies between the features of these different data sources, noise may be introduced. This noise may be caused by inaccurate feature calculation, inconsistent data sources or the nonlinear relationship between features is not well modeled. Considering that the work of the classifier is based on the input congestion risk warning judgment feature vector, if the congestion risk warning judgment feature vector is unstable due to disturbances in the data processing process, the classifier will have insufficient classification accuracy. Therefore, in the technical solution of the present application, the congestion risk warning judgment feature vector is compensated for by nonlinear transformation based on local disturbances to obtain an optimized congestion risk warning judgment feature vector.

[0050] Among them, the congestion risk warning judgment feature vector is subjected to nonlinear transformation compensation based on local disturbance to obtain an optimized congestion risk warning judgment feature vector, including: determining a first congestion risk warning judgment weight matrix and a second congestion risk warning judgment weight matrix of the classifier before and after each iterative update; performing matrix multiplication of the first congestion risk warning judgment weight matrix and the second congestion risk warning judgment weight matrix with the congestion risk warning judgment feature vector respectively to obtain a first congestion risk warning judgment classification modulation feature vector and a second congestion risk warning judgment classification modulation feature vector; calculating the position difference between the first congestion risk warning judgment classification modulation feature vector and the second congestion risk warning judgment classification modulation feature vector to obtain a congestion risk warning judgment shift information representation vector ; Calculate the F norm of the position mean vector between the first congestion risk warning judgment classification modulation feature vector and the second congestion risk warning judgment classification modulation feature vector as the congestion risk warning judgment shift compensation scaling factor; linearly scale the congestion risk warning judgment shift information representation vector with the congestion risk warning judgment shift compensation scaling factor to obtain the scaled congestion risk warning judgment shift information representation vector, and input the scaled congestion risk warning judgment shift information representation vector into the Sigmoid activation function to obtain the congestion risk warning judgment backpropagation shift compensation representation vector; calculate the position point multiplication of the congestion risk warning judgment backpropagation shift compensation representation vector and the congestion risk warning judgment feature vector to obtain the optimized congestion risk warning judgment feature vector.

[0051] The optimization steps are specifically expressed as follows:

[0052]

[0053] Among them, V represents the congestion risk warning judgment feature vector, S represents the congestion risk warning judgment shift compensation scaling factor, M2 represents the second congestion risk warning judgment weight matrix, M1 represents the first congestion risk warning judgment weight matrix, sigmoid represents the activation function, ⊙ represents the position point multiplication, represents matrix multiplication, represents the difference by position, means adding by position, ||·|| F represents the Frobenius norm, and V' represents the optimized congestion risk warning judgment feature vector.

[0054] In the technical solution of the present application, in order to enhance the robustness of the classifier to local disturbances in the input feature vector, the congestion risk warning judgment feature vector is compensated by nonlinear transformation based on local disturbances. Among them, the key to the nonlinear transformation compensation based on local disturbances for the congestion risk warning judgment feature vector is to capture the information of the weight matrix changing with the training process, and use this information to generate the congestion risk warning judgment displacement information representation vector, and then generate the congestion risk warning judgment back propagation displacement compensation representation vector through scaling and nonlinear transformation (Sigmoid function). In this way, the congestion risk warning judgment feature vector is supported by the distribution difference of different classification scenarios in the training process of the classifier to describe the local displacement of the feature vector, so that it can not only quantify the displacement of the feature vector caused by the change of the weight matrix, but also generate a compensation vector according to the displacement information. The role of the compensation vector is to weight and adjust each element in the feature vector according to the degree of influence of the disturbance, thereby reducing the influence of the disturbance on the classification result. In this way, the classifier's robustness to local disturbances in the input data can be significantly improved, the classifier's stability and generalization ability can be enhanced, and by reducing the impact of local disturbances on the classification results, the classifier can still maintain a high classification accuracy when facing noisy or variant data.

[0055] Furthermore, the optimized congestion risk warning judgment feature vectors fuse traffic flow data and travel environment information, and contain comprehensive information about traffic status and environmental conditions. These feature vectors provide a comprehensive view of the current traffic situation and its potential risks. However, these feature vectors are usually high-dimensional, and it may be very difficult to directly extract useful information from these complex data. Therefore, a mechanism is needed to transform these high-dimensional feature information into a form that is easy to understand and operate. The classifier, as a machine learning model, can map complex optimized congestion risk warning judgment feature vectors to specific category labels. In this case, the task of the classifier is to classify the fused optimized congestion risk warning judgment feature vectors into different risk levels, such as "issue a warning" or "no warning required". Through this classification, the classifier is able to transform complex traffic data into clear and intuitive decision recommendations. The classifier is able to process and simplify high-dimensional feature data and transform it into easy-to-interpret category labels. This transformation allows the traffic management system to quickly understand the current traffic situation and potential congestion risks. By classifying the optimized congestion risk warning judgment feature vectors, the classifier can provide real-time risk assessment results. This enables traffic management systems to quickly identify potential high-risk areas and issue timely warnings so that operating trucks can adjust their travel plans to avoid congested areas.

[0056] In summary, the embodiment of the present application first obtains traffic monitoring videos of various areas within the operating truck route range collected by the camera, traffic road information within the operating truck route range collected by the database, and travel environment information within the operating truck route range, and then uses deep learning technology to perform feature extraction and correlation analysis on the three. Finally, a classifier is used to obtain classification results to determine whether to issue a congestion risk warning for operating truck travel, thereby helping drivers and operations managers to take action, reduce uncertainty in the journey, and improve transportation efficiency and punctuality.

[0057] As described above, the operating vehicle travel risk management system 100 according to the embodiment of the present application can be implemented in various terminal devices. In one example, the operating vehicle travel risk management system 100 can be integrated into the terminal device as a software module and / or a hardware module. For example, the operating vehicle travel risk management system 100 can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the operating vehicle travel risk management system 100 can also be one of the many hardware modules of the terminal device.

[0058] Alternatively, in another example, the commercial vehicle travel risk management system 100 and the terminal device may also be separate devices, and the commercial vehicle travel risk management system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0059] Figure 5 Flow chart of the method for controlling the travel risk of commercial vehicles according to an embodiment of the present application. Figure 5 As shown, according to the method for controlling the travel risk of operating vehicles in an embodiment of the present application, it includes: S110, acquiring traffic monitoring videos of each area within the route range of an operating truck collected by a camera, traffic road information within the route range of an operating truck collected by a database, and travel environment information within the route range of an operating truck; S120, extracting a multimodal feature vector of traffic congestion information and a semantic association feature vector of travel environment information from the traffic monitoring videos of each area within the route range of an operating truck collected by a camera, the traffic road information within the route range of an operating truck collected by a database, and the travel environment information within the route range of an operating truck; S130, judging whether to issue a travel congestion risk warning for an operating truck based on the multimodal feature vector of traffic congestion information and the semantic association feature vector of travel environment information.

[0060] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned operating vehicle travel risk management method have been referred to above. Figures 1 to 4 The above description has been introduced in detail in the description of the operating vehicle travel risk management system, and therefore, its repeated description will be omitted.

[0061] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0062] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0063] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0064] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0065] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0066] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit of the technical solution of the present invention.

Claims

1. A travel risk management system for commercial vehicles. It is characterized in that include: The operating vehicle travel data acquisition module is used to obtain the traffic monitoring video of each area of ​​the operating truck route range collected by the camera, the traffic road information of the operating truck route range collected by the database, and the travel environment information of the operating truck route range; An operating vehicle travel data extraction module is used to extract a multimodal feature vector of traffic congestion information and a semantic association feature vector of travel environment information from the traffic monitoring videos of each area of ​​the operating truck route range collected by the camera, the traffic road information of the operating truck route range collected by the database, and the travel environment information of the operating truck route range; The travel congestion risk warning module is used to determine whether to issue a travel congestion risk warning for operating trucks based on the multimodal feature vector of the traffic congestion information and the semantic association feature vector of the travel environment information.

2. The operating vehicle travel risk management and control system according to claim 1, It is characterized in that The operating vehicle travel data extraction module includes: A traffic congestion information feature extraction unit, used for extracting features from the traffic monitoring videos of each area of ​​the operating truck route range collected by the camera and the traffic road information of the operating truck route range collected by the database to obtain the multimodal feature vector of the traffic congestion information; The travel environment information feature extraction unit is used to extract features from the travel environment information within the route range of the operating truck to obtain a semantically associated feature vector of the travel environment information.

3. The operating vehicle travel risk management and control system according to claim 2, It is characterized in that The traffic congestion information feature extraction unit comprises: A traffic monitoring feature encoding subunit, used for performing feature encoding on the traffic monitoring video of each area of ​​the operating truck route range collected by the camera to obtain a regional traffic monitoring understanding feature vector; A traffic road information feature encoding subunit, used for performing feature encoding on the traffic road information of the operating truck route range collected from the database to obtain a plurality of traffic road information understanding feature vectors; The traffic congestion information feature association subunit is used to associate the regional traffic monitoring understanding feature vector with the multiple traffic road information understanding feature vectors to obtain the traffic congestion information multimodal feature vector.

4. The operating vehicle travel risk management and control system according to claim 3, It is characterized in that The traffic monitoring feature encoding subunit comprises: Extracting a plurality of regional traffic monitoring key frames from the traffic monitoring videos of various regions within the operating truck route range collected by the camera; Arranging the plurality of regional traffic monitoring key frames into a regional traffic monitoring input tensor; The regional traffic monitoring input tensor is passed through a regional traffic monitoring feature encoder based on a convolutional neural network to obtain the regional traffic monitoring understanding feature vector.

5. The operating vehicle travel risk management and control system according to claim 4, It is characterized in that The traffic road information feature encoding subunit includes: Segmenting the traffic road information of the operating truck route range collected from the database to obtain a traffic road information word sequence; The traffic road information word sequence is passed through a traffic road information text context semantic encoder to obtain the multiple traffic road information understanding feature vectors.

6. The operating vehicle travel risk management and control system according to claim 5, It is characterized in that The travel environment information feature extraction unit includes: Passing the travel environment information of the operating truck route range through a travel environment information semantic encoder to obtain a plurality of travel environment information semantic feature vectors; The multiple travel environment information semantic feature vectors are cascaded into the travel environment information semantic association feature vector.

7. The operating vehicle travel risk management and control system according to claim 6, It is characterized in that The travel congestion risk warning module includes: An operating vehicle travel feature fusion unit, used to fuse the traffic congestion information multimodal feature vector and the travel environment information semantic association feature vector to obtain a congestion risk warning judgment feature vector; An operating vehicle travel characteristic optimization unit, used for performing nonlinear transformation compensation based on local disturbance on the congestion risk warning judgment characteristic vector to obtain an optimized congestion risk warning judgment characteristic vector; The travel congestion risk warning classification judgment unit is used to pass the optimized congestion risk warning judgment feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether to issue a travel congestion risk warning for operating trucks.

8. The operating vehicle travel risk management and control system according to claim 7, It is characterized in that The operating vehicle travel characteristics optimization unit comprises: Determine a first congestion risk warning judgment weight matrix and a second congestion risk warning judgment weight matrix of the classifier before and after each iterative update; Performing matrix multiplication of the first congestion risk warning judgment weight matrix and the second congestion risk warning judgment weight matrix with the congestion risk warning judgment feature vector respectively to obtain a first congestion risk warning judgment classification modulation feature vector and a second congestion risk warning judgment classification modulation feature vector; Calculating the position difference between the first congestion risk warning judgment classification modulation feature vector and the second congestion risk warning judgment classification modulation feature vector to obtain a congestion risk warning judgment shift information representation vector; Calculating the F-norm of the position mean vector between the first congestion risk warning judgment classification modulation feature vector and the second congestion risk warning judgment classification modulation feature vector as a congestion risk warning judgment shift compensation scaling factor; Linearly scaling the congestion risk warning judgment shift information representation vector by the congestion risk warning judgment shift compensation scaling factor to obtain a scaled congestion risk warning judgment shift information representation vector, and inputting the scaled congestion risk warning judgment shift information representation vector into a Sigmoid activation function to obtain a congestion risk warning judgment back-propagation shift compensation representation vector; The optimized congestion risk warning judgment feature vector is obtained by calculating the point-by-point multiplication of the congestion risk warning judgment back-propagation shift compensation representation vector and the congestion risk warning judgment feature vector.

9. A method for controlling the travel risks of commercial vehicles. It is characterized in that include: Obtaining traffic monitoring videos of various areas within the operating truck route range collected by cameras, traffic road information within the operating truck route range collected by a database, and travel environment information within the operating truck route range; Extracting a multimodal feature vector of traffic congestion information and a semantic association feature vector of travel environment information from the traffic monitoring videos of each area of ​​the operating truck route range collected by the camera, the traffic road information of the operating truck route range collected by the database, and the travel environment information of the operating truck route range; Based on the multimodal feature vector of the traffic congestion information and the semantic association feature vector of the travel environment information, it is determined whether to issue a travel congestion risk warning for operating trucks.

10. The method for controlling the travel risk of commercial vehicles according to claim 9, It is characterized in that Extracting a multimodal feature vector of traffic congestion information and a semantic association feature vector of travel environment information from the traffic monitoring videos of each area of ​​the operating truck route range collected by the camera, the traffic road information of the operating truck route range collected by the database, and the travel environment information of the operating truck route range, including: Performing feature extraction on the traffic monitoring videos of each area of ​​the operating truck route range collected by the camera and the traffic road information of the operating truck route range collected by the database to obtain the multimodal feature vector of the traffic congestion information; Feature extraction is performed on the travel environment information of the operating truck route range to obtain a semantically associated feature vector of the travel environment information.

Images