Multimodal digital monitoring method for intelligent transportation management based on deep learning

Abstract

The present invention provides a multimodal digital monitoring method for intelligent transportation management based on deep learning, including the steps of using satellite positioning data to track the position and driving path of the target vehicle to form a multi-source data set; according to the obtained multi-source data set, setting a threshold to divide the violation of the target vehicle in the set into four levels: normal, warning, attention, and confirmed abnormal; and matching the violation of the target vehicle in the key monitoring area after classification with the monitoring factors and monitoring rules. The present invention proposes to collect trajectory data, driving path positioning data and combine map index construction technology to integrate the target vehicle's unresponsive records, trajectory abnormality type records and no-flow records in the operation, so as to form a comprehensive driving data set, conduct multi-directional monitoring, and reduce the impact on data integrity and reliability due to equipment failure, signal obstruction, technical limitations or malicious tampering.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic management, and in particular to a multimodal digital monitoring method for intelligent transportation management based on deep learning. Background Art

[0002] At present, in traffic management, when the illegal vehicles are unlicensed vehicles, vehicles with duplicate license plates, vehicles with fake license plates, etc., the traffic management personnel can only judge whether the vehicles are illegal by repeatedly watching surveillance videos or manually identifying illegal vehicles through inspections on the road, which consumes the time and energy of the traffic management personnel.

[0003] To address this problem, patent CN110148300A discloses a vehicle license plate duplicate recognition method, device and computer-readable medium, and proposes a vehicle license plate duplicate recognition technology. Specifically, a camera installed on the road by the transportation department is used to capture the target vehicle, and then the license plate number in the captured image is identified through existing or future text recognition technology. The image is compared with the vehicle information registered in the Internet of Vehicles, and the vehicle color, model and trajectory information are analyzed to determine whether the license plate is a duplicate, thereby automatically identifying the target vehicle's license plate number.

[0004] However, this technology relies too much on camera clarity and coverage, and has high requirements for real-time updating and accuracy of vehicle information. For example, the accuracy of image recognition technology and its robustness under different lighting and weather conditions are also technical challenges faced by this technology.

[0005] Patent CN112802344A discloses a vehicle-mounted intelligent networked real-time traffic violation monitoring device and system. The urban road monitoring network can only realize fixed-point monitoring at present, lacks intelligent monitoring of the entire road section, and requires traffic police to take photos on the spot to collect evidence before punishment can be imposed. There are problems such as the difficulty of collecting evidence and the subjectivity of punishment. It is proposed to combine the violation recognition system with the driving recorder to realize automatic and real-time capture of traffic violations, and complete the recording and analysis of the violations of other vehicles and the vehicle itself during driving on its own vehicle, provide evidence for traffic violations and accidents, and at the same time be able to monitor road traffic conditions in real time, reduce traffic violations and accidents, and realize more accurate intelligent transportation.

[0006] However, this technology has the problem of a single data evidence source, that is, it over-relies on the information collected in real time by on-board equipment. If it is due to equipment failure, obstruction, technical limitations or malicious tampering, it will directly affect the integrity and reliability of the data. Therefore, in order to solve the above problems, the present invention proposes a multi-modal digital monitoring method for intelligent transportation management based on deep learning to provide an optimal solution to this industry defect. Summary of the invention

[0007] In view of the shortcomings of the prior art, the present invention aims to provide a multimodal digital monitoring method for intelligent transportation management based on deep learning, which collects trajectory data, driving path positioning data and combines map index construction technology to form multi-source data to ensure accurate monitoring and rapid response to abnormal vehicle behavior. Solve the problems raised in the above background technology.

[0008] In order to achieve the above object, the present invention is implemented through the following technical solutions: a multimodal digital monitoring method for intelligent transportation management based on deep learning, comprising:

[0009] Track the target vehicle in the key monitoring area, capture satellite signals through the vehicle-mounted positioning equipment, complete the real-time trajectory data collection and driving path positioning of the vehicle, the trajectory data collection process is as follows: first, the trajectory data records that cannot be answered are classified as violations to form the S set; secondly, the real-time trajectory data is cleaned, the valid data is screened, the U set is constructed, and the number of abnormal types is counted; thirdly, the traffic consumption of the IoT card when the vehicle uploads the real-time trajectory data is identified, and the no traffic record in the abnormal trajectory time period is defined as the K set; finally, the S, U and K sets are integrated to form the multi-source data set X;

[0010] According to the acquired multi-source data set X, evidence sources are collected based on the evidence space defined by DS evidence theory, and the following operations are performed: first, the trust is assigned according to the basic probability and weight of each evidence source; second, a distance measurement formula is constructed to calculate the consistency between different evidence sources, and the violation of the target vehicle is comprehensively evaluated and graded by analyzing the measurement results; when there is inconsistency between different evidence sources, the Lagrange multiplier method is used to optimize the weight allocation to adjust the weight of the evidence source to minimize the conflict; third, the evidence sources are synthesized with the adjusted optimal weight to obtain multi-source evidence; finally, a threshold is set to divide the violation of the target vehicle in the set X into four levels: normal, warning, attention, and confirmed abnormal;

[0011] Construct a monitoring knowledge database, and perform feature extraction on key variable data that characterize the attributes of the violation to obtain a feature vector. Use the feature vector and monitoring knowledge data to train a deep learning model, analyze the correlation between existing cases, and extract and form monitoring factors and rules related to the violation based on the correlation results, so as to realize the monitoring of violations of different attributes in the key monitoring areas after classification.

[0012] Compared with the prior art, the present invention has the following beneficial effects:

[0013] 1. In order to solve the problem of excessive and single reliance on information collected in real time by on-board equipment in the existing process of monitoring vehicle violations, the present invention proposes to collect trajectory data, driving path positioning data and combine with map index construction technology to integrate the target vehicle's unresponsive records, trajectory abnormality type records and no-flow records in the operation, so as to form a comprehensive driving data set, conduct multi-directional monitoring, and reduce the impact on data integrity and reliability caused by equipment failure, signal obstruction, technical limitations or malicious tampering;

[0014] 2. The present invention also uses a clustering algorithm to classify the violations of target vehicles during real-time monitoring of road traffic conditions, efficiently identifies and prioritizes the most urgent and high-risk violations, ensures the timeliness of monitoring, and realizes more accurate intelligent transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The disclosure of the present invention is described with reference to the accompanying drawings. It should be understood that the drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In the drawings, the same reference numerals are used to refer to the same components. Among them:

[0016] Figure 1 A schematic diagram of a process of a multimodal digital monitoring method for intelligent transportation management based on deep learning proposed in one embodiment of the present invention;

[0017] Figure 2 A schematic diagram of the entire system flow of a multimodal digital monitoring method for intelligent transportation management based on deep learning proposed in one embodiment of the present invention;

[0018] Figure 3 This is a principle block diagram of a multimodal digital monitoring method for intelligent transportation management based on deep learning proposed in one embodiment of the present invention. DETAILED DESCRIPTION

[0019] It is easy to understand that according to the technical solution of the present invention, without changing the essential spirit of the present invention, a person skilled in the art can propose a variety of interchangeable structural modes and implementation modes. Therefore, the following specific implementation modes and drawings are only exemplary descriptions of the technical solution of the present invention, and should not be regarded as the whole of the present invention or as a limitation or restriction to the technical solution of the present invention.

[0020] The present invention is further described in detail below with reference to the accompanying drawings, but is not intended to limit the present invention.

[0021] As an understanding of the technical concept and implementation principle of the present invention, the multimodal digital monitoring method for intelligent transportation management based on deep learning proposed in the present invention is mainly aimed at solving the problems of excessive reliance on camera clarity and coverage, and single source of data evidence in the prior art.

[0022] Therefore, the present invention adopts a multi-dimensional data collection strategy: through the vehicle positioning device, the problem data of the target vehicle during driving is independently collected, including the track record of unresponsiveness, track interruption, straightening, constant speed problem record, and the traffic consumption record of the Internet of Things card, and this data record is formed into multiple independent evidence sources, which can corroborate each other to reduce the dependence on a single data source and improve the integrity and reliability of the data;

[0023] At the same time, in order to further improve the accuracy of monitoring, the present invention adopts map index construction technology, which is applied to vehicle path modeling to achieve secondary monitoring of vehicle driving paths and enhance the ability to identify abnormal driving behaviors.

[0024] like Figure 1-3 As shown, as an embodiment of the present invention, a multimodal digital monitoring method for intelligent transportation management based on deep learning is proposed, comprising the steps of:

[0025] S1. Using satellite positioning data to track the location and driving path of the target vehicle, and combining with map services, respectively complete the real-time trajectory data collection and driving path positioning of the vehicle. The specific implementation steps of real-time trajectory data collection include:

[0026] S100. Based on GPS or Beidou satellite positioning system or auxiliary positioning technology, positioning is performed through vehicle-mounted positioning equipment, such as GPS, Beidou receivers and Wi-Fi and cellular data modules when necessary. The receiver captures satellite signals and calculates pseudoranges, determines the latitude and longitude of the vehicle through the pseudoranges of at least three satellites, calculates the speed using time difference, records timestamps, and collects the location data of the target vehicle. In specific implementation, the above-mentioned location data at least includes timestamps, latitude and longitude coordinates, and speed data. After collection, these data are integrated and packaged and sent to the server, and the server stores and processes the data.

[0027] S101. Encode and verify the trajectory data of the vehicle-mounted positioning device that has been put online, classify the trajectory records that cannot be answered as violations, form an S set, record the number of violations, and record it as the S set, S = {S 1, S2...Sn}, where S1...Sn corresponds to the number of unanswered; at the same time, clean the real-time trajectory data, filter out valid data, and use the abnormal algorithm to handle trajectory interruption, straight line pulling, and constant speed problems, construct the U set, and count the number of abnormal types, U = {U 1, U2...Un}, where U1...Un corresponds to trajectory interruption, straight line pulling, and constant speed types, and n represents the number of related type abnormalities; then identify the traffic consumption of the Internet of Things card when the target vehicle uploads the real-time trajectory data, and define the K set as the one with no traffic records during the abnormal trajectory period, and the K set is from K1 to K N Composition, each K i Corresponding to a traffic data period, N represents the number of times the trajectory data is uploaded, denoted as, K = {K1, K2…K N}, where K1…KN correspond to the period without traffic data.

[0028] S102, perform multi-source data fusion, integrate the S set (unable to respond records), the U set (trajectory abnormality type records) and the K set (no flow records), and form a multi-source data set X, denoted as X = {S, U, K}.

[0029] In step S1, the specific implementation steps for locating the target vehicle's driving path include:

[0030] S110, selecting or calculating control points according to the aforementioned trajectory data, using a cubic Bezier curve to take the control points as input, and fitting a smooth curve for vehicle travel. It can be understood that the control points are key points on the trajectory of the target vehicle, including at least a starting point, an end point and / or a turning point and / or a speed change point and / or a trajectory interruption point and / or a lane change point; the specific implementation method is as follows:

[0031] Custom basic control points: starting point P0, indicating the position where the target vehicle starts to travel; first control point P1, indicating the first turning point or speed change point of the target vehicle; second control point P2, indicating the second turning point or speed change point of the target vehicle; target vehicle lane change point P3, indicating the position where the target vehicle changes lanes; trajectory interruption point P4, indicating the position where the trajectory data is temporarily missing when encoding and verifying the trajectory data of the on-line vehicle positioning device; end point P3!, indicating the first new point where the trajectory of the target vehicle is restored after the trajectory is interrupted;

[0032] Select the above four points and construct the cubic Bezier curve formula: (1), where B(t) represents the point on the curve at parameter t; is the number of combinations, expressed as the number of ways to select i elements from 3 different elements; P i It is represented as the i-th control point; t is the curve parameter, with a value of [0, 1];

[0033] Parameter adjustment: adjust the control point positions and / or add and / or delete control points to improve the curve's fit to the actual driving path; the process is as follows:

[0034] First, construct the least squares calculation formula: (2), where N is the number of the starting point and / or the first and second turning points or speed change points and / or lane change points and / or trajectory interruption points and the end point after trajectory recovery; S is the sum of square errors under the current control point; t j It is represented by the curve parameter t corresponding to the jth control point; P j It is represented as the position of the jth control point on the actual driving path of the target vehicle; B(t j ) is represented as a cubic Bezier curve at t j The predicted position of

[0035] Secondly, by adjusting the coordinates of the basic control point, the gradient descent method is used to find the new position of the basic control point that minimizes the sum of squared errors, and the coordinates of the basic control point are updated;

[0036] Finally, repeat the above steps until the stopping condition is met;

[0037] Curve evaluation: Evaluate the accuracy and smoothness of the fitted curve to ensure that the curve can reasonably reflect the vehicle's driving path, and apply the fitted Bezier curve to vehicle path modeling for subsequent analysis or visualization.

[0038] In one embodiment of the present invention, after applying the fitted Bezier curve to vehicle path modeling, the present invention further needs to perform the following steps:

[0039] S111, using a map indexing method to divide the space where the trajectory data is located, the specific implementation method is as follows:

[0040] Based on the vehicle driving trajectory data collected in S110 including the starting point and / or the turning point and / or the speed change point and / or the trajectory interruption point and / or the lane change point and the end point, a cubic Bezier curve segment representing the trajectory data under the current curve parameter t is defined:

[0041] Let P 起点 =(x0,y0)、P 第一控制点 =(x1,y1) 、P 第二控制点 =(x2,y2)、P 目标车辆车道变化点=(x3,y3)、P 轨迹中断点 =(x4,y4) 、P 终点 =(x 00 ,y 00 ); then, the cubic Bezier curve segment is (3),

[0042] In the spatial object definition stage, each vehicle trajectory path point P starting point, P first control point, P second control point, P target vehicle lane change point, P trajectory interruption point, and P end point are defined as specific points in the two-dimensional space. Entering the space division stage, the entire space is divided into multiple rectangular or quartered square areas. The process is as follows:

[0043] b1. First, calculate the minimum and maximum coordinate values ​​of the acquired trajectory path points P starting point, P first control point, P second control point, P target vehicle lane change point, P trajectory interruption point, and P end point to determine the boundary of the entire trajectory data: (4),

[0044] Where x0, x1, ..., x n and y0,y1,...,y n Respectively represent the horizontal and vertical coordinates of the trajectory path point coordinate values;

[0045] b2. Based on the density or area function of the trajectory path points, the irregular grid area is divided into rectangular spaces, and the regular grid area is divided into four equal parts. The four-equal square division is selected, starting from the outermost layer, the entire area is divided into four equal parts, and then each square is divided into four equal parts until the preset specific conditions are met, such as the size of each area or the number of trajectory points contained in it is moderate;

[0046] Based on the coordinates of the region divided into multiple rectangles or four-divided squares, each spatial object is assigned to the corresponding region, and a location index is created for fast retrieval. The process is as follows:

[0047] c1. Set the latitude and longitude or UTM coordinates of the entire space division to Space point (x) and Space point (y), recorded as the starting point coordinates, then determine the latitude and longitude or UTM coordinates of each trajectory path point as trajectory point (x) and trajectory point (y), calculate the position of the trajectory path point relative to the coordinates of the starting point:

[0048] position x = trajectory point (x) - Spacepoint (x)(5),

[0049] position y = trajectory point (y) - Space point (y)(6),

[0050] In the formula, position x and position y Respectively represent the position difference of the trajectory path point in the x-axis and y-axis directions relative to the starting point. Through the relative position mentioned above, the specific position of the trajectory path point in the space divided by multiple rectangular or quartered square areas can be further determined;

[0051] c2. Assign each spatial object to the corresponding area and create a location index (x, y) for fast retrieval: (7), (8),

[0052] In the formula, represents rounding down; R represents the grid area resolution;

[0053] In the query phase, the location index is used to respond to various spatial queries, such as locating points within a specific area or tracking the movement trajectory of points, thereby improving the efficiency and speed of retrieving the target vehicle's location and path information.

[0054] As an embodiment of the present invention, the steps are also proposed:

[0055] S2. Based on the acquired multi-source data set X, evidence sources are collected based on the evidence space defined by DS evidence theory to integrate and evaluate violation data from different data sources in the intelligent traffic management system. The specific implementation steps include:

[0056] S200, based on step S102, collect evidence about vehicle violations, i.e., S set (unable to answer records), U set (track abnormality type records) and K set (no flow records), and define the evidence space based on Dempster-Shafer evidence theory To represent the possible violation results of the target vehicle, denoted as ={normal, warning, attention, confirmed abnormal}; and define basic probability distribution functions for the S set (unable to respond records), the U set (trajectory abnormality type records) and the K set (no flow records), respectively, denoted as m s 、m u 、m K; and regard the S set, U set and K set as three independent evidence sources, assign trust according to their respective weights and violation record characteristics, and ensure normalization to meet the requirements of probability theory. The specific implementation method is as follows:

[0057] If there are n unanswered records in the S set, the trust level of the S set is assigned as "warning" or "confirmed abnormality":

[0058] m s (Warning) = ω S ×Ch S (warn),

[0059] m s (determine anomaly) = ω S ×Ch S (determine the abnormality),

[0060] In the formula, m s It is represented as the trust degree of set S; m s (Warning) is expressed as the confidence level of the “Warning” category based on the unanswerable records in the S set; ω S Expressed as the weight of the S set; Ch S It is expressed as the probability evaluated according to n unanswered records in the S set, where the evaluated probability Ch S Based on the proportion of target vehicles that were eventually confirmed to have violated regulations in similar situations in the past; m s (Determined anomaly) is expressed as the confidence of the “determined anomaly” category based on the S set;

[0061] If the set U records m different types of trajectory anomalies, then each type of trajectory anomaly has its corresponding occurrence probability Ch U , the trust distribution for set U is: m u (Exception type) = ω u ×Ch u (abnormal type); where m u It is represented as the trust degree of set U; m u (Anomaly type) is represented by the confidence obtained based on the trajectory anomaly type recorded in the U set;

[0062] If the set U records m different types of trajectory anomalies, the probability of each anomaly type is Ch U It is determined based on how often these types of anomalies have led to violations in the past;

[0063] If there are p no-flow situations recorded in the K set, then each no-flow situation has a corresponding probability Ch that the vehicle does not travel according to the normal flow pattern. K , the trust distribution for the K set is: mK (no flow) = ω k ×Ch k (no flow condition),ω k It is expressed as the weight of the K set, which is determined based on the frequency of no-flow records.

[0064] S201. Constructing distance measurement formula (9),

[0065] Compute the differences between different evidence sets, i.e., evidence sources, and assess consistency by measuring the proportion of common and opposing parts between sets, where Represented as two sources of evidence and The evidence distance between them; D(A,B) is a scalar value used to measure the similarity or difference between two data sets, representing the distance measure between data set A and data set B, where ; Represented as the same evidence space Next, the source of evidence and For the evidence space The basic probability distribution function of data set A and data set B in ;

[0066] S202. It can be understood that by analyzing the above distance measurement results, the consistency or conflict between the evidence sets is determined, so as to comprehensively evaluate and grade the vehicle violations. The smaller the distance, the higher the consistency between the evidence sets.

[0067] Based on the above technical concept, it should be noted that during the execution of steps S200 to S202, using the distance measurement formula, it can be found that the smaller the distance, the higher the consistency between the evidence sets. On the contrary, this means that if the distance is large, there is a significant inconsistency or conflict between the evidence sets, and the evidence is considered to be defective, that is, the inconsistency between the evidence leads to uncertainty or error in the assessment of the violation. Since during the execution of steps S100 to S111, the evidence of the violation obtained comes from multiple different data sources, based on different data sources and standards, it is suspected that contradictory information is provided. Based on this, it is also necessary to adjust the weight of the evidence, that is, by minimizing the distance or conflict between different evidence sets, and adjusting the weights of different evidence sets. The specific implementation steps are:

[0068] S203, define a distance function d that represents the distance between different evidence sets, which is used to measure the degree of conflict between different evidence sets: , , (10),

[0069] The objective function is constructed to minimize the sum of the distances between all evidence sets so that the contribution of all evidence sets to the final decision is consistent: (11),

[0070] ω S +ω U +ω k =1,ω S ,ω U ,ω k ≥0,

[0071] The Lagrange multiplier λ is introduced to handle the constraints and the Lagrange function L is constructed to simplify the solution process: (12),

[0072] In the formula, ω1+ω2+...+ω n-1 =0, indicating a constraint condition;

[0073] Calculate the weight ω of the Lagrangian function L on each evidence set S, U and K S ,ω U ,ω k Taking the partial derivative of and setting it equal to 0, we get the Lagrange multiplier λ: ,

[0074] Construct a system of equations and solve for the optimal weight ω S ,ω U ,ω k : (16),

[0075] Similarly, as in the above formula, by constructing ω U， ω k system of equations, and using the constraint ω S +ω U +ω k =1,

[0076] Solving for the optimal weight ω S ,ω U ,ω k .

[0077] Based on the above S203 concept, the accuracy and fairness of the assessment of violations are improved through weight adjustment and optimization. The following steps need to be continued:

[0078] S204, combined with optimal weight ω S ,ω U ,ω k , respectively for the basic probability distribution function m of the S set (unable to answer records), the U set (trajectory abnormality type records) and the K set (no flow records) s 、m u、m K The synthesis is performed, and at the same time, according to the synthesized basic probability distribution function, a threshold is set to grade the target vehicle's violation behavior, and the grading result is output as an evaluation support for the vehicle's violation behavior.

[0079] Based on the above technical concept, it can be understood that the principle of this classification is: initially define the violation severity scoring standard of each element in the S, U, and K sets, then calculate the total score of each set, that is, the sum of the scores of all elements in the set, and then assign different weights to the total scores of the S, U, and K sets to reflect their importance in the classification, and sum up the weighted scores of each set, weighted score = total score multiplied by weight, to obtain the weighted total score of each target vehicle, and finally, compare the weighted total score with the preset threshold value, and classify the vehicle violation into four levels: normal, warning, attention, or determined abnormal. It should be noted that in the implementation of the present invention, In the example, the preset threshold is a predefined numerical standard or parameter used to judge and identify abnormal situations in the data, including at least a trajectory interruption threshold and a speed threshold, wherein the trajectory interruption threshold is to determine the maximum acceptable time or distance interval between trajectory points. If this interval is exceeded, the system will consider that the trajectory has been interrupted; the speed threshold is a reasonable upper and lower limit of the set target vehicle speed. If the vehicle speed is detected to exceed the upper limit or fall below the lower limit, it is marked as an abnormal speed. At this time, combined with the driving path positioning, if inconsistencies are found, the data quality is first evaluated to confirm whether the data collection is incorrect or whether the equipment has failed, and the inconsistent data is manually reviewed.

[0080] In specific implementation, it is preferred to use three thresholds to classify the target vehicle's violations:

[0081] Set the first level threshold TV1=0.3;

[0082] The second level threshold TV2=0.6;

[0083] The third level threshold TV3=0.8; where

[0084] The first level threshold TV1 is used to distinguish between normal and warning among the possible violation results of the target vehicle;

[0085] The second level threshold TV2 is used to distinguish between warning and attention in the possible violation results of the target vehicle;

[0086] The third threshold TV3 is used to determine whether the target vehicle is abnormal. Based on step S200, the violation results that have been obtained are known. ={normal, warning, attention, confirmed abnormal}, and there are m normal, m warning, m attention, and m confirmed abnormal basic probability distribution functions corresponding to the S set (unable to answer records), U set (trajectory abnormality type records), and K set (no flow records), respectively. At this time:

[0087] When mNormal>TV1 and mNormal≥mWarning and mNormal≥mAttention and mNormal≥mConfirmed Abnormal, the target vehicle's violation behavior is classified as normal;

[0088] When mWarning>TV1 and mWarning≥mNormal and mWarning≥mAttention and mWarning≥mConfirmed Abnormal, the target vehicle's violation behavior is classified as Warning;

[0089] When mAttention>TV2 and mAttention≥mNormal and mAttention≥mWarning and mAttention≥mConfirmed Abnormal, the target vehicle violation is classified as Attention;

[0090] When m determined abnormality > TV3 and m determined abnormality ≥ m normal and m determined abnormality ≥ m warning and m determined abnormality ≥ m attention, the violation of the target vehicle is classified as abnormal. Otherwise, when any of the above conditions is not met, the management personnel is required to further investigate or manually review. At the same time, the set three-level threshold is adjusted according to the actual implementation.

[0091] It is also necessary to implement intelligent monitoring based on the following step S3, and the implementation steps include:

[0092] S3-1. Construct a monitoring knowledge database by coding existing laws and regulations and historical administrative cases. law , and obtain the complete violation data to form the violation data set D illegal， , where V i The identifier or violation type name representing the occurrence of the i-th violation; a i Expressed as with the violation V i Related historical attributes of the actor, such as historical violation records; i Expressed as with the violation V i The associated contextual information of the time and location of occurrence; m represents the total number of violations identified;

[0093] S3-2. Feature extraction is performed on the key variable data representing the attributes of the violation behavior, and the feature vector Fvi, ti is constructed, and each data point in the feature vector Fvi, ti is normalized. It can be understood that the violation behavior attribute refers to the historical attribute of the actor associated with the violation behavior, such as historical violation records, and the key variable data representing the attribute of the violation behavior refers to the identifier or violation type name representing a certain violation behavior, such as speeding.

[0094] S3-3, using the extracted feature vectors Fvi, ti and the monitoring knowledge database D law As training data, the deep learning model MLDeep is trained, and the forward propagation function f of the model MLDeep is used to generate the prediction output of the correlation between regulations and cases: ^y = f(Fvi,ti,θMLDeep), where ^y represents the prediction result of the deep learning model MLDeep, θ MLDeep Represents the model parameters, and f represents the model forward propagation function;

[0095] In one embodiment of the present invention, a cross entropy loss function Loss is used to train a deep learning model MLDeep to optimize model parameters through a back propagation algorithm.

[0096] S3-4. Use the softmax activation function as the output layer of the MLDeep model to ensure that the matching result between the output regulations and cases is a legal probability distribution: , where P(yi = j) represents the normalized probability of category j, z j denoted as the original prediction output of the model network for category j, k is denoted as the total number of categories, and in the embodiment of the present invention, category j refers to the classification of illegal behaviors associated with specific monitoring factors and rules;

[0097] S3-5. Select the regulation category with the highest probability as the regulation most relevant to the case, and map it into a query data table F = {fij= (vi, pj, sj)|i = 1, 2, ..., m, j = 1, 2, ..., nj}, which is the violation data set D illegal Different types of violations in V i Match the corresponding monitoring factor P j and monitoring rules S j , to achieve the query matching of the violation behavior of the target vehicle in the key monitoring area after classification with the monitoring factors and monitoring rules. In the formula, fij represents the violation behavior V i and the corresponding monitoring factor P j , corresponding monitoring rules S j Combination ofj Expressed as with the violation V i The number of associated monitoring factor and monitoring rule combinations.

[0098] In one embodiment of the present invention, before matching the violation behavior of the target vehicle in the key monitoring area after the classification with the monitoring factor and the monitoring rule, it is also necessary to perform the following steps based on step S4:

[0099] Use clustering algorithm to classify the violation data set D illegal Different types of violations in V i Perform cluster division, label each cluster to identify its common characteristics, build and verify cluster queues to optimize monitoring resource allocation, in order to ensure the timeliness and effectiveness of monitoring. The specific implementation steps include:

[0100] S400, for illegal behavior set D illegal Each violation in V i , construct at least one V that contains both evaluation and quantification of violations i The system P of multi-dimensional evaluation indicators of characteristics and their impact, and the set D of violations illegal Perform cluster analysis to group violations with similar characteristics into i are classified into the same gray class, denoted as C = {C1, C2, . . . , CM}, where CM represents the violation V obtained by cluster analysis i A classification or gray class is used to describe the classification of illegal behaviors obtained through cluster analysis. It can be understood that in the embodiment of the present invention, CM and Cm are conceptually the same and are used to describe the same data structure, that is, to represent the classification of illegal behaviors obtained through cluster analysis. Each gray class Cm represents a specific illegal behavior V i The gray class set Cm is defined by the actual value Aij: where i ∈ {1, 2, ..., n}, j ∈ {1, 2, ..., ρ}, Aij represents the actual value of the jth evaluation index of the ith violation unit; n represents the number of schemes in the multidimensional evaluation index system. The purpose of this step is to facilitate classification evaluation and management. By establishing multidimensional evaluation indicators and preprocessing violation data, it is possible to ensure that the violation data set D illegal quality and availability, thereby increasing the risk of violations i At the same time, it can be understood that in the embodiment of the present invention, the number of proposed schemes refers to the number of different evaluation schemes or situations set for each multi-dimensional evaluation index, and each multi-dimensional evaluation index covers at least the violation V i Frequency, severity and scope of impact.

[0101] To further implement different violations V i Accurate management and effective prevention of violations i To understand and predict, we need to continue to implement:

[0102] S401, for the illegal behavior set D illegal Each multidimensional evaluation index ρ and each gray class Cm in the whitening function f pm (x) Quantify the degree to which each multi-dimensional evaluation index ρ belongs to different gray categories Cm, so as to classify each violation V i are correctly classified into the corresponding categories. It should be noted that ρ = 1, 2, . . . , P, represents the serial number of the multidimensional evaluation index, and CM = 1, 2, . . . , Cm, represents the serial number of the gray category Cm.

[0103] S402, based on the whitening function f pm (x) and the weight of the multi-dimensional evaluation index ρ in the preset gray category Cm , the clustering coefficient b that quantifies the degree to which each multidimensional evaluation index ρ belongs to different gray categories Cm is obtained to form a clustering vector. ypm is expressed as the actual value Aij corresponding to the whitening function value of the multidimensional evaluation index ρ when it is 1. The clustering coefficient b is calculated as follows:

[0104] First, let R be the ρ*m-dimensional whitening function f pm (x) Outputs the matrix when converted to a specific function value, where the element r in the matrix R pm =f pm (x), expressed as the whitening function value of the ρth multidimensional evaluation index in the mth gray class Cm; at the same time, let ω m is the weight vector of the mth gray class Cm, where the weight vector ω m By at least one element ω ρm composition;

[0105] Secondly, calculate the clustering coefficient b: B = W*R, where B is represented by an m-dimensional clustering vector, consisting of at least one element bm, bm represents the clustering coefficient of the m-th gray class Cm; W is represented by a ρ* m-dimensional weight matrix, containing the weights of all indicators of the m-th gray class Cm;

[0106] Finally, through matrix multiplication, each row of W and R is multiplied with each column of R and the sum is calculated, and each element in B is obtained as the clustering coefficient b of each gray class Cm.

[0107] S403: According to the obtained clustering coefficient b, the illegal behavior data set D illegal Each violation in V i Assign to the most relevant gray category Cm, implement data clustering, and conduct comprehensive evaluation and sorting. Based on this evaluation, determine the optimal solution under the current data system, and find the solution set corresponding to the optimal solution. It can be understood that the solution set proposed by the present invention is the result obtained by processing and analyzing the violation data set, and the violation data set provides the original data, which is the starting point for the formation of the solution set. These data are analyzed by a multi-dimensional evaluation index system to form a solution set. It should also be noted that this evaluation process involves the evaluation of the violation V in each gray category Cm. i quantify the characteristics and impact of violations, and evaluate the relative importance of different violations. For example, if the multi-dimensional evaluation index system in step S400 initially sets the violation V i The frequency, severity and scope of impact of violations are determined through the above steps. The clustering coefficient b of each violation is determined. Then, the violations can be sorted according to the clustering coefficients b to identify the violations with the lowest resource utilization rate and the violations that need to be processed first. When the optimal solution is determined, the set of all possible solutions corresponding to the solution can be further found. At this time, if the set contains only one solution, then this solution is regarded as the final optimal solution, directly output and processed first. For example, if the optimal solution set is based on the frequency and severity of the violations, and there is only one violation that has the lowest resource utilization rate in both indicators, then this violation will be marked as the top priority object. If the optimal solution set contains multiple solutions, the operator's preference parameters are required for further decision-making. In this case, return to step S400, re-construct the evaluation index system and pre-process the data according to the operator's preference to re-evaluate and determine the optimal solution, so as to optimize the monitoring strategy and resource allocation of intelligent traffic management.

[0108] The technical scope of the present invention is not limited to the contents in the above description. Those skilled in the art can make various deformations and modifications to the above embodiments without departing from the technical idea of ​​the present invention, and these deformations and modifications should all fall within the protection scope of the present invention.

Claims

1. A multimodal digital monitoring method for intelligent transportation management based on deep learning, characterized by: Includes steps: Track the target vehicle in the key monitoring area, capture satellite signals through the vehicle-mounted positioning equipment, complete the real-time trajectory data collection and driving path positioning of the vehicle, the trajectory data collection process is as follows: first, classify the real-time trajectory data records that cannot be responded to as violations to form the S set; secondly, clean the real-time trajectory data, filter the valid data, construct the U set, and count the number of abnormal types; Secondly, identify the traffic consumption of the IoT card when the vehicle uploads real-time trajectory data, and define the K set as the period of time when there is no traffic record during the abnormal trajectory; finally, integrate the S, U and K sets to form the multi-source data set X; According to the acquired multi-source data set X, evidence sources are collected based on the evidence space defined by the DS evidence theory, which performs the following operations: first, assign trust according to the basic probability and weight of each of the evidence sources; second, construct a distance measurement formula to calculate the consistency between different evidence sources, and comprehensively evaluate and grade the violation of the target vehicle by analyzing the measurement results; When there is inconsistency between different evidence sources, the Lagrange multiplier method is used to optimize the weight distribution to adjust the weight of the evidence source to minimize the conflict; secondly, the evidence sources are synthesized with the adjusted optimal weight to obtain multi-source evidence; finally, the threshold is set to divide the violation behaviors of the target vehicles in the set X into four levels: normal, warning, attention, and confirmed abnormal; Construct a monitoring knowledge database, and perform feature extraction on key variable data that characterize the attributes of the violation to obtain a feature vector. Use the feature vector and monitoring knowledge data to train a deep learning model, analyze the correlation between existing cases, and extract and form monitoring factors and rules related to the violation based on the correlation results, so as to realize the monitoring of violations of different attributes in the key monitoring areas after classification.

2. The multimodal digital monitoring method for intelligent transportation management based on deep learning according to claim 1 is characterized by: The process of driving path positioning is as follows: First, the key points representing the target vehicle trajectory are used as basic control points to construct the cubic Bezier curve formula: , where B(t) represents the point on the curve at parameter t; is the number of combinations, expressed as the number of ways to select i elements from 3 different elements; P i It is represented as the i-th control point; t is the curve parameter, with a value of [0, 1]; Secondly, the positions of the basic control points are adjusted and / or basic control points are added and / or deleted to improve the fit of the curve to the actual driving path, which performs the following operations: Construct the least squares calculation formula: , where N is the number of basic control points obtained; S is the sum of square errors under the current control point; t j It is represented by the curve parameter t corresponding to the jth control point; P j It is represented as the position of the jth control point on the actual driving path of the target vehicle; B(t j ) is represented as a cubic Bezier curve at t j The predicted position at the location; by adjusting the coordinates of the basic control point, using the gradient descent method to find the new position of the basic control point that minimizes the sum of squared errors, and updating the coordinates of the basic control point; Repeat the above steps until the stop condition is met; Finally, the accuracy and smoothness of the fitting curve are evaluated to ensure that the curve can reasonably reflect the vehicle's driving path, and the fitted Bezier curve is applied to vehicle path modeling for subsequent analysis or visualization.

3. The multimodal digital monitoring method for intelligent transportation management based on deep learning according to claim 2 is characterized by: After the fitted Bezier curve is applied to vehicle path modeling, it is also necessary to divide the space where the trajectory data is located by constructing a map index to improve the query and retrieval efficiency of the trajectory data.

4. The multimodal digital monitoring method for intelligent transportation management based on deep learning according to claim 3 is characterized by: The process of building a map index is as follows: Based on the trajectory data of the target vehicle including the collected basic control points, a cubic Bezier curve segment is obtained to represent the trajectory path point under the current curve parameter t; In the spatial object definition stage, the trajectory path points are regarded as points in a two-dimensional space, and the entire space is further divided into a plurality of rectangular or quartered square regions, which performs the following operations: first, the minimum and maximum coordinate values ​​of the trajectory path points are calculated respectively to determine the boundaries of the trajectory data; then, according to the density or regional function of the trajectory path points, rectangles or quartered squares are selected for space division; After the space division is completed, each spatial object is assigned to the corresponding area, and a position index (x, y) is created for fast retrieval: First, the starting point coordinates of the entire space division are set, and then the position difference of each trajectory path point relative to the starting point coordinates is calculated, and the specific position of the trajectory path point in the divided space is determined by the relative position difference. Among them, the position index of each spatial object is determined by rounding down and the grid area resolution. The calculation formula is as follows: , , where represents rounding down; R represents the grid area resolution; position X and position y Respectively represent the position difference of the trajectory path point in the x-axis and y-axis directions relative to the starting point.

5. The multimodal digital monitoring method for intelligent transportation management based on deep learning according to claim 1 is characterized in that: The evidence space defined above includes possible violation results of the target vehicle, denoted as ={normal, warning, attention, confirmed abnormal}; The process of comprehensively evaluating and grading the target vehicle's violations by analyzing the results of the metrics is as follows: The following distance metric formula is constructed to calculate the differences between different evidence sources and evaluate consistency by measuring the proportion of common and opposite parts between the evidence sources: , where Represented as two sources of evidence and The evidence distance between them; D(A,B) is a scalar value used to measure the similarity or difference between two data sets, representing the distance measure between data set A and data set B, where ; Represented as the same evidence space Next, the source of evidence and For the evidence space The basic probability distribution function of data set A and data set B in ; By analyzing the measurement results of the above evidence distance, the consistency or conflict between the evidence sources is determined, so as to comprehensively evaluate and grade the violation of the target vehicle. The smaller the evidence distance, the higher the consistency between the evidence sources.

6. The multimodal digital monitoring method for intelligent transportation management based on deep learning according to claim 1 is characterized by: The S set, U set and K set are regarded as three independent evidence sources. When the trust is assigned according to the basic probability and weight of each evidence source, If there are n unanswered records in the S set, the confidence level of the S set is assigned as "warning" or "confirmed abnormality": m s (Warning) = ω S ×Ch S (warn), m s (determine anomaly) = ω S ×Ch S (determine the abnormality), In the formula, m s It is represented as the trust degree of set S; m s (Warning) is expressed as the confidence level of the "Warning" category based on the unanswerable records in the S set; ω S Expressed as the weight of the S set; Ch S It is expressed as the probability evaluated according to n unanswered records in the S set, where the evaluated probability Ch S Based on the proportion of target vehicles that were eventually confirmed to have violated regulations in similar situations in the past; m s (Determined anomaly) is expressed as the confidence of the "determined anomaly" category based on the S set; If the set U records m different types of trajectory anomalies, then each type of trajectory anomaly has its corresponding occurrence probability Ch U and U set weight ω U , the trust distribution for set U is: m u (Exception type) = ω u ×Ch u (Exception type), In the formula, m u It is represented as the trust degree of set U; m u (Anomaly type) is represented by the confidence obtained based on the trajectory anomaly type recorded in the U set; If the set U records m different types of trajectory anomalies, the probability of each anomaly type is Ch u It is determined based on how often these types of anomalies have led to violations in the past; If there are p no-flow situations recorded in the K set, then each no-flow situation has a corresponding probability Ch that the vehicle does not travel according to the normal flow pattern. K , the trust distribution for K sets is: m K (no flow) = ω k ×Ch k (no flow condition), In the formula, m K Represented as the trust of K set; m K (No-flow situation) is expressed as the trust degree obtained based on the no-flow situation recorded in the K set; ω k It is expressed as the weight of the K set, which is determined based on the frequency of no-flow records.

7. The multimodal digital monitoring method for intelligent transportation management based on deep learning according to claim 1 is characterized by: The process of adjusting the weights of evidence sources to minimize conflict is as follows: First, a distance function is defined to characterize the distance between different evidence sources and measure the degree of conflict between different evidence sources. Secondly, an objective function is constructed to minimize the sum of the distances between all evidence sources so that all evidence sources contribute to the final decision in the same way. At the same time, the Lagrangian multiplier λ is introduced to handle the constraints and the Lagrangian function L is constructed. After simplifying the solution process, the weights ω of the Lagrangian function L for each evidence source set S, set U and set K are calculated. S ,ω U ,ω k The partial derivative of and set it equal to 0, and get the Lagrange multiplier λ; Again, using the constraint ω S +ω U +ω k =1, solve the optimal weight ω in sequence S ,ω U ,ω k .

8. The multimodal digital monitoring method for intelligent transportation management based on deep learning according to claim 1 is characterized by: Before matching the violation behaviors of the target vehicles in the key monitoring area after classification with the monitoring factors and monitoring rules, it is also necessary to use a clustering algorithm to classify the violation behavior data set D illegal Different types of violations in V i Perform clustering and then label each cluster to identify their common characteristics. Build and validate cluster fleets to optimize monitoring resource allocation.

9. The multimodal digital monitoring method for intelligent transportation management based on deep learning according to claim 8 is characterized by: The process of optimizing monitoring resource allocation is as follows: First, the violation data set Each violation in V i , where a i Expressed as with the violation V i The associated historical attributes; i Expressed as with the violation V i The associated contextual information of the time and location of occurrence; m represents the total number of violations that have been identified; construct a model that includes evaluating and quantifying violations V i The multi-dimensional evaluation index system P of the characteristics and their impact is established, and based on this index system, the violations V with similar characteristics are clustered through cluster analysis. i are classified into the same gray category and recorded as , where CM represents the violation V obtained through cluster analysis i A classification or gray class is used to describe the classification of violations obtained through cluster analysis, and each gray class is defined by an actual value Aij, where i represents the number of solutions and j represents the multi-dimensional evaluation index; Secondly, use the whitening function f pm (x) quantifying the degree to which each multidimensional evaluation indicator belongs to different gray categories so as to correctly classify each violation into a corresponding category, and obtaining a clustering coefficient for quantifying the degree to which each multidimensional evaluation indicator belongs to different gray categories based on the whitening function and the weights of the multidimensional evaluation indicators in the preset gray categories; Thirdly, according to the obtained clustering coefficient, each violation in the violation behavior data set is assigned to the most relevant gray category, and a comprehensive evaluation and sorting of the violation behavior data after clustering is achieved to determine the optimal solution under the current data system, and find out the solution set corresponding to the optimal solution, which performs the following operations: Quantitative analysis of the characteristics and impact of violations in each gray category, as well as an assessment of the relative importance of different violations; Finally, if the optimal solution set contains only one solution, this solution is directly output and treated as a priority. Otherwise, further decisions are made based on the operator's preference parameters to re-evaluate and determine the optimal solution.

Images