A vehicle trajectory statistics method and query method based on large model

Abstract

The present invention provides a vehicle trajectory statistics method and query method based on a large model, and belongs to the field of vehicle trajectory target detection technology. The vehicle trajectory statistics method includes the following steps: collecting traffic vehicle videos, obtaining vehicle information features of each frame image through CAM; obtaining license plate number sequences, and forming new license plate information features through encoding; replacing the license plate information features of the original license plate information features into the Transformer network, and setting position codes for each input feature vector; calculating the vehicle information features to perform attention scores at different positions, establishing connections between vehicle information features of adjacent frames, and generating target trajectory videos. The vehicle trajectory query method includes: establishing a vehicle trajectory keyword information library; querying vehicle trajectory videos through first-level dangerous vocabulary reverse query supervision and second-level keyword optimization matching. The present invention improves the extraction accuracy of vehicle attribute information, reduces the problems of missed detection and false detection, and improves the pertinence and accuracy of queries.

Description

Technical Field

[0001] The invention relates to a vehicle track statistics method and a query method based on a large model, and belongs to the technical field of vehicle track target detection. Background Art

[0002] The current vehicle trajectory statistics technology faces multiple challenges in practical applications, especially in complex traffic scenarios, where its accuracy and flexibility are significantly affected. First, in terms of target detection and recognition, the existing vehicle trajectory statistics methods have certain errors. Complex and changeable traffic environments, such as dense traffic, changing lighting, occlusion, etc., often lead to missed detection and false detection problems, thus affecting the accuracy of trajectory statistics. These errors may not only come from the limitations of the sensor itself, but also be limited by data processing and analysis algorithms. Secondly, when facing traffic emergencies, the traditional vehicle driving query method mainly relies on specific time and vehicle ID. However, when errors occur in ID statistics, the reliability of vehicle trajectory statistics will be seriously weakened. In addition, this query method lacks flexibility and is difficult to meet diverse query needs. For example, users may want to query based on vehicle color, model or other attributes, but traditional query methods often cannot provide such functions. Furthermore, with the widespread application of large models in the field of picture and text question answering, their unexplainability has become a problem that needs to be solved urgently. In some cases, query systems based on large models will have interaction errors, resulting in deviations in query results. This bias may stem from the complexity and uncertainty within the model, making it difficult for users to understand and accept query results.

[0003] In summary, the existing vehicle trajectory statistics and query methods have deficiencies in accuracy, flexibility and interpretability. Therefore, it is necessary to develop a more efficient, accurate and flexible vehicle trajectory statistics and query method to cope with complex and changing traffic environments and diverse query needs. Summary of the invention

[0004] The purpose of the present invention is to provide a vehicle trajectory statistics method and query method based on a large model, to provide an efficient, accurate and flexible vehicle trajectory statistics and query method to cope with complex and changeable traffic environments and diverse query needs.

[0005] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0006] A vehicle trajectory statistics method based on a large model comprises the following steps:

[0007] Collect traffic vehicle videos and obtain vehicle information features of each frame through deep separable convolution CAM based on multi-attention mechanism CAHS, including vehicle presence features, vehicle attribute features and background features;

[0008] The vehicle information features are output through the LPRNet network to output the license plate number sequence, and the license plate number cutting characters in the license plate number sequence are one-hot encoded to form a new license plate information feature. ;

[0009] The vehicle features, vehicle attribute features other than license plate information, and background features are input into the Transformer network, and word embedding and license plate information features are performed through one-hot Perform concatenation and set position encoding for each input feature vector;

[0010] The query, key and value matrices obtained through linear transformation are used to calculate the vehicle information features for attention scores at different positions, establish the connection between the vehicle information features of adjacent frames, and generate the target trajectory video.

[0011] Preferably, the vehicle information features of each frame image are obtained by a depth-separable convolution CAM based on a multi-attention mechanism CAHS, specifically in the following manner:

[0012] Each frame of the traffic vehicle video is input into the deep separable convolution CAM based on the multi-attention mechanism CAHS, and is processed in sequence through 1x1 convolution, deep separable network DW layer, multi-attention mechanism CAHS and 1x1 convolution to output vehicle features, vehicle attribute features and background features.

[0013] Preferably, the multi-attention mechanism CAHS uses global maximum pooling and global average pooling to obtain local and global feature information, uses 1x1 convolution to replace the original fully connected layer, and uses hard-swish and hard-sigmoid as activation functions;

[0014] The feature processing process is as follows: the input original features are processed by global maximum pooling and global average pooling, and then by 1x1 convolution, hard-swish activation function, and 1x1 convolution in sequence. The output features are processed by global maximum pooling and global average pooling again, and after splicing, they are processed by hard-sigmoid activation function and output after multiplication with the original features.

[0015] Preferably, the position encoding formula is as follows:

[0016] ,

[0017] ,

[0018]

[0019] in, and is a learnable parameter, is the position dynamic correction parameter, for function, is the activation function, Represents the position index in the input sequence, used to encode each position, represents the dimension index of the feature vector, represents the length of the input feature, Indicates the location The position code.

[0020] Preferably, the query, key and value matrices are as follows:

[0021] ,

[0022] in, , , represents the query, key, and value matrices corresponding to the feature, represents the dimension of the key vector, Represents matrix transpose.

[0023] Preferably, the vehicle information features are calculated to obtain attention scores at different positions, and the connection between the vehicle information features of adjacent frames is established to generate the target trajectory video. The specific method is as follows:

[0024] The features are divided into two categories according to the presence or absence of vehicles, the vehicle trajectory connection is established, and the video frames are numbered and counted, as follows:

[0025] When there is no vehicle, the attention calculation is performed on the network input background features and the vehicle attribute features of the previous frame image to obtain a multi-head attention weight score; the weight score is used to determine whether it is greater than the vehicle feature threshold. If so, the current frame background features are added to the current frame vehicle attribute features; the multi-head attention weight score of the current frame vehicle attribute features and the vehicle attribute features of the previous frame image is calculated to determine the number of vehicle attribute features that meet the vehicle feature threshold. If the number of vehicle attribute features exceeds the set number threshold, the same vehicle exists in the current frame and the previous frame image, and a vehicle trajectory connection is established to generate a target trajectory video;

[0026] When there is a vehicle, the multi-head attention weight score of the vehicle attribute features of the current frame and the vehicle attribute features of the previous frame image is directly calculated to determine the number of vehicle attribute features that meet the vehicle feature threshold. If the number of vehicle attribute features sets the number threshold, the same vehicle exists in the current frame and the previous frame image, and the vehicle trajectory connection is established to generate the target trajectory video.

[0027] A vehicle trajectory query method, comprising:

[0028] The vehicle trajectory statistics method based on the large model is used to establish the connection between the vehicle information features of adjacent frames, and the text information of the current vehicle attribute features and the video frame number are used to establish a target-text pair with the vehicle to generate a vehicle trajectory keyword information library;

[0029] Collecting road system environment feature information of traffic vehicles, extracting road system environment feature information entries through a text encoder, establishing road system environment image-text pairs, obtaining intersection distance information and converting it into entries, and inputting the entries and road system environment image-text pairs into a vehicle trajectory keyword information database;

[0030] Input vehicle query statement information and perform preprocessing, extract statement terms to obtain query terms;

[0031] Preliminarily searching the danger dictionary according to the query term, if no term matching the query term is found, generating a candidate term list, using a similarity function to calculate the similarity between the query term and each word in the candidate term list, setting a threshold and comparing with the similarity function calculation result, if the similarity function calculation result is greater than or equal to the threshold, there is a term matching the query term, and the danger dictionary contains a non-traffic vehicle trajectory term;

[0032] If there is no match, perform inverse query verification. For each candidate term, calculate the inverse mapping similarity. According to the inverse mapping similarity calculation, output the terms whose inverse mapping similarity is greater than or equal to the inverse query threshold as the filtering result terms.

[0033] The filtered result terms are matched with secondary keywords and their matching scores are calculated. If the matching score is greater than or equal to the keyword matching threshold, the current term meets the search conditions. If there are four or more terms that meet the search conditions, the total score of the keyword terms is calculated.

[0034] If the total score of the keyword entries meets the set conditions, the corresponding keyword entries that are successfully matched are returned. According to the obtained successfully matched keyword entries, the corresponding vehicle trajectory video in the vehicle trajectory keyword information library is queried, and the video of the trajectory and the corresponding text description are output.

[0035] Preferably, the specific method of extracting the sentence terms to obtain the query terms is as follows:

[0036] Divide the preprocessed statement information into characters and fill the value array separately , calculate the minimum cost of each character , according to the calculated minimum cost from the array Backtrack to the optimal segmentation path and output the best word segmentation set; the minimum cost The calculation formula is as follows:

[0037] ,

[0038]

[0039] in, represents the cost function, Represents a substring, Indicates the character position currently being processed, and indicates the position of segmentation during word segmentation. Indicates the previous split point, Represents a security dictionary, containing entries for traffic vehicle trajectories, Indicates the substring length;

[0040] Backtrack to the optimal segmentation path from the array and output the best word segmentation set.

[0041] Preferably, the specific formulas of the similarity function and the inverse mapping similarity are as follows:

[0042] ,

[0043] ,

[0044] in, represents the similarity function, express The TF-IDF value of Indicates query terms, Indicates candidate terms Middle The value of a feature;

[0045] represents the inverse mapping similarity, and is the weight parameter, for The edit distance to the candidate term, Indicates the length of the query term. Indicates the length of the candidate term, Represents the dimension of the term.

[0046] Preferably, the total score of the keyword entries The calculation formula is as follows:

[0047] ,

[0048] ,

[0049] ,

[0050] in, Indicates the matching score of the license plate information, Indicates the matching score of the keyword entry. Indicates the total number of keywords. , and represents the weight, It means calculating the similarity between the keyword and the expected match;

[0051] Indicates that the word frequency is calculated by TF-IDF. represents the position weight, represents the characteristics of the calculation, Represents the target feature.

[0052] The advantages of the present invention are: By combining a lightweight convolutional network based on feature fusion with an optimized Transformer network, the present invention can more accurately extract vehicle trajectory information in the video. The lightweight feature extraction convolutional network improves the extraction accuracy of vehicle attribute information, and at the same time, combined with the multi-head attention mechanism of the Transformer network, it ensures the consistency of vehicle feature information and effectively reduces the problems of missed detection and false detection.

[0053] The present invention designs an information query system based on keyword entries, which significantly improves the pertinence and accuracy of the query through a two-level cascade matching algorithm. The first-level dangerous word reverse query can monitor whether the input sentence contains dangerous words, and effectively exclude potential mismatched entries through similarity calculation and reverse verification. The second-level keyword optimization matching further reduces the possibility of mismatching by calculating the weight score of each entry and judging whether it meets the query conditions.

[0054] With the processing power of the large model, the present invention can quickly respond to query requests and provide accurate vehicle trajectory information in a short time. At the same time, by establishing a vehicle trajectory keyword information library, the present invention can combine query results with relevant text descriptions to provide users with more intuitive and comprehensive information display, significantly improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0056] Figure 1 The figure is a flow chart of the vehicle trajectory statistics method based on a large model of the present invention.

[0057] Figure 2 It is a schematic diagram of the CAHS structure of the present invention.

[0058] Figure 3 It is a schematic diagram of the CAM structure of the present invention.

[0059] Figure 4 The figure is a flow chart of the vehicle trajectory query method of the present invention.

[0060] Figure 5 The figure is a flow chart of a vehicle trajectory query method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] Example 1

[0063] like Figure 1 As shown, a vehicle trajectory statistics method based on a large model includes the following steps:

[0064] S1: Collect traffic vehicle videos and obtain vehicle information features of each frame image through deep separable convolution CAM based on multi-attention mechanism CAHS, including vehicle features ( Indicates yes, Indicates none), vehicle attribute characteristics And background characteristics .

[0065] S2: Output the license plate number sequence through the LPRNet network, and perform one-hot encoding on the license plate number cut characters in the license plate number sequence to form a new license plate information feature. .

[0066] S3: Input the vehicle presence or absence features, vehicle attribute features other than license plate information, and background features into the Transformer network, and use one-hot word embedding and license plate information features Concatenate and set the position encoding for each input feature vector.

[0067] S4: The query, key and value matrices obtained by linear transformation are used to calculate the vehicle information features to perform attention scores at different positions, establish the connection between the vehicle information features of adjacent frames, and generate the target trajectory video.

[0068] As a refinement of the above embodiment, a lightweight convolutional network based on feature fusion is combined with Transformer as an image encoder to convert traffic vehicle videos into position-encoded feature data.

[0069] (1) If Figure 1As shown in the figure, the video data is first parsed, and convolutional features are extracted for each frame. Under the premise of ensuring the retention of feature information and reducing the computational complexity of the network, a deep separable convolution CAM (Channel Attention MobileNet V3) based on the multi-attention mechanism CAHS (Channel Attention based on the Hard Swish) is used. Figure 2 As shown in the figure, global maximum pooling and global average pooling are used to obtain local and global feature information; by using 1x1 convolution to replace the original fully connected layer, and using hard-swish and hard-sigmoid as activation functions, the amount of network calculation is reduced. The feature processing process is as follows Figure 3 As shown:

[0070] Each frame of the traffic vehicle video is dimensionalized by 1x1 convolution, and then input into the deep separable network DW layer for feature filtering to obtain deep information. The designed multi-attention mechanism CAHS is used to further obtain effective features. Finally, 1x1 convolution replaces the fully connected layer to output whether there are vehicle features. , , Vehicle attribute characteristics And background characteristics ,in For license plate information 。

[0071] Specifically, the multi-attention mechanism CAHS adopts global maximum pooling and global average pooling to obtain local and global feature information, uses 1x1 convolution to replace the original fully connected layer, and adopts hard-swish and hard-sigmoid as activation functions.

[0072] The feature processing process is as follows: the input original features are processed by global maximum pooling and global average pooling, and then by 1x1 convolution, hard-swish activation function, and 1x1 convolution in sequence. The output features are processed by global maximum pooling and global average pooling again, and after splicing, they are processed by hard-sigmoid activation function and output after multiplication with the original features.

[0073] (2) Since road vehicle videos are continuously collected at specific locations, in order to ensure the continuity of the data’s temporal information, the presence or absence of vehicle features, vehicle attribute features other than license plate information, background features, and new license plate information features are recorded. Input the Transformer network, and embed the vehicle features, vehicle attribute features other than license plate information, and background features through one-hot word embedding. , , as well as Set the position code to obtain the vehicle feature. and , vehicle attributes (including license plate information, vehicle color, vehicle model, vehicle local area, etc.) are ,in Represents license plate information, background features .

[0074] The position encoding formula is as follows:

[0075] ,

[0076] ,

[0077]

[0078] in, and is a learnable parameter, is the position dynamic correction parameter, for Function, introduce nonlinearity, enhance representation ability, is the activation function, , Indicates coordinate information. Represents the position index in the input sequence, used to encode each position, Represents the dimension index of the feature vector, because the positional encoding needs to generate different values ​​for each dimension, through and Distinguish between odd and even dimensions, represents the length of the input feature, Indicates the location The position code.

[0079] As a refinement of the above embodiment, the query, key and value matrix are as follows:

[0080] ,

[0081] in, , , represents the query, key, and value matrices corresponding to the feature, represents the dimension of the key vector, Represents the matrix transpose; through Perform vector dot product and softmax to obtain the score of the input word vector , get the output of self-attention at this position.

[0082] (3) By utilizing the Transformer’s position encoding and multi-head attention mechanism, we can distinguish the presence or absence of a vehicle, vehicle attribute information, and background with different position attention scores, and establish a connection between the presence or absence of a vehicle, vehicle attribute information, and background.

[0083] Specifically, the steps include:

[0084] The features are divided into two categories according to the presence or absence of vehicles, the vehicle trajectory connection is established, and the video frames are numbered and counted, as follows:

[0085] When there is no vehicle, the network input background features Perform attention calculation with the vehicle attribute features of the previous frame image to obtain the multi-head attention weight score ; Using multi-head attention weight score Determine whether it is greater than the vehicle feature threshold If yes, then add the background features of the current frame to the vehicle attribute features of the current frame; calculate the multi-head attention weight score of the vehicle attribute features of the current frame and the vehicle attribute features of the previous frame image , determine the number of vehicle attribute features that meet the vehicle feature threshold. If the number of vehicle attribute features exceeds the set number threshold, the same vehicle exists in the current frame and the previous frame image, establish a vehicle trajectory connection, and generate a target trajectory video.

[0086] When there is a vehicle, directly calculate the multi-head attention weight score of the vehicle attribute features of the current frame and the vehicle attribute features of the previous frame image , determine the number of vehicle attribute features that meet the vehicle feature threshold. If the number of vehicle attribute features exceeds the set number threshold, the same vehicle exists in the current frame and the previous frame image, establish a vehicle trajectory connection, and generate a target trajectory video.

[0087] Example 2

[0088] like Figure 4-Figure 5 As shown, a vehicle trajectory query method is provided. The vehicle trajectory query mainly enhances the question asked by a preset trajectory query large model to answer the corresponding target question. Specifically, the text information query is performed according to the set license plate information, vehicle color, vehicle model, text information of vehicle local attributes, video frame number and surrounding environment information. According to the term similarity calculation, the corresponding vehicle trajectory information video is obtained. In order to prevent the term matching from mismatching due to the instability of the generation model, a term information security matching strategy is established in advance. A two-level cascade matching algorithm is established, and a first-level dangerous word inverse query and a second-level keyword optimization matching are established respectively.

[0089] Specifically, they include:

[0090] S1: Establish vehicle trajectory keyword information database .

[0091] S2: Input vehicle query statement information and perform preprocessing, extract statement terms to obtain query terms.

[0092] S3: Perform a preliminary search in the danger dictionary based on the query term. If no term matching the query term is found, generate a candidate term list, use a similarity function to calculate the similarity between the query term and each word in the candidate term list, set a threshold and compare with the similarity function calculation result, if the similarity function calculation result is greater than or equal to the threshold, there is a term that matches the query term, and the danger dictionary contains a non-traffic vehicle trajectory term.

[0093] S4: If there is no match, perform inverse query verification. For each candidate term, calculate the inverse mapping similarity. According to the inverse mapping similarity calculation, output the terms whose inverse mapping similarity is greater than or equal to the inverse query threshold as the filtering result terms.

[0094] S5: Perform secondary keyword matching on the filtered result terms and calculate their matching scores. If the matching scores are greater than or equal to the keyword matching threshold, the current terms meet the search conditions. If there are four or more terms that meet the search conditions, calculate the total score of the keyword terms.

[0095] S6: If the total score of the keyword entries meets the set conditions, the corresponding keyword entries that are successfully matched are returned, and the corresponding vehicle trajectory video in the vehicle trajectory keyword information library is queried based on the obtained successfully matched keyword entries, and the video of the trajectory and the corresponding text description are output.

[0096] As a refinement of the above embodiment, a vehicle trajectory keyword information database is established. The steps include:

[0097] Traffic vehicle videos are collected and the vehicle information features of each frame are obtained through the deep separable convolution CAM based on the multi-attention mechanism CAHS.

[0098] The vehicle attribute features are output through the lightweight convolutional network layer to output the license plate position features, the license plate position features are output through the LPRNet network to output the license plate number sequence, and the license plate number cut characters in the license plate number sequence are one-hot encoded to form a new license plate information feature. .

[0099] Replace the vehicle information features in the vehicle information features with the new license plate information features , and input into the Transformer network, word embedding is performed through one-hot, and position encoding is set for each input feature vector.

[0100] The query, key and value matrices obtained by linear transformation are used to calculate the vehicle information features to perform attention scores at different positions, establish the connection between the vehicle information features of adjacent frames, and generate the target trajectory video.

[0101] The text information of the current vehicle attribute features and the video frame number are used to establish a target-text pair with the vehicle to generate a vehicle trajectory keyword information library .

[0102] Collect the road system environmental feature information of traffic vehicles, extract the road system environmental feature information entries through the text encoder, establish the road system environmental image-text pairs, obtain the intersection distance information and convert it into entries, and input the entries and road system environmental image-text pairs into the vehicle trajectory keyword information database .

[0103] As a refinement of the above embodiment, the first-level dangerous word reverse query mainly monitors whether the input sentence contains dangerous words, and pre-processes the input sentence to remove punctuation marks and redundant spaces.

[0104] The specific method for extracting sentence terms to obtain query terms is as follows:

[0105] Divide the preprocessed statement information into characters and fill the value array separately , calculate the minimum cost of each character , according to the calculated minimum cost from the array Backtrack to the optimal segmentation path, output the best word segmentation set, and use an array to store the optimal solution for each step; Initialize For each position Scan from front to back, for each possible prefix ,if In the security dictionary D, the minimum cost is calculated , the calculation formula is as follows:

[0106] ,

[0107]

[0108] in, represents the cost function, Represents a substring, Indicates the character position currently being processed, and indicates the position of segmentation during word segmentation. Indicates the previous split point, Represents a security dictionary, containing entries for traffic vehicle trajectories, Indicates the length of the substring. Used to measure the length of the substring. As the cost of an entry. In the security dictionary D, is 0, indicating no additional cost and it is a valid entry. Not in the security dictionary D, the cost is equal to the length of the substring , penalizing segments that are not in the dictionary.

[0109] Backtrack to the optimal segmentation path from the array and output the best word segmentation set.

[0110] As a refinement of the above embodiment, according to the query terms Perform a preliminary search in the danger dictionary and generate a list of candidate terms if no terms matching the query are found , use the similarity function to calculate the similarity between the query term and each word in the candidate term list;

[0111] The specific formula for using the similarity function is as follows:

[0112] ,

[0113] in, represents the similarity function, express The TF-IDF value of Indicates query terms, Indicates candidate terms Middle The value of a feature.

[0114] Set the threshold and Compare, if Greater than or equal to threshold , then there is a term that matches the query term; if there is no match, perform reverse query verification, for each candidate term , calculate the inverse mapping similarity, and output the terms whose inverse mapping similarity is greater than or equal to the inverse query threshold according to the inverse mapping similarity calculation as the filtering result terms; the inverse mapping similarity calculation formula is as follows:

[0115] ,

[0116] in, represents the inverse mapping similarity, and is the weight parameter, the sum of the two is 1, for The edit distance to the candidate term, Indicates the length of the query term. Indicates the length of the candidate term, Represents the dimension of the term.

[0117] As a refinement of the above embodiment, the output is calculated based on the inverse mapping similarity. , among which The reverse query threshold is set. By combining similarity calculation and reverse verification, it is ensured that the term is not in the term library, and the closest match can be found for secondary exclusion to obtain the filtered result term. .

[0118] As a refinement of the above embodiment, the obtained filtering result entries , enter the secondary keyword matching, K is A set of keywords, ,in For each key word input , calculate its matching score :

[0119] ,

[0120] ,

[0121] Among them, the calculated features , the target feature is , using cosine similarity to measure the similarity between keywords and expected matches. Calculate word frequency through TF-IDF , position weight Consider the position of the keyword in the input, where α+β+γ=1, is the weight of each part. Calculate the corresponding similarity, word frequency and position weight, and obtain the score of each part according to the formula to get the matching score . Set the threshold for a single keyword match , according to the judgment conditions , then the current term meets the search criteria. If there are four or more terms that meet the criteria , calculate the total score of the keyword entry, where if the keyword entry contains license plate information, then set the license plate information The weight of is 0.25, and the total weight of the remaining keywords is 0.75, which is evenly distributed to other keywords. Therefore, the total score is:

[0122] ,

[0123] According to the set score condition T is greater than the set score threshold .

[0124] If the match is satisfied, the corresponding keyword entry is returned. , query the vehicle trajectory keyword information database based on the obtained entries The corresponding vehicle trajectory video in the dictionary will be fed back to the acquired trajectory video and the text description of the trajectory in the dictionary will be output at the same time.

[0125] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A vehicle trajectory statistics method based on a large model, characterized in that: The following steps are involved: Collect traffic vehicle videos and obtain vehicle information features of each frame through deep separable convolution CAM based on multi-attention mechanism CAHS, including vehicle presence features, vehicle attribute features and background features; The vehicle information features are output through the LPRNet network to output the license plate number sequence, and the license plate number cutting characters in the license plate number sequence are one-hot encoded to form a new license plate information feature. ; The vehicle features, vehicle attribute features other than license plate information, and background features are input into the Transformer network, and word embedding and license plate information features are performed through one-hot Perform concatenation and set position encoding for each input feature vector; The query, key and value matrices obtained by linear transformation are used to calculate the attention scores of different positions of vehicle information features, establish the connection between the vehicle information features of adjacent frames, and generate the target trajectory video; The vehicle information features of each frame image are obtained through the deep separable convolution CAM based on the multi-attention mechanism CAHS. The specific method is as follows: Each frame of the traffic vehicle video is input into the deep separable convolution CAM based on the multi-attention mechanism CAHS, and is processed in sequence by 1x1 convolution, deep separable network DW layer, multi-attention mechanism CAHS and 1x1 convolution to output vehicle presence features, vehicle attribute features and background features; The multi-attention mechanism CAHS uses global maximum pooling and global average pooling to obtain local and global feature information, uses 1x1 convolution to replace the original fully connected layer, and uses hard-swish and hard-sigmoid as activation functions; The feature processing process is as follows: the input original features are processed by global maximum pooling and global average pooling, and then processed by 1x1 convolution, hard-swish activation function, and 1x1 convolution in sequence. The output features are processed by global maximum pooling and global average pooling again, and then concatenated and processed by hard-sigmoid activation function, and then multiplied with the original features and output; Calculate the vehicle information features to get the attention scores of different positions, establish the connection between the vehicle information features of adjacent frames, and generate the target trajectory video. The specific method is as follows: The features are divided into two categories according to the presence or absence of vehicles, the vehicle trajectory connection is established, and the video frames are numbered and counted, as follows: When there is no vehicle, the attention calculation is performed on the network input background features and the vehicle attribute features of the previous frame image to obtain the multi-head attention weight score; the weight score is used to determine whether it is greater than the vehicle feature threshold. If so, the background features of the current frame are added to the vehicle attribute features of the current frame; Calculate the multi-head attention weight score of the vehicle attribute features of the current frame and the vehicle attribute features of the previous frame image, determine the number of vehicle attribute features that meet the vehicle feature threshold, if the number of vehicle attribute features exceeds the set number threshold, then the current frame and the previous frame image have the same vehicle, establish vehicle trajectory connection, and generate a target trajectory video; When there is a vehicle, the multi-head attention weight score of the vehicle attribute features of the current frame and the vehicle attribute features of the previous frame image is directly calculated to determine the number of vehicle attribute features that meet the vehicle feature threshold. If the number of vehicle attribute features sets a number threshold, the same vehicle exists in the current frame and the previous frame image, and the vehicle trajectory connection is established to generate the target trajectory video; The position encoding formula is as follows: ， ， in, and is a learnable parameter, is the position dynamic correction parameter, for function, is the activation function, Represents the position index in the input sequence, used to encode each position, represents the dimension index of the feature vector, represents the length of the input feature, Indicates the location The position code.

2. The vehicle trajectory statistics method based on a large model according to claim 1 is characterized in that: The query, key and value matrices are as follows: ， in, , , represents the query, key, and value matrices corresponding to the feature, represents the dimension of the key vector, Represents matrix transpose.

3. A vehicle trajectory query method, characterized in that: include: Establishing the connection between the vehicle information features of adjacent frames by the vehicle trajectory statistics method based on a large model as described in any one of claims 1-2, establishing a target-text pair with the text information of the current vehicle attribute features and the video frame number and the vehicle, and generating a vehicle trajectory keyword information library; Collecting road system environment feature information of traffic vehicles, extracting road system environment feature information entries through a text encoder, establishing road system environment image-text pairs, obtaining intersection distance information and converting it into entries, and inputting the entries and road system environment image-text pairs into a vehicle trajectory keyword information database; Input vehicle query statement information and perform preprocessing, extract statement terms to obtain query terms; Preliminarily searching the danger dictionary according to the query term, if no term matching the query term is found, generating a candidate term list, using a similarity function to calculate the similarity between the query term and each word in the candidate term list, setting a threshold and comparing with the similarity function calculation result, if the similarity function calculation result is greater than or equal to the threshold, there is a term matching the query term, and the danger dictionary contains a non-traffic vehicle trajectory term; If there is no match, perform inverse query verification. For each candidate term, calculate the inverse mapping similarity. According to the inverse mapping similarity calculation, output the terms whose inverse mapping similarity is greater than or equal to the inverse query threshold as the filtering result terms. The inverse mapping similarity calculation formula is as follows: ， in, represents the inverse mapping similarity, and is the weight parameter, the sum of the two is 1, for The edit distance to the candidate term, Indicates the length of the query term. Indicates the length of the candidate term, Indicates the dimension of the term; The filtered result terms are matched with secondary keywords and their matching scores are calculated. If the matching score is greater than or equal to the keyword matching threshold, the current term meets the search conditions. If there are four or more terms that meet the search conditions, the total score of the keyword terms is calculated. If the total score of the keyword entries meets the set conditions, the corresponding keyword entries that are successfully matched are returned. According to the obtained successfully matched keyword entries, the corresponding vehicle trajectory video in the vehicle trajectory keyword information library is queried, and the video of the trajectory and the corresponding text description are output.

4. The vehicle trajectory query method according to claim 3, characterized in that: The specific method for extracting sentence terms to obtain query terms is as follows: Divide the preprocessed statement information into characters and fill the value array separately , calculate the minimum cost of each character , according to the calculated minimum cost from the array Backtrack to the optimal segmentation path and output the best word segmentation set; the minimum cost The calculation formula is as follows: ， in, represents the cost function, Represents a substring, Indicates the character position currently being processed, and indicates the position of segmentation during word segmentation. Indicates the previous split point, Represents a security dictionary, containing entries for traffic vehicle trajectories, Indicates the substring length; Backtrack to the optimal segmentation path from the array and output the best word segmentation set.

5. The vehicle trajectory query method according to claim 4, characterized in that: The specific formulas of similarity function and inverse mapping similarity are as follows: ， in, represents the similarity function, express The TF-IDF value of Indicates query terms, Indicates candidate terms Middle The value of a feature.

6. The vehicle trajectory query method according to claim 3, characterized in that: Total score of key words The calculation formula is as follows: ， ， ， in, Indicates the matching score of the license plate information, Indicates the matching score of the keyword entry. Indicates the total number of keywords. , and represents the weight, It means calculating the similarity between the keyword and the expected match; Indicates that the word frequency is calculated by TF-IDF. represents the position weight, represents the characteristics of the calculation, Represents the target feature.

Images