Highway video retrieval method and system based on feature matching

Abstract

This application provides a method and system for highway video retrieval based on feature matching. The method includes: performing motion physical feature encoding on continuous highway video frames to obtain motion physical feature vectors; extracting features from the user's query video to obtain the query motion physical feature vector; then retrieving the initial set of matching video segments through a preset federated dynamic index; when the fog concentration monitoring data is less than or equal to a preset fog concentration warning monitoring threshold, the segments are sorted in descending order of matching degree, and the initial matching video segment with the highest matching degree is output to the user terminal for display; when the fog concentration is greater than the preset fog concentration warning monitoring threshold, fog-containing video frames are acquired and processed to obtain defogging optimized feature vectors and fog distribution maps; then, the segments are retrieved through a preset federated dynamic index and sorted in descending order, and the optimized matching video segment with the highest matching degree is output to the user terminal for display; thereby realizing highway video retrieval based on feature matching.

Description

Technical Field

[0001] This application relates to the field of highway video surveillance technology, and more specifically, to a highway video retrieval method and system based on feature matching. Background Technology

[0002] In the field of highway traffic monitoring, video retrieval technology is of great significance for accident tracing, vehicle tracking, and traffic flow analysis. Traditional techniques, in terms of feature generation, rely on manual features or CNN features, which struggle to address feature distortion caused by the high-speed movement of vehicles on highways, leading to reduced retrieval accuracy. In terms of system architecture, centralized indexes or static topologies are often used, making it difficult to balance retrieval efficiency and privacy protection. Regarding adaptability to extreme scenarios, existing technologies primarily focus on nighttime enhancement and debris detection, but are poorly adapted to fog conditions, resulting in retrieval blind spots. Fog blurs video images and reduces contrast, severely impacting the accuracy of feature extraction and consequently affecting retrieval performance. Therefore, existing video retrieval technologies have many shortcomings in fog scenarios. Thus, there is an urgent need for an efficient, accurate, real-time video retrieval method suitable for highway fog scenarios that also protects privacy.

[0003] Effective technical solutions are urgently needed to address the above problems. Summary of the Invention

[0004] The purpose of this application is to provide a method and system for highway video retrieval based on feature matching. It can solve the feature distortion caused by high-speed motion through motion physical feature encoding, federated dynamic index retrieval, and fog adaptive processing, thereby improving the retrieval accuracy and efficiency in foggy scenes. At the same time, it takes into account both retrieval efficiency and privacy protection.

[0005] In a first aspect, this application provides a highway video retrieval method based on feature matching, comprising the following steps:

[0006] The motion physics feature evaluation data of continuous highway video frames are acquired and processed by motion physics feature encoding to obtain motion physics feature vectors.

[0007] The system retrieves the user's query video, extracts its features to obtain the physical feature vector of the query motion, and then retrieves and processes it through a preset federated dynamic index to obtain an initial set of matching video segments.

[0008] Acquire fog concentration monitoring data and compare it with the preset fog concentration early warning monitoring threshold;

[0009] If the initial matching video clip set is less than or equal to the preset fog concentration warning monitoring threshold, then the initial matching video clip set is sorted in descending order of matching degree, and the initial matching video clip with the highest matching degree is output to the user terminal for display.

[0010] If the fog concentration exceeds the preset fog concentration warning monitoring threshold, a fog-containing video frame is acquired, and the motion physical feature vector is combined with the preset fog adaptive processing method to obtain a defogging optimized feature vector and a fog distribution map.

[0011] The fog distribution map and defogging optimization feature vector are retrieved and processed using a preset federated dynamic index to obtain an optimized matching video clip set;

[0012] The optimized matching video clip set is sorted in descending order of matching degree, and the optimized matching video clip with the highest matching degree is output to the user terminal for display.

[0013] Optionally, in the highway video retrieval method based on feature matching described in this application, the step of acquiring motion physical feature evaluation data of continuous highway video frames and performing motion physical feature encoding processing to obtain motion physical feature vectors includes:

[0014] Motion physical characteristic evaluation data is obtained from continuous highway video frames, including video frames and radar speed measurement data and vehicle parameter data within a preset time period;

[0015] The vehicle body parameter data includes vehicle length, vehicle width, and aerodynamic shape characteristic parameters;

[0016] The video frames and radar speed measurement data are preprocessed to obtain corrected video frames and smoothed vehicle speed data.

[0017] The corresponding pressure coefficient is obtained by querying the preset vehicle speed and pressure coefficient mapping table based on the smoothed vehicle speed data.

[0018] The corrected video frame is processed based on the pressure coefficient and the vehicle body length, vehicle body width and aerodynamic shape feature parameters to obtain pixel displacement prediction values;

[0019] Based on the corrected video frame and smoothed vehicle speed data, FFT transformation and phase correction processing are performed to obtain the texture frequency domain energy spectrum;

[0020] Motion physics feature vectors are obtained by encoding motion physics features based on the pixel displacement prediction value and the texture frequency domain energy spectrum.

[0021] Optionally, the highway video retrieval method based on feature matching described in this application further includes:

[0022] Obtain the feature gradient of the motion physical feature vector of the edge node;

[0023] The feature gradients are then subjected to federated averaging and aggregation to obtain aggregated feature gradients.

[0024] The preset federated shared model is updated based on the aggregated feature gradient, and a dynamic index table is generated.

[0025] Optionally, in the highway video retrieval method based on feature matching described in this application, the step of acquiring the user's query video, extracting features to obtain the query motion physical feature vector, and then obtaining an initial set of matching video segments through a preset federated dynamic index retrieval process includes:

[0026] Obtain the user's query video and extract its features to obtain the query motion physical feature vector;

[0027] The dimensionality of the query motion physical feature vector is reduced to obtain the dimensionality-reduced query feature vector.

[0028] The dimensionality reduction query feature vector is hashed and matched with a preset hash table to obtain a coarse matching video clip set;

[0029] The coarsely matched video clip set and the query motion physical feature vector are processed through a dynamic index table to obtain the initial matched video clip set.

[0030] Optionally, in the highway video retrieval method based on feature matching described in this application, the step of acquiring a fog-containing video frame if the fog concentration exceeds a preset fog concentration warning monitoring threshold, and processing it using a preset fog patch adaptive processing method in conjunction with the motion physical feature vector to obtain a defogging optimized feature vector and a fog patch distribution map, includes:

[0031] If the fog concentration exceeds the preset fog concentration warning monitoring threshold, then obtain fog-containing video frames;

[0032] The foggy video frame is dehazed through a preset dehazing and feature joint network to obtain a dehazed video frame, and features are extracted to obtain a dehazed feature vector.

[0033] The dehazing feature vector is aligned with the motion physics feature vector to obtain the dehazing optimized feature vector.

[0034] The foggy video frame is input into a preset fog distribution map generation model for processing to obtain an initial fog distribution map.

[0035] The similarity values ​​of the dehazed video frames and the initial fog distribution map are compared to obtain the image similarity values.

[0036] If the similarity is greater than or equal to the preset image similarity threshold, the initial fog distribution map is determined to be a fog distribution map.

[0037] Conversely, adjust the parameters of the preset fog distribution map generation model.

[0038] Optionally, the highway video retrieval method based on feature matching described in this application further includes:

[0039] The fog concentration monitoring data is compared with a preset fog concentration level threshold, and the fog concentration level is determined according to the range in which it falls, including level one, level two, level three, level four or level five.

[0040] Based on the fog concentration level, query the preset defogging adaptive weight value list to obtain the defogging adaptive weight value;

[0041] The preset dehazing and feature joint network is optimized based on the dehazing adaptive weight values.

[0042] Optionally, in the highway video retrieval method based on feature matching described in this application, the step of retrieving the fog distribution map and the defogging optimized feature vector through a preset federated dynamic index to obtain an optimized matching video segment set includes:

[0043] The dynamic retrieval similarity threshold is determined based on the aforementioned fog distribution map;

[0044] The dehazing optimized feature vector is combined with the dynamic retrieval similarity threshold and processed through a preset federated dynamic index retrieval to obtain an optimized matching video segment set.

[0045] Secondly, this application provides a highway video retrieval system based on feature matching. The system includes a memory and a processor. The memory includes a program for a highway video retrieval method based on feature matching. When the program for the highway video retrieval method based on feature matching is executed by the processor, it performs the following steps:

[0046] The motion physics feature evaluation data of continuous highway video frames are acquired and processed by motion physics feature encoding to obtain motion physics feature vectors.

[0047] The system retrieves the user's query video, extracts its features to obtain the physical feature vector of the query motion, and then retrieves and processes it through a preset federated dynamic index to obtain an initial set of matching video segments.

[0048] Acquire fog concentration monitoring data and compare it with the preset fog concentration early warning monitoring threshold;

[0049] If the initial matching video clip set is less than or equal to the preset fog concentration warning monitoring threshold, then the initial matching video clip set is sorted in descending order of matching degree, and the initial matching video clip with the highest matching degree is output to the user terminal for display.

[0050] If the fog concentration exceeds the preset fog concentration warning monitoring threshold, a fog-containing video frame is acquired, and the motion physical feature vector is combined with the preset fog adaptive processing method to obtain a defogging optimized feature vector and a fog distribution map.

[0051] The fog distribution map and defogging optimization feature vector are retrieved and processed using a preset federated dynamic index to obtain an optimized matching video clip set;

[0052] The optimized matching video clip set is sorted in descending order of matching degree, and the optimized matching video clip with the highest matching degree is output to the user terminal for display.

[0053] Optionally, in the feature-matching-based highway video retrieval system described in this application, the step of acquiring motion physical feature evaluation data of continuous highway video frames and performing motion physical feature encoding processing to obtain motion physical feature vectors includes:

[0054] Motion physical characteristic evaluation data is obtained from continuous highway video frames, including video frames and radar speed measurement data and vehicle parameter data within a preset time period;

[0055] The vehicle body parameter data includes vehicle length, vehicle width, and aerodynamic shape characteristic parameters;

[0056] The video frames and radar speed measurement data are preprocessed to obtain corrected video frames and smoothed vehicle speed data.

[0057] The corresponding pressure coefficient is obtained by querying the preset vehicle speed and pressure coefficient mapping table based on the smoothed vehicle speed data.

[0058] The corrected video frame is processed based on the pressure coefficient and the vehicle body length, vehicle body width and aerodynamic shape feature parameters to obtain pixel displacement prediction values;

[0059] Based on the corrected video frame and smoothed vehicle speed data, FFT transformation and phase correction processing are performed to obtain the texture frequency domain energy spectrum;

[0060] Motion physics feature vectors are obtained by encoding motion physics features based on the pixel displacement prediction value and the texture frequency domain energy spectrum.

[0061] Optionally, the highway video retrieval system based on feature matching described in this application further includes:

[0062] Obtain the feature gradient of the motion physical feature vector of the edge node;

[0063] The feature gradients are then subjected to federated averaging and aggregation to obtain aggregated feature gradients.

[0064] The preset federated shared model is updated based on the aggregated feature gradient, and a dynamic index table is generated.

[0065] As can be seen from the above, the highway video retrieval method and system based on feature matching provided in this application solves the feature distortion caused by high-speed motion and improves the retrieval accuracy and efficiency in foggy scenes by using motion physical feature encoding, federated dynamic index retrieval, and fog adaptive processing. At the same time, it takes into account both retrieval efficiency and privacy protection.

[0066] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description

[0067] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 A flowchart illustrating the highway video retrieval method based on feature matching provided in this application embodiment;

[0069] Figure 2 A flowchart illustrating the process of obtaining motion physical feature vectors in a highway video retrieval method based on feature matching, as provided in this application embodiment.

[0070] Figure 3 The flowchart illustrates the process of obtaining dehazing optimized feature vectors and fog distribution maps in the highway video retrieval method based on feature matching provided in this application embodiment. Detailed Implementation

[0071] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0072] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0073] Please refer to Figure 1 , Figure 1 This is a flowchart of a highway video retrieval method based on feature matching according to some embodiments of this application. This highway video retrieval method based on feature matching is used in terminal devices, such as computers and mobile phones. The highway video retrieval method based on feature matching includes the following steps:

[0074] S11. Obtain motion physical feature evaluation data of continuous highway video frames, and perform motion physical feature encoding processing to obtain motion physical feature vectors.

[0075] S12. Obtain the user's query video, extract its features to obtain the query motion physical feature vector, and then retrieve and process it through a preset federated dynamic index to obtain an initial set of matching video segments.

[0076] S13. Obtain fog concentration monitoring data and compare it with the preset fog concentration early warning monitoring threshold.

[0077] S141. If the initial matching video segment set is less than or equal to the preset fog concentration warning monitoring threshold, then the initial matching video segment set is sorted in descending order of matching degree, and the initial matching video segment with the highest matching degree is output to the user terminal for display.

[0078] S142. If the fog concentration exceeds the preset fog concentration warning monitoring threshold, then obtain the fog-containing video frame, and process it using the preset fog adaptive processing method in combination with the motion physical feature vector to obtain the defogging optimized feature vector and fog distribution map.

[0079] S15. The fog distribution map and the defogging optimization feature vector are processed by a preset federated dynamic index retrieval to obtain an optimized matching video clip set;

[0080] S16. Arrange the optimized matching video clip set in descending order of matching degree, and output the optimized matching video clip with the highest matching degree to the user terminal for display.

[0081] It should be noted that, firstly, based on continuous highway video frames combined with corresponding radar speed measurement data and vehicle parameters, a motion physical feature encoder is used to predict pixel displacement and perform fast Fourier transform processing to obtain motion physical feature vectors. Then, the feature gradients are uploaded by edge nodes and aggregated to construct a federated dynamic index. Next, based on the fog concentration, a fog distribution map is generated according to the atmospheric scattering model and meteorological physical constraints, and the preset defogging and feature joint network is optimized. When a user initiates a query, the local motion physical feature encoder extracts the features of the query video and matches them with the dynamic index table generated based on the federated sharing model. The search results are returned based on the matching degree and output to the user terminal for display.

[0082] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating the process of obtaining motion physical feature vectors in a highway video retrieval method based on feature matching according to some embodiments of this application. According to embodiments of the present invention, the step of acquiring motion physical feature evaluation data of continuous highway video frames and performing motion physical feature encoding processing to obtain motion physical feature vectors includes:

[0083] S21. Obtain motion physical characteristic evaluation data based on continuous highway video frames, including video frames and radar speed measurement data and vehicle parameter data within a preset time period;

[0084] S22, The vehicle body parameter data includes vehicle body length, vehicle body width, and aerodynamic shape characteristic parameters;

[0085] S23. Preprocess the video frames and radar speed measurement data to obtain corrected video frames and smoothed vehicle speed data;

[0086] S24. Query the preset vehicle speed and pressure coefficient mapping table according to the smoothed vehicle speed data to obtain the corresponding pressure coefficient;

[0087] S25. Process the corrected video frame based on the pressure coefficient and the vehicle body length, vehicle body width and aerodynamic shape characteristic parameters to obtain pixel displacement prediction values;

[0088] S26. Perform FFT transformation and phase correction processing on the corrected video frame and smoothed vehicle speed data to obtain the texture frequency domain energy spectrum;

[0089] S27. Perform motion physical feature encoding processing based on the pixel displacement prediction value and the texture frequency domain energy spectrum to obtain the motion physical feature vector.

[0090] It should be noted that the video frames within the preset time period undergo distortion correction and time synchronization preprocessing. The radar speed measurement data is smoothed using Kalman filtering. Based on the smoothed vehicle speed data, a preset vehicle speed-pressure coefficient mapping table is consulted to obtain the corresponding pressure coefficient. This preset vehicle speed-pressure coefficient mapping table is pre-constructed by those skilled in the art and can be dynamically adjusted. Based on the obtained pressure coefficient, combined with vehicle length, width, and aerodynamic shape characteristic parameters, the airflow pressure distribution on the vehicle surface is calculated using a lookup table method and interpolation to process the corrected video frames and obtain pixel displacement prediction values. The aerodynamic shape characteristic parameters include the vehicle streamline coefficient and frontal area, used to assist in the analysis of motion physical characteristics. The encoder performs more accurate airflow deformation simulation, improving the accuracy of pixel displacement prediction. An FFT transformation is performed on local texture regions of the video frame (e.g., using 8×8 pixel blocks) to obtain a frequency domain spectrum. The Doppler frequency shift of the texture features in the motion direction is calculated based on radar velocity data. Linear compensation is performed on the texture phase in the frequency domain. After correction, the texture is transformed back to the spatial domain via IFFT to obtain a texture frequency domain energy spectrum that eliminates motion blur. Finally, motion physical feature encoding is performed based on the pixel displacement prediction value and the texture frequency domain energy spectrum to obtain a motion physical feature vector with 512 dimensions. The first 256 dimensions represent spatial motion features, and the last 256 dimensions represent frequency domain correction features.

[0091] According to an embodiment of the present invention, it further includes:

[0092] Obtain the feature gradient of the motion physical feature vector of the edge node;

[0093] The feature gradients are then subjected to federated averaging and aggregation to obtain aggregated feature gradients.

[0094] The preset federated shared model is updated based on the aggregated feature gradient, and a dynamic index table is generated.

[0095] It should be noted that the edge nodes are equipped with lightweight motion physics feature encoders to process local highway video data, generate motion physics feature vectors, and calculate their feature gradients relative to a preset federated sharing model. The edge nodes then upload these feature gradients to the central server. The central server receives the feature gradients from each edge node and aggregates them using federated averaging. The aggregated gradients are used to update the parameters of the preset federated sharing model. After the update, the federated sharing model generates a dynamic index table based on the updated feature distribution. For example, when the federated sharing model learns a new distribution pattern of vehicle motion physics features in a foggy scene, it will adjust the structure or content of the dynamic index table accordingly to adapt to this change. Subsequently, the central server broadcasts the updated federated sharing model and dynamic index table to each edge node.

[0096] According to an embodiment of the present invention, the step of acquiring the user's query video, extracting features to obtain the query motion physical feature vector, and then obtaining an initial set of matching video segments through a preset federated dynamic index retrieval process includes:

[0097] Obtain the user's query video and extract its features to obtain the query motion physical feature vector;

[0098] The dimensionality of the query motion physical feature vector is reduced to obtain the dimensionality-reduced query feature vector.

[0099] The dimensionality reduction query feature vector is hashed and matched with a preset hash table to obtain a coarse matching video clip set;

[0100] The coarsely matched video clip set and the query motion physical feature vector are processed through a dynamic index table to obtain the initial matched video clip set.

[0101] It should be noted that when a user initiates a query, the local motion physics feature encoder extracts the corresponding motion physics feature vector from the query video based on the motion physics feature encoding principle, reduces the dimensionality to 128 dimensions to obtain the dimensionality-reduced query feature vector, and calculates the hash value. The pre-constructed motion physics feature vector containing features of all historical video frames is then reduced to 128 dimensions to obtain the dimensionality-reduced feature vector, and its hash value is calculated to construct a preset hash table. Based on the hash value of the dimensionality-reduced query feature vector, all feature records that are exactly the same as the preset hash value in the preset hash table or have a Hamming distance ≤ 2 are queried to obtain a coarse-matched video segment set. This coarse-matched video segment set is then mapped back to a 512-dimensional space. Finally, based on the query motion physics feature vector (512-dimensional), the nearest neighbor feature is queried in the dynamic index table (using a weighted distance of 70% Euclidean distance + 30% cosine similarity) to obtain the initial set of precisely matched video segments.

[0102] Please refer to Figure 3 , Figure 3 This is a flowchart illustrating the process of obtaining defogging optimized feature vectors and fog distribution maps in a highway video retrieval method based on feature matching according to some embodiments of this application. According to an embodiment of the present invention, if the fog concentration exceeds a preset fog concentration warning monitoring threshold, a fog-containing video frame is acquired, and the motion physical feature vector is combined with a preset fog adaptive processing method to obtain a defogging optimized feature vector and a fog distribution map, including:

[0103] S31. If the fog concentration exceeds the preset fog concentration warning monitoring threshold, then acquire fog-containing video frames.

[0104] S32. The foggy video frame is dehazed through a preset dehazing and feature joint network to obtain a dehazed video frame, and features are extracted to obtain a dehazed feature vector.

[0105] S33. Perform feature alignment processing on the dehazing feature vector and the motion physical feature vector to obtain the dehazing optimized feature vector;

[0106] S34. Input the foggy video frame into a preset fog distribution map generation model for processing to obtain an initial fog distribution map;

[0107] S35. Compare the similarity between the dehazed video frame and the initial fog distribution map to obtain an image similarity value;

[0108] S361. If the similarity is greater than or equal to the preset image similarity threshold, then the initial fog distribution map is determined to be a fog distribution map.

[0109] S362. Conversely, adjust the parameters of the preset fog distribution map generation model.

[0110] It should be noted that, for foggy scenes, the foggy video frames are dehazed using a pre-defined dehazing and feature joint network to obtain dehazed video frames, and features are extracted to obtain dehazed feature vectors. The pre-defined dehazing and feature joint network includes a dehazing branch and a feature matching branch. The dehazing branch uses a residual-connected Encoder-Decoder structure and outputs the dehazed feature vector of the clear image. The feature matching branch calculates the similarity matrix between the dehazed feature vector of the clear image and the motion physical feature vectors of the continuous highway video frames. The pre-defined dehazing and feature joint network is pre-constructed by those skilled in the art. The obtained dehazed feature vectors and motion physical feature vectors are then aligned using an attention mechanism to strengthen the weights of overlapping areas, resulting in optimized dehazing feature vectors for matching retrieval. The foggy video frames are then input... The initial fog distribution map is generated by processing the data into a preset fog distribution map generation model. The pixel values ​​(0-255) of each pixel are calculated based on the extinction coefficient of the pixel and the minimum and maximum extinction coefficients for a visibility of 500 meters and 50 meters, respectively. This preset fog distribution map generation model is obtained by a person skilled in the art using a GAN (Generative Adversarial Network) with a "generator-discriminator" architecture and pre-trained with physical constraints. The obtained initial fog distribution map is then subjected to physical consistency verification, i.e., the similarity value between the dehazed video frame and the initial fog distribution map is calculated. If the similarity value is less than a preset image similarity threshold, the generated initial fog distribution map is considered unqualified. The generator parameters are then fine-tuned using gradient descent, and the map is regenerated and verified. Otherwise, it is considered qualified, and the initial fog distribution map is the valid fog distribution map.

[0111] According to an embodiment of the present invention, it further includes:

[0112] The fog concentration monitoring data is compared with a preset fog concentration level threshold, and the fog concentration level is determined according to the range in which it falls, including level one, level two, level three, level four or level five.

[0113] Based on the fog concentration level, query the preset defogging adaptive weight value list to obtain the defogging adaptive weight value;

[0114] The preset dehazing and feature joint network is optimized based on the dehazing adaptive weight values.

[0115] It should be noted that, in order to ensure a balance between the "dehazing effect" and "feature matching" of the preset dehazing and feature joint network, corresponding optimization is required based on the fog concentration level. The fog concentration levels include level 1, level 2, level 3, level 4, or level 5, with level 1 having the lowest concentration and the best visibility, and level 5 having the highest concentration and the worst visibility. The preset dehazing adaptive weight value list is consulted according to the fog concentration level to obtain the dehazing adaptive weight value, including adjusting the α value for the dehazing effect and the β value for ensuring matching stability. For example, when the fog concentration is high (e.g., visibility <100m), α is increased (e.g., α = 0.8) to prioritize optimizing the dehazing loss and ensure image clarity; when the fog concentration is low, β is increased (e.g., β = 0.7) to enhance matching stability.

[0116] According to an embodiment of the present invention, the step of obtaining an optimized matching video segment set by retrieving the fog distribution map and the defogging optimization feature vector through a preset federated dynamic index includes:

[0117] The dynamic retrieval similarity threshold is determined based on the aforementioned fog distribution map;

[0118] The dehazing optimized feature vector is combined with the dynamic retrieval similarity threshold and processed through a preset federated dynamic index retrieval to obtain an optimized matching video segment set.

[0119] It should be noted that during the retrieval process, the feature matching threshold for regions with high fog concentration (e.g., pixel values ​​close to 255) is relaxed (e.g., the similarity threshold is reduced by 15%), while a strict threshold is applied to regions with low fog concentration (pixel values ​​close to 0). This balances the retrieval accuracy for different fog areas. Based on the defogging optimized feature vector, a coarse and fine retrieval process is performed using a dynamic index table according to different dynamic retrieval similarity thresholds to obtain an optimized matching video clip set. The clips are then sorted from high to low according to their matching scores, and the optimized matching video clip with the highest matching score is output to the user for display, thus becoming the retrieval result.

[0120] This invention also discloses a highway video retrieval system based on feature matching, comprising a memory and a processor. The memory includes a highway video retrieval method program based on feature matching. When the feature matching highway video retrieval method program is executed by the processor, it performs the following steps:

[0121] The motion physics feature evaluation data of continuous highway video frames are acquired and processed by motion physics feature encoding to obtain motion physics feature vectors.

[0122] The system retrieves the user's query video, extracts its features to obtain the physical feature vector of the query motion, and then retrieves and processes it through a preset federated dynamic index to obtain an initial set of matching video segments.

[0123] Acquire fog concentration monitoring data and compare it with the preset fog concentration early warning monitoring threshold;

[0124] If the initial matching video clip set is less than or equal to the preset fog concentration warning monitoring threshold, then the initial matching video clip set is sorted in descending order of matching degree, and the initial matching video clip with the highest matching degree is output to the user terminal for display.

[0125] If the fog concentration exceeds the preset fog concentration warning monitoring threshold, a fog-containing video frame is acquired, and the motion physical feature vector is combined with the preset fog adaptive processing method to obtain a defogging optimized feature vector and a fog distribution map.

[0126] The fog distribution map and defogging optimization feature vector are retrieved and processed using a preset federated dynamic index to obtain an optimized matching video clip set;

[0127] The optimized matching video clip set is sorted in descending order of matching degree, and the optimized matching video clip with the highest matching degree is output to the user terminal for display.

[0128] It should be noted that, firstly, based on continuous highway video frames combined with corresponding radar speed measurement data and vehicle parameters, a motion physical feature encoder is used to predict pixel displacement and perform fast Fourier transform processing to obtain motion physical feature vectors. Then, the feature gradients are uploaded by edge nodes and aggregated to construct a federated dynamic index. Next, based on the fog concentration, a fog distribution map is generated according to the atmospheric scattering model and meteorological physical constraints, and the preset defogging and feature joint network is optimized. When a user initiates a query, the local motion physical feature encoder extracts the features of the query video and matches them with the dynamic index table generated based on the federated sharing model. The search results are returned based on the matching degree and output to the user terminal for display.

[0129] According to an embodiment of the present invention, the step of acquiring motion physics feature evaluation data of continuous highway video frames and performing motion physics feature encoding processing to obtain motion physics feature vectors includes:

[0130] Motion physical characteristic evaluation data is obtained from continuous highway video frames, including video frames and radar speed measurement data and vehicle parameter data within a preset time period;

[0131] The vehicle body parameter data includes vehicle length, vehicle width, and aerodynamic shape characteristic parameters;

[0132] The video frames and radar speed measurement data are preprocessed to obtain corrected video frames and smoothed vehicle speed data.

[0133] The corresponding pressure coefficient is obtained by querying the preset vehicle speed and pressure coefficient mapping table based on the smoothed vehicle speed data.

[0134] The corrected video frame is processed based on the pressure coefficient and the vehicle body length, vehicle body width and aerodynamic shape feature parameters to obtain pixel displacement prediction values;

[0135] Based on the corrected video frame and smoothed vehicle speed data, FFT transformation and phase correction processing are performed to obtain the texture frequency domain energy spectrum;

[0136] Motion physics feature vectors are obtained by encoding motion physics features based on the pixel displacement prediction value and the texture frequency domain energy spectrum.

[0137] It should be noted that the video frames within the preset time period undergo distortion correction and time synchronization preprocessing. The radar speed measurement data is smoothed using Kalman filtering. Based on the smoothed vehicle speed data, a preset vehicle speed-pressure coefficient mapping table is consulted to obtain the corresponding pressure coefficient. This preset vehicle speed-pressure coefficient mapping table is pre-constructed by those skilled in the art and can be dynamically adjusted. Based on the obtained pressure coefficient, combined with vehicle length, width, and aerodynamic shape characteristic parameters, the airflow pressure distribution on the vehicle surface is calculated using a lookup table method and interpolation to process the corrected video frames and obtain pixel displacement prediction values. The aerodynamic shape characteristic parameters include the vehicle streamline coefficient and frontal area, used to assist in the analysis of motion physical characteristics. The encoder performs more accurate airflow deformation simulation, improving the accuracy of pixel displacement prediction. An FFT transformation is performed on local texture regions of the video frame (e.g., using 8×8 pixel blocks) to obtain a frequency domain spectrum. The Doppler frequency shift of the texture features in the motion direction is calculated based on radar velocity data. Linear compensation is performed on the texture phase in the frequency domain. After correction, the texture is transformed back to the spatial domain via IFFT to obtain a texture frequency domain energy spectrum that eliminates motion blur. Finally, motion physical feature encoding is performed based on the pixel displacement prediction value and the texture frequency domain energy spectrum to obtain a motion physical feature vector with 512 dimensions. The first 256 dimensions represent spatial motion features, and the last 256 dimensions represent frequency domain correction features.

[0138] According to an embodiment of the present invention, it further includes:

[0139] Obtain the feature gradient of the motion physical feature vector of the edge node;

[0140] The feature gradients are then subjected to federated averaging and aggregation to obtain aggregated feature gradients.

[0141] The preset federated shared model is updated based on the aggregated feature gradient, and a dynamic index table is generated.

[0142] It should be noted that the edge nodes are equipped with lightweight motion physics feature encoders to process local highway video data, generate motion physics feature vectors, and calculate their feature gradients relative to a preset federated sharing model. The edge nodes then upload these feature gradients to the central server. The central server receives the feature gradients from each edge node and aggregates them using federated averaging. The aggregated gradients are used to update the parameters of the preset federated sharing model. After the update, the federated sharing model generates a dynamic index table based on the updated feature distribution. For example, when the federated sharing model learns a new distribution pattern of vehicle motion physics features in a foggy scene, it will adjust the structure or content of the dynamic index table accordingly to adapt to this change. Subsequently, the central server broadcasts the updated federated sharing model and dynamic index table to each edge node.

[0143] According to an embodiment of the present invention, the step of acquiring the user's query video, extracting features to obtain the query motion physical feature vector, and then obtaining an initial set of matching video segments through a preset federated dynamic index retrieval process includes:

[0144] Obtain the user's query video and extract its features to obtain the query motion physical feature vector;

[0145] The dimensionality of the query motion physical feature vector is reduced to obtain the dimensionality-reduced query feature vector.

[0146] The dimensionality reduction query feature vector is hashed and matched with a preset hash table to obtain a coarse matching video clip set;

[0147] The coarsely matched video clip set and the query motion physical feature vector are processed through a dynamic index table to obtain the initial matched video clip set.

[0148] It should be noted that when a user initiates a query, the local motion physics feature encoder extracts the corresponding motion physics feature vector from the query video based on the motion physics feature encoding principle, reduces the dimensionality to 128 dimensions to obtain the dimensionality-reduced query feature vector, and calculates the hash value. The pre-constructed motion physics feature vector containing features of all historical video frames is then reduced to 128 dimensions to obtain the dimensionality-reduced feature vector, and its hash value is calculated to construct a preset hash table. Based on the hash value of the dimensionality-reduced query feature vector, all feature records that are exactly the same as the preset hash value in the preset hash table or have a Hamming distance ≤ 2 are queried to obtain a coarse-matched video segment set. This coarse-matched video segment set is then mapped back to a 512-dimensional space. Finally, based on the query motion physics feature vector (512-dimensional), the nearest neighbor feature is queried in the dynamic index table (using a weighted distance of 70% Euclidean distance + 30% cosine similarity) to obtain the initial set of precisely matched video segments.

[0149] According to an embodiment of the present invention, if the fog concentration exceeds a preset fog concentration warning monitoring threshold, a fog-containing video frame is acquired, and the motion physical feature vector is combined with a preset fog adaptive processing method to obtain a defogging optimized feature vector and a fog distribution map, including:

[0150] If the fog concentration exceeds the preset fog concentration warning monitoring threshold, then obtain fog-containing video frames;

[0151] The foggy video frame is dehazed through a preset dehazing and feature joint network to obtain a dehazed video frame, and features are extracted to obtain a dehazed feature vector.

[0152] The dehazing feature vector is aligned with the motion physics feature vector to obtain the dehazing optimized feature vector.

[0153] The foggy video frame is input into a preset fog distribution map generation model for processing to obtain an initial fog distribution map.

[0154] The similarity values ​​of the dehazed video frames and the initial fog distribution map are compared to obtain the image similarity values.

[0155] If the similarity is greater than or equal to the preset image similarity threshold, the initial fog distribution map is determined to be a fog distribution map.

[0156] Conversely, adjust the parameters of the preset fog distribution map generation model.

[0157] It should be noted that, for foggy scenes, the foggy video frames are dehazed using a pre-defined dehazing and feature joint network to obtain dehazed video frames, and features are extracted to obtain dehazed feature vectors. The pre-defined dehazing and feature joint network includes a dehazing branch and a feature matching branch. The dehazing branch uses a residual-connected Encoder-Decoder structure and outputs the dehazed feature vector of the clear image. The feature matching branch calculates the similarity matrix between the dehazed feature vector of the clear image and the motion physical feature vectors of the continuous highway video frames. The pre-defined dehazing and feature joint network is pre-constructed by those skilled in the art. The obtained dehazed feature vectors and motion physical feature vectors are then aligned using an attention mechanism to strengthen the weights of overlapping areas, resulting in optimized dehazing feature vectors for matching retrieval. The foggy video frames are then input... The initial fog distribution map is generated by processing the data into a preset fog distribution map generation model. The pixel values ​​(0-255) of each pixel are calculated based on the extinction coefficient of the pixel and the minimum and maximum extinction coefficients for a visibility of 500 meters and 50 meters, respectively. This preset fog distribution map generation model is obtained by a person skilled in the art using a GAN (Generative Adversarial Network) with a "generator-discriminator" architecture and pre-trained with physical constraints. The obtained initial fog distribution map is then subjected to physical consistency verification, i.e., the similarity value between the dehazed video frame and the initial fog distribution map is calculated. If the similarity value is less than a preset image similarity threshold, the generated initial fog distribution map is considered unqualified. The generator parameters are then fine-tuned using gradient descent, and the map is regenerated and verified. Otherwise, it is considered qualified, and the initial fog distribution map is the valid fog distribution map.

[0158] According to an embodiment of the present invention, it further includes:

[0159] The fog concentration monitoring data is compared with a preset fog concentration level threshold, and the fog concentration level is determined according to the range in which it falls, including level one, level two, level three, level four or level five.

[0160] Based on the fog concentration level, query the preset defogging adaptive weight value list to obtain the defogging adaptive weight value;

[0161] The preset dehazing and feature joint network is optimized based on the dehazing adaptive weight values.

[0162] It should be noted that, in order to ensure a balance between the "dehazing effect" and "feature matching" of the preset dehazing and feature joint network, corresponding optimization is required based on the fog concentration level. The fog concentration levels include level 1, level 2, level 3, level 4, or level 5, with level 1 having the lowest concentration and the best visibility, and level 5 having the highest concentration and the worst visibility. The preset dehazing adaptive weight value list is consulted according to the fog concentration level to obtain the dehazing adaptive weight value, including adjusting the α value for the dehazing effect and the β value for ensuring matching stability. For example, when the fog concentration is high (e.g., visibility <100m), α is increased (e.g., α = 0.8) to prioritize optimizing the dehazing loss and ensure image clarity; when the fog concentration is low, β is increased (e.g., β = 0.7) to enhance matching stability.

[0163] According to an embodiment of the present invention, the step of obtaining an optimized matching video segment set by retrieving the fog distribution map and the defogging optimization feature vector through a preset federated dynamic index includes:

[0164] The dynamic retrieval similarity threshold is determined based on the aforementioned fog distribution map;

[0165] The dehazing optimized feature vector is combined with the dynamic retrieval similarity threshold and processed through a preset federated dynamic index retrieval to obtain an optimized matching video segment set.

[0166] It should be noted that during the retrieval process, the feature matching threshold for regions with high fog concentration (e.g., pixel values ​​close to 255) is relaxed (e.g., the similarity threshold is reduced by 15%), while a strict threshold is applied to regions with low fog concentration (pixel values ​​close to 0). This balances the retrieval accuracy for different fog areas. Based on the defogging optimized feature vector, a coarse and fine retrieval process is performed using a dynamic index table according to different dynamic retrieval similarity thresholds to obtain an optimized matching video clip set. The clips are then sorted from high to low according to their matching scores, and the optimized matching video clip with the highest matching score is output to the user for display, thus becoming the retrieval result.

[0167] A third aspect of the present invention provides a readable storage medium storing a highway video retrieval method program based on feature matching, wherein when the highway video retrieval method program based on feature matching is executed by a processor, it implements the steps of the highway video retrieval method based on feature matching as described in any of the preceding claims.

[0168] The present invention discloses a highway video retrieval method and system based on feature matching. By using motion physical feature encoding, federated dynamic index retrieval, and fog adaptive processing, it solves the feature distortion caused by high-speed motion, improves the retrieval accuracy and efficiency in foggy scenes, and at the same time takes into account both retrieval efficiency and privacy protection.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0170] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0171] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0172] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0173] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A method for expressway video retrieval based on feature matching, characterized in that, The method comprises the following steps: Obtain motion physical feature evaluation data of continuous highway video frames and perform motion physical feature coding processing to obtain a motion physical feature vector; Obtain a user's query video, perform feature extraction to obtain a query motion physical feature vector, and then perform preset federated dynamic index retrieval processing to obtain an initial matching video segment set; Obtain fog concentration monitoring data and compare it with a preset fog concentration early warning monitoring threshold value; If the fog concentration monitoring data is less than or equal to the preset fog concentration early warning monitoring threshold value, arrange the initial matching video segment set in descending order of matching degree, and output the initial matching video segment with the highest matching degree to a user terminal for display; If the fog concentration monitoring data is greater than the preset fog concentration early warning monitoring threshold value, obtain a fog-containing video frame, combine the motion physical feature vector, and perform preset fog self-adaptive processing to obtain a de-fogging optimized feature vector and a fog distribution map; Perform preset federated dynamic index retrieval processing on the fog distribution map and the de-fogging optimized feature vector to obtain an optimized matching video segment set; Arrange the optimized matching video segment set in descending order of matching degree, and output the optimized matching video segment with the highest matching degree to a user terminal for display; The method for obtaining motion physical feature evaluation data of continuous highway video frames and performing motion physical feature coding processing to obtain a motion physical feature vector comprises: Obtain motion physical feature evaluation data from continuous highway video frames, including video frames and radar speed data and vehicle body parameter data in a preset time period; The vehicle body parameter data includes vehicle body length, vehicle body width, and aerodynamic shape characteristic parameters; Preprocess the video frames and radar speed data to obtain corrected video frames and smoothed speed data; Query a preset vehicle speed and pressure coefficient mapping table according to the smoothed speed data to obtain a corresponding pressure coefficient; Process the corrected video frames according to the pressure coefficient and the vehicle body length, vehicle body width, and aerodynamic shape characteristic parameters to obtain pixel displacement prediction values; Perform FFT transformation and phase correction processing on the corrected video frames and smoothed speed data to obtain texture frequency energy spectrum; Perform motion physical feature coding processing on the pixel displacement prediction values and the texture frequency energy spectrum to obtain a motion physical feature vector; Further comprising: Obtain feature gradients of the motion physical feature vector of the edge node; Perform federated average aggregation processing on the feature gradients to obtain an aggregated feature gradient; Update a preset federated shared model according to the aggregated feature gradient to generate a dynamic index table.

2. The feature matching-based motorway video retrieval method according to claim 1, characterized in that, The method for obtaining a user's query video, performing feature extraction to obtain a query motion physical feature vector, and then performing preset federated dynamic index retrieval processing to obtain an initial matching video segment set comprises: Obtain a user's query video and perform feature extraction to obtain a query motion physical feature vector; Perform dimension reduction processing on the query motion physical feature vector to obtain a dimension-reduced query feature vector; Perform hash calculation on the dimension-reduced query feature vector and combine a preset hash table for matching to obtain a coarse matching video segment set; The rough matching video segment set and the query motion physical feature vector are processed through a dynamic index table to obtain an initial matching video segment set.

3. The feature matching-based motorway video retrieval method according to claim 2, characterized in that, If the value is greater than a preset fog concentration early warning monitoring threshold, a fog-containing video frame is obtained, and the motion physical feature vector is processed through a preset fog cluster adaptive processing method to obtain a defogging optimization feature vector and a fog cluster distribution map, including: If the value is greater than a preset fog concentration early warning monitoring threshold, a fog-containing video frame is obtained; The fog-containing video frame is processed through a preset defogging and feature joint network to obtain a defogging video frame, and feature extraction is performed to obtain a defogging feature vector; The defogging feature vector and the motion physical feature vector are processed through feature alignment to obtain a defogging optimization feature vector; The fog-containing video frame is input into a preset fog cluster distribution map generation model for processing to obtain an initial fog cluster distribution map; The defogging video frame and the initial fog cluster distribution map are compared in similarity to obtain an image similarity value; If the value is greater than or equal to a preset image similarity threshold, the initial fog cluster distribution map is determined to be a fog cluster distribution map; Otherwise, the parameters of the preset fog cluster distribution map generation model are adjusted.

4. The feature matching-based motorway video retrieval method according to claim 3, characterized in that, Further comprising: The fog cluster concentration monitoring data is compared with a preset fog cluster concentration level threshold, and the fog cluster concentration level is determined according to the range, including first, second, third, fourth or fifth levels; According to the fog cluster concentration level, a preset defogging adaptive weight value list is queried to obtain a defogging adaptive weight value; The preset defogging and feature joint network is optimized according to the defogging adaptive weight value. 5.The method of claim 4, wherein, The fog cluster distribution map and the defogging optimization feature vector are processed through a preset federal dynamic index retrieval to obtain an optimized matching video segment set, including: A dynamic retrieval similarity threshold is determined according to the fog cluster distribution map; The defogging optimization feature vector is processed through a preset federal dynamic index retrieval in combination with the dynamic retrieval similarity threshold to obtain an optimized matching video segment set.

6. A highway video retrieval system based on feature matching, characterized by, The memory includes a highway video retrieval method based on feature matching program, and the highway video retrieval method based on feature matching program is executed by the processor to implement the following steps: Motion physical feature evaluation data of continuous highway video frames are obtained, and motion physical feature coding processing is performed to obtain a motion physical feature vector; A user's query video is obtained, and feature extraction is performed to obtain a query motion physical feature vector, and an initial matching video segment set is obtained through a preset federal dynamic index retrieval; Fog cluster concentration monitoring data are obtained, and a threshold comparison is performed with a preset fog concentration early warning monitoring threshold; If the value is less than or equal to a preset fog concentration early warning monitoring threshold, the initial matching video segment set is arranged in descending order of matching degree, and the initial matching video segment with the highest matching degree is output to a user terminal for display; If the value is greater than a preset fog concentration early warning monitoring threshold, a fog-containing video frame is obtained, and the motion physical feature vector is processed through a preset fog cluster adaptive processing method to obtain a defogging optimization feature vector and a fog cluster distribution map; The group fog distribution map and the defogging optimization feature vector are subjected to preset federation dynamic index retrieval processing to obtain an optimized matching video segment set; The optimized matching video segment set is arranged in descending order of matching degree, and the optimized matching video segment with the highest matching degree is output to a user terminal for display; The motion physical feature evaluation data of the continuous highway video frames are obtained, and motion physical feature coding processing is performed to obtain a motion physical feature vector, including: The motion physical feature evaluation data are obtained from continuous highway video frames, including video frames and radar speed measurement data and vehicle body parameter data in a preset time period; The vehicle body parameter data include vehicle body length, vehicle body width, and aerodynamic shape feature parameters; The video frames and the radar speed measurement data are preprocessed to obtain corrected video frames and smoothed vehicle speed data; A preset vehicle speed and pressure coefficient mapping relationship table is queried according to the smoothed vehicle speed data to obtain a corresponding pressure coefficient; The corrected video frames are processed according to the pressure coefficient and the vehicle body length, the vehicle body width, and the aerodynamic shape feature parameters to obtain a pixel displacement prediction value; FFT transformation and phase correction processing are performed on the corrected video frames and the smoothed vehicle speed data to obtain a texture frequency energy spectrum; Motion physical feature coding processing is performed on the pixel displacement prediction value and the texture frequency energy spectrum to obtain a motion physical feature vector; Further comprising: A feature gradient of the motion physical feature vector of the edge node is obtained; The feature gradient is subjected to federation average aggregation processing to obtain an aggregated feature gradient; A preset federation shared model is updated according to the aggregated feature gradient to generate a dynamic index table.

Images