Video trajectory positioning method, device, computer equipment and storage medium

Abstract

The present application relates to a video trajectory positioning method, device, computer equipment and storage medium. The method makes full use of the spatiotemporal context information contained between consecutive image frames in a video. By establishing motion constraints and spatiotemporal dependency constraints between adjacent image frames, the method can effectively eliminate the interference of outliers with large errors in the matching sequence, thereby reconstructing a more consistent and accurate video spatiotemporal trajectory from potential candidate matching sequences.

Description

Technical Field

[0001] The present application relates to the technical field of video trajectory positioning, and in particular to a video trajectory positioning method, device, computer equipment and storage medium. Background Art

[0002] In modern urban environments, due to the wide coverage of street view data and the popularity of video data, how to accurately extract and reconstruct the camera's motion trajectory from these data has become an important research topic. Traditional trajectory localization methods often rely on specific sensors or external devices, such as GPS signals. However, in some environments (such as urban canyons or indoors), these signals may be interfered with or unavailable. Therefore, it is of great significance to develop a trajectory localization method that does not rely on external sensors. Summary of the invention

[0003] Based on this, it is necessary to provide a video trajectory positioning method, device, computer equipment and storage medium that are independent of external sensors to address the above technical problems.

[0004] A video trajectory positioning method, the method comprising:

[0005] Reprojecting and cropping the pre-collected street view panoramic image into street view perspective images of different viewing angles, inputting the street view perspective images into a pre-trained ViT encoder to extract scene visual features, and constructing a street view feature library according to the scene visual features;

[0006] Extracting key frames from the video based on a spatial kernel density analysis method, inputting the key frames into the ViT encoder to obtain video scene visual features, performing similarity matching between the video scene visual features and scene visual features in the street view feature library, and obtaining a candidate street view perspective image;

[0007] Access the remaining frames in the video in sequence according to the time difference with the key frame, and obtain the motion relationship constraint of the shooting positions of any two frames in the video according to the constant speed motion model;

[0008] Taking the candidate street view perspective image corresponding to the currently accessed frame as the center, if the distance between the candidate street view perspective image and the successfully matched frame is less than the set radius, the candidate street view perspective image with the highest similarity in the candidate street view perspective image is taken as the video frame matching result to generate the video trajectory.

[0009] In one of the embodiments, it also includes: constructing training samples; the training samples include: a target image, a positive sample and a negative sample; based on the semantic prior information of the training samples and the attention layer in the ViT encoder, filtering out key local features from the feature map output by the encoder, performing feature aggregation to obtain scene visual features; and performing supervised learning through the Triplet Loss function until the target image of the training sample is closer to the positive sample and farther away from the negative sample in the feature space.

[0010] In one embodiment, the method further includes: dividing the video into video clips; calculate the Top- The kernel density of the candidate street scene points is estimated, and the video frame corresponding to the candidate street scene point with the highest kernel density value is used as the key frame.

[0011] In one embodiment, the method further includes: sequentially calculating the Top- The kernel density estimate of the candidate street attractions is:

[0012] ;

[0013] in, is the spatial kernel density estimate of the current video frame, is the search radius, represents the candidate street attractions within the search radius, is the feature similarity between the candidate street attractions within the search radius and the video frame, It is the geographic distance between the candidate points within the search radius and the current point.

[0014] In one embodiment, the method further includes: sequentially accessing the remaining frames in the video according to the time difference between the frames and the key frame, and obtaining, based on a constant speed motion model, the motion relationship constraint between the shooting positions of any two frames in the video as follows:

[0015] ;

[0016] Represents a video frame The distance between Represents a video frame The time difference between Represents the velocity of the constant speed motion model.

[0017] A video track positioning device, the device comprising:

[0018] A street view feature library construction module is used to reproject and crop the pre-collected street view panoramic images into street view perspective images of different viewing angles, input the street view perspective images into a pre-trained ViT encoder to extract scene visual features, and construct a street view feature library based on the scene visual features;

[0019] A matching module, used for extracting key frames from the video based on a spatial kernel density analysis method, inputting the key frames into the ViT encoder to obtain video scene visual features, and performing similarity matching between the video scene visual features and scene visual features in the street view feature library to obtain a candidate street view perspective image;

[0020] The trajectory generation module is used to sequentially access the remaining frames in the video according to the time difference with the key frame, and obtain the motion relationship constraints of the shooting positions of any two frames in the video according to the constant speed motion model; taking the candidate street view perspective image corresponding to the current accessed frame as the center, if the distance between the candidate street view perspective image and the successfully matched frame is less than the set radius, the candidate street view perspective image with the highest similarity in the candidate street view perspective image and the successfully matched radius is used as the video frame matching result, so as to generate the video trajectory.

[0021] In one of the embodiments, a model training module is used to construct training samples; the training samples include: a target image, a positive sample and a negative sample; based on the semantic prior information of the training samples and the attention layer in the ViT encoder, key local features are screened out from the feature map output by the encoder, and feature aggregation is performed to obtain scene visual features; supervised learning is performed through the Triplet Loss function until the target image of the training sample is closer to the positive sample and away from the negative sample in the feature space.

[0022] In one embodiment, the matching module is further configured to divide the video into video clips; calculate the Top- The kernel density of the candidate street scene points is estimated, and the video frame corresponding to the candidate street scene point with the highest kernel density value is used as the key frame.

[0023] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0024] Reprojecting and cropping the pre-collected street view panoramic image into street view perspective images of different viewing angles, inputting the street view perspective images into a pre-trained ViT encoder to extract scene visual features, and constructing a street view feature library according to the scene visual features;

[0025] Extracting key frames from the video based on a spatial kernel density analysis method, inputting the key frames into the ViT encoder to obtain video scene visual features, performing similarity matching between the video scene visual features and scene visual features in the street view feature library, and obtaining a candidate street view perspective image;

[0026] Access the remaining frames in the video in sequence according to the time difference with the key frame, and obtain the motion relationship constraint of the shooting positions of any two frames in the video according to the constant speed motion model;

[0027] Taking the candidate street view perspective image corresponding to the currently accessed frame as the center, if the distance between the candidate street view perspective image and the successfully matched frame is less than the set radius, the candidate street view perspective image with the highest similarity in the candidate street view perspective image is taken as the video frame matching result to generate the video trajectory.

[0028] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0029] Reprojecting and cropping the pre-collected street view panoramic image into street view perspective images of different viewing angles, inputting the street view perspective images into a pre-trained ViT encoder to extract scene visual features, and constructing a street view feature library according to the scene visual features;

[0030] Extracting key frames from the video based on a spatial kernel density analysis method, inputting the key frames into the ViT encoder to obtain video scene visual features, performing similarity matching between the video scene visual features and scene visual features in the street view feature library, and obtaining a candidate street view perspective image;

[0031] Access the remaining frames in the video in sequence according to the time difference with the key frame, and obtain the motion relationship constraint of the shooting positions of any two frames in the video according to the constant speed motion model;

[0032] Taking the candidate street view perspective image corresponding to the currently accessed frame as the center, if the distance between the candidate street view perspective image and the successfully matched frame is less than the set radius, the candidate street view perspective image with the highest similarity in the candidate street view perspective image is taken as the video frame matching result to generate the video trajectory.

[0033] The above-mentioned video trajectory positioning method, device, computer equipment and storage medium make full use of the spatiotemporal context information contained between consecutive image frames in the video. By establishing motion constraints and spatiotemporal dependency constraints between adjacent image frames, this method can effectively eliminate the interference of outliers with large errors in the matching sequence, thereby reconstructing a more consistent and accurate video spatiotemporal trajectory from potential candidate matching sequences. The advantage of the present invention is that it improves the consistency and overall accuracy of trajectory reconstruction, so that high-precision trajectory positioning can be achieved even in complex urban environments. This not only has broad application prospects in the fields of autonomous driving, intelligent transportation, urban planning, etc., but also provides new ideas and technical support for the further development of vision-based geolocation technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of a flow chart of a video trajectory positioning method in one embodiment;

[0035] Figure 2 A flowchart of video trajectory positioning in one embodiment;

[0036] Figure 3 A schematic diagram of a key frame recognition process based on spatial kernel density analysis in one embodiment;

[0037] Figure 4 Schematic diagram of a spatiotemporal trajectory reconstruction process based on a constant speed motion model in one embodiment;

[0038] Figure 5 is a structural block diagram of a video track positioning device in one embodiment;

[0039] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0041] In one embodiment, Figure 1 As shown, a frequency trajectory positioning method is provided, comprising the following steps:

[0042] Step 102, reprojecting and cropping the pre-collected street view panoramic image into street view perspective images of different viewing angles, inputting the street view perspective images into a pre-trained ViT encoder to extract scene visual features, and constructing a street view feature library based on the scene visual features.

[0043] In this step, the street view feature library of the study area is finally generated by collecting and reprojecting the street view panoramic images in the study area.

[0044] Step 104, extract key frames from the video based on the spatial kernel density analysis method, input the key frames into the ViT encoder to obtain the video scene visual features, perform similarity matching between the video scene visual features and the scene visual features in the street view feature library, and obtain a candidate street view perspective image.

[0045] Keyframe recognition based on spatial kernel density analysis aims to determine the keyframe that is most likely to be the correct match based on the geographic spatial location distribution and feature similarity of the candidate retrieval results.

[0046] Step 106, access the remaining frames in the video in sequence according to the time difference with the key frame, and obtain the motion relationship constraint of the shooting positions of any two frames in the video based on the constant speed motion model.

[0047] Step 108, taking the candidate street view perspective image corresponding to the currently accessed frame as the center, if the distance between the candidate street view perspective image and the successfully matched frame is less than the set radius, the candidate street view perspective image with the highest similarity in the candidate street view perspective image that successfully matches the frame is used as the video frame matching result, thereby generating a video trajectory.

[0048] In the above-mentioned video trajectory positioning method, full use is made of the spatiotemporal context information contained between consecutive image frames in the video. By establishing motion constraints and spatiotemporal dependency constraints between adjacent image frames, this method can effectively eliminate the interference of outliers with large errors in the matching sequence, thereby reconstructing a more consistent and accurate video spatiotemporal trajectory from potential candidate matching sequences. The advantage of the present invention is that the consistency and overall accuracy of trajectory reconstruction are improved, so that high-precision trajectory positioning can be achieved even in complex urban environments. This not only has broad application prospects in the fields of autonomous driving, intelligent transportation, urban planning, etc., but also provides new ideas and technical support for the further development of vision-based geolocation technology.

[0049] In one embodiment, a training sample is constructed; the training sample includes: a target image, a positive sample, and a negative sample; based on the semantic prior information of the training sample and the attention layer in the ViT encoder, key local features are screened out from the feature map output by the encoder, and feature aggregation is performed to obtain scene visual features;

[0050] Supervised learning is performed through the Triplet Loss function until the target image of the training sample is closer to the positive sample and farther away from the negative sample in the feature space.

[0051] Specifically, Figure 2As shown in the figure, in the training stage, training samples are first constructed according to the training data set, and the training samples are input into the ViT encoder to learn the scene visual feature representation. Then, according to the semantic prior information of the training samples and the attention layer in ViT, key local features are screened out from the feature map output by the encoder, and feature aggregation is performed to obtain the scene visual features. Finally, the triplet loss function is used for supervision, so that the target image of the triplet training sample is closer to the positive sample and away from the negative sample in the feature space. In the inference stage, the street view data in the study area is first preprocessed, and the street view panoramic image is reprojected and cropped into street view perspective images of different viewing angles, which are then input into the pre-trained ViT encoder to extract the scene visual features and build a street view feature library. When performing online inference, key frame images are first extracted from the query video and sent to the encoder to extract the scene visual features. Then, the feature similarity between the key frame image and the street view image is calculated in the feature space, and the most similar previous image of each key frame image is returned. The video spatiotemporal trajectory reconstruction algorithm is used to estimate the spatiotemporal trajectory of the camera motion from the Top-K candidate results.

[0052] In one embodiment, the video is divided into video clips; calculate the Top- The kernel density of the candidate street attractions is estimated, and the video frame corresponding to the candidate street attraction with the highest kernel density value is used as the key frame. It is worth noting that the above process is based on the assumption that the feature similarity of the correct matching results of the video frames in the video clip is higher and the distribution pattern of spatial agglomeration is presented geographically.

[0053] Specifically, Figure 3 As shown, the Top- The kernel density estimate of the candidate street attractions is:

[0054] ;

[0055] in, is the spatial kernel density estimate of the current video frame, is the search radius, represents the candidate street attractions within the search radius, is the feature similarity between the candidate street attractions within the search radius and the video frame, is the geographic distance between the candidate point within the search radius and the current point. The Top- The kernel density of the candidate street scene points is estimated, and the video frame corresponding to the candidate street scene point with the highest kernel density value is used as the key frame.

[0056] In another embodiment, the remaining frames in the video are accessed in sequence according to the time difference with the key frame, and according to the constant speed motion model, the motion relationship constraint between the shooting positions of any two frames in the video is obtained as follows:

[0057] ;

[0058] Represents a video frame The distance between Represents a video frame The time difference between Represents the speed of the constant speed motion model. Figure 4 As shown, based on the motion constraint relationship between any two frames of images, first find the video frame that is currently being accessed The closest successfully matched frame in time , calculate the time difference between the two , and then A set of candidate street attractions Centered on is the search radius. If the candidate street scene successfully matches the frame The distance between If the search radius is smaller than the search radius, the candidate street point is retained; otherwise, the candidate street point is removed from the candidate results. The candidate street point with the highest feature similarity is selected from the candidate results that meet the motion constraint conditions as the matching result of the currently accessed video frame. If all candidate results of the video frame do not meet the motion constraint relationship with the nearest neighbor successfully matched frame, the frame is marked as a wrong match and the spatial position of the frame is generated by interpolation.

[0059] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0060] In one embodiment, Figure 5 As shown, a video trajectory positioning device is provided, including: a street view feature library construction module 502, a matching module 504 and a trajectory generation module 506, wherein:

[0061] A street view feature library construction module 502 is used to reproject and crop the pre-collected street view panoramic image into street view perspective images of different viewing angles, input the street view perspective images into a pre-trained ViT encoder to extract scene visual features, and construct a street view feature library according to the scene visual features;

[0062] A matching module 504 is used to extract key frames from the video based on a spatial kernel density analysis method, input the key frames into the ViT encoder to obtain video scene visual features, and perform similarity matching between the video scene visual features and scene visual features in the street view feature library to obtain a candidate street view perspective image;

[0063] The trajectory generation module 506 is used to sequentially access the remaining frames in the video according to the time difference with the key frame, and obtain the motion relationship constraints of the shooting positions of any two frames in the video according to the constant speed motion model; take the candidate street view perspective image corresponding to the currently accessed frame as the center, if the distance between the candidate street view perspective image and the successfully matched frame is less than the set radius, then take the candidate street view perspective image with the highest similarity in the candidate street view perspective image and the successfully matched radius as the video frame matching result, so as to generate the video trajectory.

[0064] In one of the embodiments, it also includes: a model training module, used to construct training samples; the training samples include: a target image, a positive sample and a negative sample; according to the semantic prior information of the training samples and the attention layer in the ViT encoder, key local features are screened out from the feature map output by the encoder, and feature aggregation is performed to obtain scene visual features; supervised learning is performed through the Triplet Loss loss function until the target image of the training sample is closer to the positive sample and away from the negative sample in the feature space.

[0065] In one embodiment, the matching module 504 is further configured to divide the video into video clips; calculate the Top- The kernel density of the candidate street scene points is estimated, and the video frame corresponding to the candidate street scene point with the highest kernel density value is used as the key frame.

[0066] In one embodiment, the matching module 504 is further configured to sequentially calculate the top- The kernel density estimate of the candidate street attractions is:

[0067] ;

[0068] in, is the spatial kernel density estimate of the current video frame, is the search radius, represents the candidate street attractions within the search radius, is the feature similarity between the candidate street attractions within the search radius and the video frame, It is the geographic distance between the candidate points within the search radius and the current point.

[0069] In one embodiment, the trajectory generation module 506 is further used to sequentially access the remaining frames in the video according to the time difference with the key frame, and obtain the motion relationship constraint between the shooting positions of any two frames of the video according to the constant speed motion model:

[0070] ;

[0071] Represents a video frame The distance between Represents a video frame The time difference between Represents the velocity of the constant speed motion model.

[0072] For the specific definition of the video trajectory positioning device, please refer to the definition of the video trajectory positioning method above, which will not be repeated here. Each module in the above-mentioned video trajectory positioning device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0073] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a video track positioning method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0074] Those skilled in the art will understand that Figure 6The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0075] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0076] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0077] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0078] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A video trajectory positioning method, characterized in that: The method comprises: Reprojecting and cropping the pre-collected street view panoramic image into street view perspective images of different viewing angles, inputting the street view perspective images into a pre-trained ViT encoder to extract scene visual features, and constructing a street view feature library according to the scene visual features; Extracting key frames from the video based on a spatial kernel density analysis method, inputting the key frames into the ViT encoder to obtain video scene visual features, performing similarity matching between the video scene visual features and scene visual features in the street view feature library, and obtaining a candidate street view perspective image; Access the remaining frames in the video in sequence according to the time difference with the key frame, and obtain the motion relationship constraint of the shooting positions of any two frames in the video according to the constant speed motion model; Taking the candidate street view perspective image corresponding to the current access frame as the center, if the distance between the candidate street view perspective image and the successfully matched frame is less than the set radius, the candidate street view perspective image with the highest radius similarity among the candidate street view perspective images is used as the video frame matching result, so as to generate a video trajectory; The remaining frames in the video are accessed in sequence according to the time difference with the key frame, and the motion relationship constraints of the shooting positions of any two frames in the video are obtained according to the constant speed motion model, including: The remaining frames in the video are accessed in sequence according to the time difference with the key frame, and according to the constant speed motion model, the motion relationship constraint of the shooting positions of any two frames in the video is obtained as follows: Represents a video frame The distance between Represents a video frame The time difference between Represents the velocity of the constant speed motion model.

2. The method according to claim 1, characterized in that The steps to train the ViT encoder include: Constructing training samples; the training samples include: target images, positive samples and negative samples; According to the semantic prior information of the training samples and the attention layer in the ViT encoder, key local features are selected from the feature map output by the encoder, and feature aggregation is performed to obtain the scene visual features; Supervised learning is performed through the Triplet Loss function until the target image of the training sample is closer to the positive sample and farther away from the negative sample in the feature space.

3. The method according to claim 1, characterized in that Extract key frames from the video based on spatial kernel density analysis, including: Divide the video into video clips; Calculate the Top- The kernel density of the candidate street scene points is estimated, and the video frame corresponding to the candidate street scene point with the highest kernel density value is used as the key frame.

4. The method according to claim 3, characterized in that Calculate the Top- Kernel density estimates of candidate street attractions, including: Calculate the Top- The kernel density estimate of the candidate street attractions is: in, is the spatial kernel density estimate of the current video frame, is the search radius, represents the candidate street attractions within the search radius, is the feature similarity between the candidate street attractions within the search radius and the video frame, It is the geographic distance between the candidate points within the search radius and the current point.

5. A video trajectory positioning device, characterized in that: The device comprises: A street view feature library construction module is used to reproject and crop the pre-collected street view panoramic images into street view perspective images of different viewing angles, input the street view perspective images into a pre-trained ViT encoder to extract scene visual features, and construct a street view feature library based on the scene visual features; A matching module, used for extracting key frames from the video based on a spatial kernel density analysis method, inputting the key frames into the ViT encoder to obtain video scene visual features, and performing similarity matching between the video scene visual features and scene visual features in the street view feature library to obtain a candidate street view perspective image; The trajectory generation module is used to sequentially access the remaining frames in the video according to the time difference with the key frame, and obtain the motion relationship constraints of the shooting positions of any two frames in the video according to the constant speed motion model; taking the candidate street view perspective image corresponding to the currently accessed frame as the center, if the distance between the candidate street view perspective image and the successfully matched frame is less than a set radius, the candidate street view perspective image with the highest similarity in the candidate street view perspective image and the successfully matched radius is used as the video frame matching result, thereby generating a video trajectory; The trajectory generation module is also used to sequentially access the remaining frames in the video according to the time difference with the key frame, and obtain the motion relationship constraint between the shooting positions of any two frames in the video according to the constant speed motion model: Represents a video frame The distance between Represents a video frame The time difference between Represents the velocity of the constant speed motion model.

6. The device according to claim 5, characterized in that Also includes: Model training module, used to construct training samples; The training samples include: target images, positive samples and negative samples; According to the semantic prior information of the training samples and the attention layer in the ViT encoder, key local features are screened out from the feature map output by the encoder, and feature aggregation is performed to obtain the scene visual features. Supervised learning is performed through the Triplet Loss function until the target image of the training sample is closer to the positive sample and away from the negative sample in the feature space.

7. The device according to claim 5, characterized in that The matching module is also used to divide the video into video clips; calculate the Top- The kernel density of the candidate street scene points is estimated, and the video frame corresponding to the candidate street scene point with the highest kernel density value is used as the key frame.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images