Video query method based on visual large model, electronic device, and medium

Abstract

The application discloses a video query method based on a visual large model, an electronic device and a medium, and the method comprises the following steps: performing value evaluation on each video frame in an original video stream, setting a value score threshold, regarding a video frame with a value score greater than the value score threshold as a high-value video frame, and regarding a video frame with a value score less than the value score threshold as a low-value video frame; storing the preprocessed high-value video frame and the low-value video frame in a structured video database; in response to a user natural language query, a large language model compiles the user natural language query into a structured query specification representation; converting the structured query specification representation into a directed acyclic graph, determining the query execution order of each operator based on the directed acyclic graph; obtaining a plurality of candidate video clips; and based on a context multi-armed bandit algorithm, scheduling the query plan of each operator to be executed on the corresponding candidate video clip, so as to obtain a video query result.

Description

Technical Field

[0001] This invention belongs to the field of video data processing and database technology, and particularly relates to a video query method, electronic device, and medium based on a large visual model. Background Technology

[0002] With the widespread adoption of internet video platforms, intelligent surveillance, vehicle dashcams, autonomous driving, and industrial inspection, video data is growing exponentially, becoming one of the most important forms of "spatiotemporal big data" besides structured data. There is an urgent need for intelligent systems capable of automatically analyzing and understanding video content to support various business scenarios such as retrieval, monitoring and early warning, behavioral analysis, and decision support.

[0003] In recent years, multimodal large models and video-language large models (VideoLLM / VLM) have developed rapidly. Examples include the VideoLLM framework, CogVLM2, and VideoLLM-online. By encodeing video sequences and then using large language models for inference, these models demonstrate excellent multi-granularity and cross-temporal reasoning capabilities in tasks such as video question answering, description, and temporal localization. Research shows that VideoLLM already possesses extremely strong understanding and reasoning capabilities even for scenarios involving only a single video and a single question.

[0004] However, truly applying VideoLLM's capabilities to querying and analyzing "large-scale video databases" at the TB or even PB level faces several challenges, including the conflict between expressive power and computational cost, and the tension between complex semantics and indexable structures. Current technologies attempt to address these issues through several methods, but all have significant limitations: 1. Full-scale video-language model scanning approach: This approach slices the entire video library into fixed time windows and calls the video-language model to evaluate each video slice to confirm whether it meets the query conditions. Its drawbacks are: while this approach leverages the model's zero-shot inference capabilities to achieve flexible semantic expression, its computational cost is approximately linearly related to the total video duration. When dealing with large-scale video libraries, a single query may trigger a massive number of model calls, resulting in extremely high GPU computing power costs and unacceptable query latency. Furthermore, this approach fails to utilize the highly redundant characteristics of video data (such as long periods of empty scenes and static backgrounds), leading to a significant waste of computational resources.

[0005] 2. Vector retrieval schemes based on lightweight visual models or CLIPs: This scheme uses lightweight models to generate feature vectors from video frames or slices and constructs vector indexes, retrieving candidate segments by calculating similarity. Its drawbacks are: while fast and simple in structure, it struggles to handle complex semantics and temporal logic. This scheme is prone to obfuscation of fine-grained semantics, and its accuracy is low for complex queries involving causal or temporal relationships, such as "a vehicle suddenly brakes while changing lanes, causing a following vehicle to swerve," as well as for identifying rare events or subjective semantics, failing to meet the needs of in-depth video analysis.

[0006] 3. Video database methods based on lightweight proxy models and operator optimization: These methods (such as NoScope, BlazeIt, SVQ, etc.) attempt to use inexpensive proxy models (such as differential detectors, specialized small models) for pre-filtering, only calling the heavy model on frames that pass the filter, or accelerating the model internally at the operator level. Their drawbacks are: First, the supported predicate expressions are relatively rigid. Most of these methods only support simple queries such as "does a certain type of target exist?" or "simple action counting," failing to cover the fine-grained semantics and open reasoning that VideoLLM excels at. Second, there is a lack of a unified cost model and scheduling strategy. Existing methods mostly treat deep models as "black box operators" for local optimization, lacking a complete mechanism to treat VideoLLM as a "scarce computing resource" for global scheduling. Specifically, this manifests as: a lack of a unified data model and statistical information; a lack of a cost estimation model for the number of VideoLLM calls; and a lack of a dynamic scheduling strategy that can adaptively determine "how many times to call the model and which segments to verify" based on the semantic difficulty of the query and data distribution. Third, there is a lack of coordination between offline and online processing. Most existing systems perform localized optimizations at the "model level" or "execution level," lacking a systematic offline-online collaborative design. For example, they fail to pre-digest video redundancy through value-aware indexing in the offline phase, and they fail to fully utilize the different roles of VideoLLM in "understanding query semantics" and "verifying video segments" for budget allocation in the online phase.

[0007] In summary, existing video analytics and retrieval technologies are either accurate but extremely expensive (full scan), efficient but limited in semantic capabilities (vector retrieval), or only perform micro-optimizations at the operator level (surrogate model). The industry urgently needs a unified solution that can support complex natural language queries on large-scale video databases while effectively controlling inference costs through a collaborative mechanism of offline indexing and online scheduling. Summary of the Invention

[0008] To address the shortcomings of existing technologies, embodiments of the present invention provide a video query method, electronic device, and medium based on a large visual model.

[0009] In a first aspect, embodiments of the present invention provide a video query method based on a large visual model, the method comprising the following steps: Each video frame in the original video stream is evaluated for value. A value score threshold is set, and video frames with a value score greater than the value score threshold are considered high-value video frames, while those with a value score less than the threshold are considered low-value video frames. The pre-processed high-value and low-value video frames are then stored in a structured video database. In response to user natural language queries, the large language model compiles them into a structured query specification representation; The structured query canonical representation is converted into a directed acyclic graph, and the query execution order of each operator is determined based on the directed acyclic graph; Several candidate video segments are obtained; the query plan of each operator is scheduled to be executed on its corresponding candidate video segment based on the context multi-armed slot machine algorithm, thereby obtaining the video query results.

[0010] In a second aspect, embodiments of the present invention provide an electronic device, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the video query method based on the visual big model described above.

[0011] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described video query method based on a large visual model.

[0012] Fourthly, embodiments of the present invention provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the aforementioned video query method based on a large visual model.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention adopts an offline-online collaborative architecture, which significantly reduces model call costs and improves query efficiency. Specifically, this invention evaluates the value of each video frame in the original video stream in the offline stage, pre-eliminating a large number of redundant frames with low information and building an index only for high-value frames; and combines large model compilation and cost-driven optimization in the online stage, calling the expensive video-language model only on demand in the compressed candidate video segments. This collaborative mechanism overcomes the problem of excessively high computational cost of traditional full-scan schemes, and significantly reduces the number of model calls while maintaining near-full-scan retrieval accuracy, thus significantly reducing GPU computing power costs and interactive query latency.

[0014] (2) This invention assigns a weighted value score to each video frame based on motion intensity, object density, and visual entropy. This value scoring strategy is independent of the specific query content and can objectively evaluate the information content of video frames and filter out low-value frames. Compared with the fixed-interval sampling or simple difference detection commonly used in the prior art, the method of this invention is more selective and information-sensitive to video content, can more accurately retain key information, and reduce the waste of storage and computing resources.

[0015] (3) This invention designs a scheduling strategy based on a context-based multi-armed slot machine algorithm, which can dynamically determine the call budget (i.e., how many segments to verify) of the video-language model according to the difficulty of the current query (such as text ambiguity) and the distribution characteristics of candidate data (such as the degree of similarity dispersion). This strategy enables the system to automatically save budget under simple queries and automatically improve recall under complex long-tail queries, thereby achieving the best retrieval effect under the constraint of limited computing resources. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of a video query method based on a large visual model provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the user query parsing process provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0020] like Figure 1 As shown, this embodiment of the invention provides a video query method based on a large visual model, the method comprising the following steps: Step S1, offline processing stage: value evaluation is performed on each video frame in the original video stream, a value score threshold is set, and video frames with a value score greater than the value score threshold are regarded as high-value video frames, and vice versa as low-value video frames; the pre-processed high-value video frames and low-value video frames are stored in the structured video database. In addition, several candidate video clips are obtained.

[0021] Specifically, step S1 includes the following sub-steps: Step S101: Evaluate the value of each video frame in the original video stream. The value score of each video frame is a weighted sum of motion intensity, object density, and visual entropy. The expression is as follows: In the formula, This represents the value score corresponding to the current video frame f. This indicates the motion intensity corresponding to the current video frame f. This represents the object density corresponding to the current video frame f. This represents the visual entropy corresponding to the current video frame f. This represents hyperparameters.

[0022] Furthermore, the motion intensity of each video frame is calculated as follows: the average amplitude of the dense optical flow between the current video frame and its adjacent video frames is calculated to obtain the motion intensity corresponding to the current video frame; in this example, the average amplitude is used to quantify the degree of pixel change between video frames, and the higher the motion intensity, the richer the dynamics of the picture.

[0023] The process of calculating the object density of each video frame is as follows: input the current video frame into a lightweight object detection model (e.g., YOLO v10 nano) to obtain the region of interest (ROI) and / or the bounding box of the target. Calculate the ratio of the area of ​​all ROIs and / or the bounding box of the target to the total area of ​​the current video frame to obtain the object density of the current video frame. The calculation process for the visual entropy of each video frame is as follows: the current video frame is converted into a grayscale image, the distribution histogram of grayscale values ​​in the grayscale image is statistically analyzed, and the information entropy of the distribution histogram is calculated based on Shannon entropy to obtain the visual entropy corresponding to the current video frame. The visual entropy is used to measure the texture richness or disorder of the scene in the video frame, thereby distinguishing scenes with rich semantics from backgrounds with low information content (such as solid-color walls and skies).

[0024] Step S102: Set a value score threshold. Video frames with a value score greater than the value score threshold are considered high-value video frames, while those with a value score less than the value score threshold are considered low-value video frames.

[0025] Specifically, setting a value scoring threshold. ; If the current video frame f corresponds to the value score Greater than the value rating threshold Then the current video frame f is considered a high-value video frame; if the value score corresponding to the current video frame f is... Less than the value rating threshold If the current video frame f is not considered a low-value video frame, then the current video frame f will be treated as a low-value video frame.

[0026] Step S103: Store the preprocessed high-value video frames into a structured video database.

[0027] Specifically, step S103 includes: Using a multi-target tracking algorithm, based on the intersection-union ratio of the target bounding boxes and / or the similarity of appearance feature vectors, the targets detected in the current high-value video frame are associated and matched with the targets or existing trajectories in the previous video frames; A unified trajectory identifier is assigned to the successfully matched target, and the motion feature vector of the trajectory is calculated. The motion feature vector includes the direction and magnitude of velocity and acceleration, thereby generating target trajectory data that contains temporal continuity. Extract discrete text attribute features and high-dimensional continuous features of the target. The discrete text attribute features include at least object category labels and color attributes, and the high-dimensional continuous features include at least global semantic vectors and object appearance feature vectors. For the discrete text attribute features, an inverted index is constructed; for the high-dimensional continuous features, an approximate nearest neighbor vector index is constructed. The target trajectory data, the inverted index, and the approximate nearest neighbor vector index are stored in a structured video database.

[0028] Step S104: Store the preprocessed low-value video frames into a structured video database.

[0029] Specifically, step S104 includes: For each low-value video frame, establish the temporal location information of the low-value video frame and generate a reference pointer pointing to the nearest stored high-value key frame in the time dimension. The original pixel data of the low-value video frames are discarded, and the time position information and the reference pointer are stored as compressed low-value video frame records in the structured video database.

[0030] The structured video database includes: Frame Table: Used to store metadata for video frames. Each tuple contains (video_id, timestamp, global_clip_vec, value_score). Here, video_id is the ID of the video frame, timestamp is the timestamp of the video frame, global_clip_vec is the global semantic vector extracted using an image encoder (such as CLIP ViT-L / 14), and value_score is the value score of the video frame.

[0031] Object Feature Table: Each tuple contains (object_id, frame_id, bbox, class_label, conf, appearance_vec). Here, object_id represents the unique identifier ID assigned to each object, frame_id represents the ID number of the video frame, bbox is the normalized bounding box coordinates, class_label represents the category to which the object belongs, conf represents the confidence score of the object detector, and appearance_vec is fine-grained appearance features (such as ReID features) to support visual similarity search.

[0032] Track table: Used to aggregate consecutive instances of the same object over time. It contains (track_id, object_sequence, motion_feats, interaction_feats). Here, track_id represents the identifier ID of the track, object_sequence represents a set of appearances of the target at consecutive time points, motion_feats contains velocity and acceleration curves to answer queries such as "fast movement," and interaction_feats contains proximity information to other tracks.

[0033] Video Segment Table: Used to represent logical video segments. Fields include segment identifier, start and end time, list of key objects, and segment vector, etc. It can be used as the basic unit for retrieval and result display.

[0034] VideoStats: Used to maintain histograms of object category distribution and motion intensity distribution (average driving speed, acceleration distribution), and is used for cost estimation by the query optimizer.

[0035] Step S105: Obtain several candidate video segments.

[0036] The original video stream is processed by a lens detection algorithm to calculate the pixel histogram differences between video frame sequences in the original video stream. Based on the differences in the pixel histogram, the lens boundaries are determined, and the original video stream is divided into several physical lenses with consistent internal scenes, thereby obtaining several candidate video segments.

[0037] Step S2, Online Query Phase: In response to the user's natural language query, the large language model compiles it into a structured query canonical representation; the structured query canonical representation is converted into a directed acyclic graph, and the query execution order of each operator is determined based on the directed acyclic graph; the query plan of each operator is scheduled to be executed on its corresponding candidate video segment based on the context multi-armed slot machine algorithm, thereby obtaining the video query results.

[0038] Specifically, step S2 includes the following sub-steps: In step S201, in response to the user's natural language query, a large language model (such as LLaMA-3-8B) combined with few-shot prompting technology compiles the user's natural language query into a structured query specification representation (Query Spec) in JSON format.

[0039] For example, the prompt words of the large language model are set as follows: Query: Find a red sports car that is making a sharp left turn.

[0040] Large language model output: {"target": "Object", "filters": [{"attr": "class", "op": "eq", "val":"car"}, / / Attribute: Class, Operation: Equal to, Value: Car {"attr": "color", "op": "eq", "val": "red"} / / Attribute: Color, Operation: Equal to, Value: Red],"temporal": {"action": "turn_left", / / Action: Turn left "duration_min": 2.0 / / Minimum duration: 2.0 seconds},"llm_verify": "Is the cardriving aggressively or drifting?", / / Requires large model verification: Is the car being driven aggressively or drifting? "task_type": "retrieval" / / Task type: Retrieval } Here, `target` represents the query target (e.g., "Object"). `Filters` represents hard constraints that can be mapped to the inverted index (e.g., `class='car'`, `color='red'`). `temporal` represents spatiotemporal constraints that can be mapped to the trajectory table (e.g., `action='turn_left'`, `duration > 2s`). `llm_verify` extracts complex semantics that cannot be accurately determined by traditional indexes and must be verified by the VisionLLM (e.g., "aggressively", "near miss").

[0041] Step S202: Convert the structured query canonical representation into a directed acyclic graph, and determine the query execution order of each operator based on the directed acyclic graph.

[0042] Specifically, the structured query canonical representation is converted into a directed acyclic graph (DAG). Based on a preset cost model, the cost of different candidate physical execution paths in the directed acyclic graph is estimated; the cumulative cost of different candidate physical execution paths is compared, and the candidate physical execution path with the lowest cumulative cost is selected as the optimal physical execution path. The query execution order of each operator is determined based on the optimal physical execution path; wherein, operators with lower execution costs are configured to be executed first, and operators with higher execution costs are configured to be executed later.

[0043] The expression for calculating cumulative cost is as follows: In the formula, The input base is estimated using the VideoStats table. This represents the unit cost of the operator. Let P represent the i-th execution path, and let P represent the set of execution paths. This represents the cost of all execution paths. For example, for the query "red car turns left", comparing the cost of "index filtering first and then vector search" with "direct vector search", it is usually preferable to use highly selective lightweight operators (such as IndexScan and MotionFilter) for pre-pruning, and postpone the expensive LLM Merify operator to the end. Step S203: Based on the contextual multi-armed slot machine algorithm, the query plan of each operator is scheduled to be executed on its corresponding candidate video segment, thereby obtaining the video query result.

[0044] Specifically, the state vectors of all candidate video segments are extracted; wherein, the state vectors include: the embedding vector corresponding to the user's natural language query, the statistical distribution features of the similarity scores of the candidate video segments (mean, variance, and the score difference between Top-1 and Top-10), and the number of matches (estimated by the video state statistics table VideoStats). The state vector is input into a pre-trained policy network to obtain the probability distribution of action a; where each action a corresponds to a call budget value k of a large visual model, and the call budget value k represents the number of candidate video segments to be verified. The first k candidate video segments are input into the LLM Mercify operator, which then calls the large visual model to execute the query.

[0045] Furthermore, in this example, the large visual model adopts GPT-4V or LLaVA; for example, the JSON format output by the large visual model is as follows: {"match": true, "confidence": 0.9}, thereby reducing the cost of outputting the token and facilitating parsing.

[0046] Furthermore, in this example, the policy network The training process includes: using historical query logs, obtaining the "ground truth" by running VisionLLM on the full dataset, building a replay buffer, constructing a reward function to maximize retrieval accuracy while penalizing computational cost, and updating network parameters using the policy gradient algorithm to enable it to learn to distinguish between simple queries (assigning small k) and complex long-tail queries (assigning large k).

[0047] The expression for the reward function is as follows: In the formula, s represents the state and a represents the action. This represents the reward obtained by taking action a in state s. ∈[0,1] represents precision and recall, Represents the balance coefficient. This indicates the cost of calling a large visual model corresponding to action 'a'.

[0048] In summary, this invention provides a video query method based on a large visual model. In the offline stage, this invention evaluates the value of each video frame in the original video stream based on dimensions such as motion intensity, object density, and visual entropy, pre-eliminating a large number of low-information redundant frames and building an index only for high-value frames. This allows for more accurate retention of key information and reduces waste of storage and computing resources. Simultaneously, combined with large model compilation and cost-driven optimization in the online stage, the expensive video-language model is invoked only on demand in the compressed candidate video segments. This invention overcomes the problem of excessively high computational costs in traditional full-scan schemes through an offline-online collaborative mechanism. While maintaining near-full-scan retrieval accuracy, it significantly reduces the number of model calls, substantially lowering GPU computing costs and interactive query latency.

[0049] Accordingly, this application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the video query method based on the visual large model as described above. Figure 3 The diagram shown illustrates a hardware structure of any data processing device capable of using the video query method based on a large visual model provided in this embodiment of the invention, except... Figure 3 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0050] Accordingly, this application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the video query method based on a large visual model as described above. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.

[0051] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only.

[0052] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A video query method based on a large visual model, characterized in that, The method includes: Each video frame in the original video stream is evaluated for value. A value score threshold is set, and video frames with a value score greater than the value score threshold are considered high-value video frames, while those with a value score less than the threshold are considered low-value video frames. The pre-processed high-value and low-value video frames are then stored in a structured video database. In response to user natural language queries, the large language model compiles them into a structured query specification representation; The structured query canonical representation is converted into a directed acyclic graph, and the query execution order of each operator is determined based on the directed acyclic graph; Obtain several candidate video segments; based on the context-based multi-armed slot machine algorithm, schedule the query plan of each operator to be executed on its corresponding candidate video segment, thereby obtaining the video query results; The process of evaluating the value of each video frame in the original video stream includes: The value score of each video frame is a weighted sum of motion intensity, object density, and visual entropy of that video frame; The motion intensity calculation process for each video frame is as follows: decode and extract motion vectors from the original video stream, or calculate the average amplitude of the dense optical flow between the current video frame and its adjacent video frames to obtain the motion intensity corresponding to the current video frame. The process of calculating the object density of each video frame is as follows: input the current video frame into the object detection model to obtain the region of interest and the target bounding box; count the number of regions of interest and / or target bounding boxes, or calculate the proportion of the area of ​​all regions of interest and / or target bounding boxes to the total area of ​​the current video frame to obtain the object density corresponding to the current video frame; The process of calculating the visual entropy of each video frame is as follows: convert the current video frame into a grayscale image, statistically analyze the distribution histogram of grayscale values ​​in the grayscale image, calculate the information entropy of the distribution histogram based on Shannon entropy, and obtain the visual entropy corresponding to the current video frame. The process of converting the structured query canonical representation into a directed acyclic graph (DAG), and determining the query execution order of each operator based on the DAG, includes: Convert the canonical representation of structured queries into a directed acyclic graph; Based on a preset cost model, the cost of different candidate physical execution paths in the directed acyclic graph is estimated; the cumulative cost of different candidate physical execution paths is compared, and the candidate physical execution path with the lowest cumulative cost is selected as the optimal physical execution path. The query execution order of each operator is determined based on the optimal physical execution path; wherein, operators with lower execution costs are configured to be executed first, and operators with higher execution costs are configured to be executed later. The process of scheduling the query plan of each operator to be executed on its corresponding candidate video segment based on the context-based multi-armed slot machine algorithm includes: Extract the state vectors of all candidate video segments; wherein, the state vectors include: the embedding vector corresponding to the user's natural language query, the statistical distribution features of the similarity scores of the candidate video segments, and the number of matches; The state vector is input into a pre-trained policy network to obtain the probability distribution of action a; where each action a corresponds to a call budget value k of a large visual model, and the call budget value k represents the number of candidate video segments to be verified. The first k candidate video segments are input into the LLM Mercify operator, which then calls the large visual model to execute the query.

2. The video query method based on a large visual model according to claim 1, characterized in that, The process of storing preprocessed high-value video frames into a structured video database includes: Using a multi-target tracking algorithm, based on the intersection-union ratio of the target bounding boxes and / or the similarity of appearance feature vectors, the targets detected in the current high-value video frame are associated and matched with the targets or existing trajectories in the previous video frames; A unified trajectory identifier is assigned to the successfully matched target, and the motion feature vector of the trajectory is calculated. The motion feature vector includes the direction and magnitude of velocity and acceleration, thereby generating target trajectory data that contains temporal continuity. Extract discrete text attribute features and high-dimensional continuous features of the target. The discrete text attribute features include at least object category labels and color attributes, and the high-dimensional continuous features include at least global semantic vectors and object appearance feature vectors. For the discrete text attribute features, an inverted index is constructed; for the high-dimensional continuous features, an approximate nearest neighbor vector index is constructed. The target trajectory data, the inverted index, and the approximate nearest neighbor vector index are stored in a structured video database.

3. The video query method based on a large visual model according to claim 1 or 2, characterized in that, The process of storing preprocessed low-value video frames into a structured video database includes: For each low-value video frame, establish the temporal location information of the low-value video frame and generate a reference pointer pointing to the nearest stored high-value keyframe in the time dimension. The original pixel data of the low-value video frames are discarded, and the time position information and the reference pointer are stored as compressed low-value video frame records in the structured video database.

4. The video query method based on a large visual model according to claim 1, characterized in that, The process of obtaining several candidate video segments includes: The original video stream is processed by a lens detection algorithm to calculate the pixel histogram differences between video frame sequences in the original video stream. Based on the differences in the pixel histogram, the lens boundaries are determined, and the original video stream is divided into several physical lenses with consistent internal scenes, thereby obtaining several candidate video segments.

5. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the video query method based on a visual large model as described in any one of claims 1-4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the video query method based on a large visual model as described in any one of claims 1-4.

7. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the video query method based on a large visual model as described in any one of claims 1-4.

Images