Visual data management method, device, medium, and product for vehicle sentry mode

Abstract

The application provides a vehicle sentry mode visual data management method, device, medium and product, relates to the technical field of vehicle data management, and obtains visual data in a vehicle sentry mode; performs feature extraction processing on the visual data based on a self-supervised feature extraction model to obtain a target feature vector of the visual data; the self-supervised feature extraction model is obtained based on self-supervised learning training of a general visual feature extraction model on unlabelled sentry visual sample data; the visual data is subjected to hierarchical clustering processing based on the target feature vector to obtain a plurality of clustering clusters of the visual data; and a semantic index database is established based on implicit identifiers of the plurality of clustering clusters to perform life cycle management on the visual data. The application can improve the management efficiency of visual data generated in the vehicle sentry mode.

Description

Technical Field

[0001] This application relates to the field of vehicle data management technology, and in particular to a visual data management method, device, medium and product for vehicle sentinel mode. Background Technology

[0002] With the rapid popularization of vehicle intelligent connectivity technology, vehicle sentry mode has gradually become a standard feature in many models, and has become one of the core technologies for vehicle parking safety protection.

[0003] However, while continuously monitoring the vehicle's surroundings and recording potential security threats such as scratches, collisions, and attempted theft, vehicle sentinel mode also generates massive amounts of image and video visual data. Related technologies primarily rely on manually adding labels or designing rule models to process this massive visual data. This dependence on manual intervention results in extremely low efficiency in managing this visual data. Summary of the Invention

[0004] The main objective of this application is to propose a visual data management method, device, medium, and product for vehicle sentry mode, aiming to improve the management efficiency of visual data generated by vehicle sentry mode.

[0005] To achieve the above objectives, a first aspect of this application proposes a visual data management method for vehicle sentry mode, the method comprising:

[0006] Acquire visual data in vehicle sentry mode;

[0007] The visual data is processed by a self-supervised feature extraction model to obtain the target feature vector of the visual data; the self-supervised feature extraction model is trained by self-supervised learning on unlabeled sentry visual sample data based on a general visual feature extraction model.

[0008] Based on the target feature vector, hierarchical clustering processing is performed on the visual data to obtain multiple clusters of the visual data.

[0009] Based on the implicit identifiers of the multiple clusters, a semantic index database is established to manage the lifecycle of the visual data.

[0010] In some embodiments, the target feature vector includes a multidimensional feature vector for each frame of the visual data;

[0011] The hierarchical clustering process based on the target feature vector of the visual data yields multiple clusters of the visual data, including:

[0012] The visual data is subjected to frame-level clustering based on the multidimensional feature vectors to obtain static visual clusters of the visual data; the multiple clusters of the visual data include the static visual clusters.

[0013] Based on at least one time segment of the visual data, the multidimensional feature vector is segmented and fused to obtain the frame feature vector fusion result of the at least one time segment;

[0014] Based on the frame feature vector fusion result, the visual data is subjected to segment-level clustering to obtain dynamic event clusters of the visual data; the multiple clusters of the visual data include the dynamic event clusters.

[0015] In some embodiments, the frame-level clustering processing of the visual data based on the multidimensional feature vector includes:

[0016] The multidimensional feature vector of each frame of the visual data is subjected to similarity measurement processing to obtain the similarity measurement result;

[0017] Based on the similarity measurement results, unsupervised clustering is performed on the multidimensional feature vectors to obtain the cluster partitioning results of the multidimensional feature vectors; the cluster partitioning results include at least one feature cluster.

[0018] Based on the at least one feature cluster, cluster matching processing is performed on each frame of the visual data to obtain at least one static visual cluster of the visual data.

[0019] In some embodiments, performing segment-level clustering processing on the visual data based on the frame feature vector fusion result includes:

[0020] Cluster analysis is performed on the frame feature vector fusion results to obtain the cluster information of each frame in each time segment of the visual data;

[0021] The cluster information is aggregated based on the temporal relationship of each time segment of the visual data to obtain the dynamic event clusters of the visual data.

[0022] In some embodiments, the implicit identifier includes machine semantic tags, and the method further includes:

[0023] The target feature vectors within the target cluster are aggregated using an aggregation function to obtain the core feature vector of the target cluster; the target cluster is any one of the plurality of clusters.

[0024] Based on a pre-defined visual language model, the core feature vectors are semantically transformed to obtain machine semantic labels for the target cluster.

[0025] In some embodiments, the step of establishing a semantic index database based on the implicit identifiers of the plurality of clusters to manage the lifecycle of the visual data includes:

[0026] A semantic index database is obtained by establishing the association between the implicit identifiers of each of the multiple clusters and the metadata of the visual data; the metadata includes the fragment identifier data of the visual data, the mapping relationship between the visual data and the multiple clusters, and the original storage location data of the visual data.

[0027] The visual data is stored in separate compartments based on the semantic index database, and semantic retrieval is performed on the visual data stored in separate compartments based on the semantic index database.

[0028] In some embodiments, the semantic retrieval of visual data stored in separate warehouses based on the semantic index database includes:

[0029] The semantic similarity is calculated between the query intent information corresponding to the visual data query statement and the visual data stored in the semantic index database in separate compartments, and the semantic similarity calculation result is obtained.

[0030] If the semantic similarity calculation result indicates that the semantic similarity between the query intent information and the target visual data is greater than or equal to a preset threshold, the target visual data is output as the query result of the visual data query statement; the target visual data is at least one fragment of visual data stored in the semantic index database in separate compartments.

[0031] In some embodiments, the step of performing feature extraction processing on the visual data based on a self-supervised feature extraction model to obtain the target feature vector of the visual data includes:

[0032] Keyframe sampling is performed on the target segment of the visual data to obtain a keyframe sequence; the target segment is at least one of multiple segments of the visual data.

[0033] The self-supervised feature extraction model is used to extract features from at least one key frame in the key frame sequence, and the extracted multidimensional feature vectors of the key frames are fused into a multidimensional feature vector of the target segment for output. The target feature vector of the visual data includes the multidimensional feature vector of the target segment.

[0034] To achieve the above objectives, a second aspect of this application provides a visual data management device for vehicle sentry mode, the device comprising:

[0035] The acquisition module is used to acquire visual data in vehicle sentry mode;

[0036] The self-supervised feature extraction module is used to perform feature extraction processing on the visual data based on the self-supervised feature extraction model to obtain the target feature vector of the visual data; the self-supervised feature extraction model is obtained by self-supervised learning training on unlabeled sentinel visual sample data based on a general visual feature extraction model.

[0037] The hierarchical clustering module is used to perform hierarchical clustering processing on the visual data based on the target feature vector to obtain multiple clusters of the visual data;

[0038] The semantic indexing module is used to establish a semantic index database based on the implicit identifiers of the multiple clusters to manage the lifecycle of the visual data.

[0039] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the visual data management method for vehicle sentry mode described in the first aspect.

[0040] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the visual data management method for vehicle sentry mode described in the first aspect.

[0041] To achieve the above objectives, a fifth aspect of the present application provides a computer program product, which includes a computer program that, when executed by a processor, implements the visual data management method for vehicle sentry mode as provided in the first aspect above.

[0042] The visual data management method, apparatus, electronic device, computer-readable storage medium, and computer program product for vehicle sentry mode proposed in this application acquire visual data in vehicle sentry mode; perform feature extraction processing on the visual data based on a self-supervised feature extraction model to obtain a target feature vector of the visual data; the self-supervised feature extraction model is obtained by self-supervised learning training on unlabeled sentry visual sample data based on a general visual feature extraction model; perform hierarchical clustering processing on the visual data based on the target feature vector to obtain multiple clusters of the visual data; and establish a semantic index database based on the implicit identifiers of the multiple clusters to manage the lifecycle of the visual data.

[0043] Compared to traditional methods that rely on manual intervention for sentry visual data management, this application's embodiments pre-train a general visual feature extraction model using self-supervised learning based on a large-scale set of unlabeled sentry visual sample data. This yields a self-supervised feature extraction model capable of performing general visual feature extraction based on self-supervised learning. After acquiring visual data generated under vehicle sentry mode, the model extracts features from the visual data to obtain a target feature vector. Then, based on this target feature vector, the visual data undergoes hierarchical clustering to obtain multiple clusters. Finally, a semantic index database is established based on the implicit identifiers of each cluster to manage the visual data's lifecycle.

[0044] Thus, in this embodiment, self-supervised learning requires no human intervention and can be trained using only massive amounts of sentinel visual sample data. This completely eliminates the need for manual annotation, significantly reducing the manual and time costs of feature extraction from visual data. It solves the efficiency bottleneck caused by reliance on manual methods and improves the efficiency of visual data processing. Furthermore, since the self-supervised learning process of the self-supervised feature extraction model is objective and data-driven, it avoids subjective bias caused by human intervention, thereby ensuring the consistency and reliability of the target feature vector extracted from the visual data.

[0045] Furthermore, this application's embodiments perform hierarchical clustering of visual data based on target feature vectors extracted from visual data. This automatically categorizes massive amounts of visual data without requiring manual intervention to define categories, thereby improving the automation level and efficiency of the overall analysis process. Moreover, by establishing a semantic index database based on the implicit identifiers of multiple clusters obtained through clustering to manage the lifecycle of visual data, similar events within the visual data can be centrally stored. This not only facilitates management but also significantly optimizes subsequent data utilization efficiency and the speed of valuable data retrieval and mining. Attached Figure Description

[0046] Figure 1 A flowchart illustrating the steps of the visual data management method for vehicle sentry mode provided in some embodiments of this application;

[0047] Figure 2 for Figure 1 A detailed flowchart of step S102;

[0048] Figure 3 for Figure 1 A detailed flowchart of step S103;

[0049] Figure 4A flowchart illustrating the steps of the visual data management method for vehicle sentry mode provided in this application in other embodiments;

[0050] Figure 5 for Figure 1 A detailed flowchart of step S104;

[0051] Figure 6 for Figure 5 A detailed flowchart of step S502;

[0052] Figure 7 A schematic diagram of the overall system flow of the visual data management method for vehicle sentry mode provided in an embodiment of this application;

[0053] Figure 8 A schematic diagram of the structure of the visual data management device for vehicle sentry mode provided in an embodiment of this application;

[0054] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0056] It should be noted that although functional modules are divided in the device / system schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device / system or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0058] First, the overall concept of the visual data management method for vehicle sentry mode provided in the embodiments of this application will be explained.

[0059] With the rapid popularization of intelligent vehicle connectivity technology, vehicle sentinel mode has gradually become a standard feature in many models, evolving from an optional function, and has become one of the core technologies for vehicle parking safety protection. While continuously monitoring the vehicle's surrounding environment and recording potential safety threats such as scratches, collisions, and attempted theft in real time, vehicle sentinel mode also generates massive amounts of image and video visual data.

[0060] Visual data generated in vehicle sentry mode is of immense value for improving vehicle safety, optimizing autonomous driving algorithms, and determining accident liability. However, efficiently storing, managing, and retrieving this massive amount of unstructured visual data has become a significant technical challenge. Traditional solutions primarily rely on manually adding labels or using manually designed rule models for simple data classification. These solutions suffer from inherent flaws such as coarse granularity, poor flexibility, low efficiency, and strong subjectivity, severely limiting the immense value of visual data in vehicle sentry mode due to inefficient and crude data management methods. Overcoming the problems caused by traditional solutions involves the following interrelated challenges:

[0061] How to meet the precise needs of complex scenarios.

[0062] Traditional solutions mostly only offer binary classification (presence or absence of an event) or very limited coarse-grained classification (such as "people" and "vehicles"). However, the practical application scenarios of vehicle sentry mode are often much more complex. For example, simply knowing "someone is approaching" is far from sufficient; it's necessary to further distinguish whether it's "the car owner retrieving the vehicle normally," "a passerby accidentally walking by," "a delivery person placing a package," or "a thief attempting to open the car door." Similarly, for events around vehicles, it's necessary to differentiate between "minor scratches," "serious collisions," and "a collision caused by an adjacent vehicle opening its door." Because this fine-grained, scenario-based recognition capability is crucial for accurate alarms, insurance claims, and improved user experience, traditional data management methods cannot provide such rich information dimensions, resulting in a large amount of valuable details being lost. Therefore, how to achieve the leap from "presence or absence of an event" to "what, what, and why," and how to perform multi-dimensional, deep semantic understanding and classification of events, is the core technical challenge in visual data management.

[0063] How to adapt to unknown or evolving scenarios.

[0064] Traditional systems based on predefined rules are inherently closed, only capable of recognizing rules pre-conceived and manually written by programmers. They become completely "blind" to novel attack methods, rare traffic accident patterns, or new objects not included in existing classification systems. The real world is infinite and constantly changing. This means traditional systems quickly become obsolete, requiring continuous investment and yielding minimal returns in maintaining and updating the rule base. Therefore, a more advanced technical challenge in managing visual data is enabling systems to "discover" and "learn," automatically extracting unknown, potential, and meaningful new patterns or anomalies from data, and continuously evolving and expanding their cognitive boundaries without requiring costly and time-consuming reprogramming each time.

[0065] How to avoid compromising data credibility.

[0066] Traditional methods, even with significant manpower invested in data annotation, struggle to guarantee output quality. This is because different annotators may have varying judgments about the same scene (e.g., defining "suspicious behavior"), and even the same annotator may use different standards when fatigued or not. This subjectivity introduces "noise," casting doubt on the accuracy and reliability of conclusions drawn from model training or accident analysis based on this data. For training autonomous driving algorithms, low-quality or inconsistent labels can even pose safety hazards. Therefore, a deeper issue in managing visual data is how to eliminate human subjectivity and ensure the objectivity, consistency, and repeatability of data interpretation, thereby guaranteeing the accuracy and fairness of downstream applications (such as model training and liability determination).

[0067] How to balance the sharp contradiction between the scale of massive data and low processing efficiency.

[0068] Traditional methods relying on manual visual inspection or simple rule-based filtering suffer from a processing speed several orders of magnitude slower than the rate of data generation. Manual annotation is extremely costly and completely inadequate for real-time or near-real-time processing. This results in a large amount of data becoming "data garbage" due to untimely processing—either overwritten or deleted, or silently stored on hard drives, unable to realize its potential value. Therefore, the primary challenge in managing visual data is achieving a qualitative leap in processing speed, ensuring that data analysis efficiency matches the data generation rate, and significantly reducing the per capita data processing cost.

[0069] How to bridge the fundamental gap between unstructured data and the needs of management retrieval?

[0070] Raw visual data, such as videos and images, is unstructured and cannot be efficiently indexed and queried by databases. For example, users cannot perform natural language searches like searching for "all incidents involving electric bicycles scraping the left side of a vehicle on [date]". To achieve this, unstructured data must first be manually converted into structured tagged text, a process that is the root of all the aforementioned problems. Therefore, the most fundamental technical challenge in managing visual data lies in breaking down this barrier and automatically generating rich, accurate, and machine-readable structured semantic descriptions (metadata) for each piece of unstructured visual data using advanced artificial intelligence technology. This lays the foundation for achieving millisecond-level, multi-condition, and intelligent retrieval and management of massive amounts of data.

[0071] To address this, embodiments of this application provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for managing visual data in vehicle sentry mode, aiming to overcome the shortcomings of the aforementioned related technologies and improve the efficiency of processing visual data generated in vehicle sentry mode.

[0072] The visual data management method, apparatus, electronic device, computer-readable storage medium, and computer program product for vehicle sentry mode proposed in this application acquire visual data in vehicle sentry mode; perform feature extraction processing on the visual data based on a self-supervised feature extraction model to obtain a target feature vector of the visual data; the self-supervised feature extraction model is obtained by self-supervised learning training on unlabeled sentry visual sample data based on a general visual feature extraction model; perform hierarchical clustering processing on the visual data based on the target feature vector to obtain multiple clusters of the visual data; and establish a semantic index database based on the implicit identifiers of the multiple clusters to manage the lifecycle of the visual data.

[0073] Compared to traditional methods that rely on manual intervention for sentry visual data management, this application's embodiments pre-train a general visual feature extraction model using self-supervised learning based on a large-scale set of unlabeled sentry visual sample data. This yields a self-supervised feature extraction model capable of performing general visual feature extraction based on self-supervised learning. After acquiring visual data generated under vehicle sentry mode, the model extracts features from the visual data to obtain a target feature vector. Then, based on this target feature vector, the visual data undergoes hierarchical clustering to obtain multiple clusters. Finally, a semantic index database is established based on the implicit identifiers of each cluster to manage the visual data's lifecycle.

[0074] Thus, in this embodiment, self-supervised learning requires no human intervention and can be trained using only massive amounts of sentinel visual sample data. This completely eliminates the need for manual annotation, significantly reducing the manual and time costs of feature extraction from visual data. It solves the efficiency bottleneck caused by reliance on manual methods and improves the efficiency of visual data processing. Furthermore, since the self-supervised learning process of the self-supervised feature extraction model is objective and data-driven, it avoids subjective bias caused by human intervention, thereby ensuring the consistency and reliability of the target feature vector extracted from the visual data.

[0075] Furthermore, this application's embodiments perform hierarchical clustering of visual data based on target feature vectors extracted from visual data. This automatically categorizes massive amounts of visual data without requiring manual intervention to define categories, thereby improving the automation level and efficiency of the overall analysis process. Moreover, by establishing a semantic index database based on the implicit identifiers of multiple clusters obtained through clustering to manage the lifecycle of visual data, similar events within the visual data can be centrally stored. This not only facilitates management but also significantly optimizes subsequent data utilization efficiency and the speed of valuable data retrieval and mining.

[0076] Next, the visual data management method, apparatus, electronic device, computer-readable storage medium, and computer program product for vehicle sentry mode provided in this application will be specifically described through the following embodiments, and firstly, the various detailed embodiments of the visual data management method for vehicle sentry mode provided in this application will be described in detail.

[0077] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0078] It should be noted that the visual data management method for vehicle sentry mode provided in this application embodiment can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be an in-vehicle terminal device (e.g., an in-vehicle computing platform) or a terminal device associated with the vehicle. The association between the terminal device and the vehicle means that the terminal device can communicate and interact with the vehicle via a network. For example, the terminal device can be a cloud server device that communicates and interacts with the vehicle. Furthermore, the terminal can also be a smartphone, tablet, laptop, desktop computer, or other computer device. The server can be a backend server terminal device, which can be configured as an independent physical server, a server cluster consisting of multiple physical servers, or a distributed system. It can also be configured as a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The software can be an application implementing the visual data management method for vehicle sentry mode, a computer program, and a storage medium carrying the computer program. It should be understood that, based on different design needs of practical applications, the terminal, server, and software of the vehicle sentry mode visual data management method provided in this application embodiment may also be other forms not listed here, and the vehicle sentry mode visual data management method provided in this application embodiment does not specifically limit these.

[0079] Furthermore, this application can also be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer computer devices, personal computers (PCs), minicomputers, mainframe computers, vehicles, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0080] For ease of understanding and explanation, the following text will use the visual data management method for vehicle sentry mode provided in the embodiments of this application as an example to describe the various specific embodiments of this application in detail. The implementation of the visual data management method for vehicle sentry mode provided in the embodiments of this application by any other subject can refer to the implementation process of the visual data management method for sentry mode described below.

[0081] Please refer to Figure 1 , Figure 1 The flowchart illustrates the steps of the visual data management method for vehicle sentry mode provided in this application embodiment in some embodiments. It should be understood that, although... Figure 1 The flowcharts illustrating subsequent steps show the execution order of some method steps. However, based on different design needs in practical applications, the visual data management method for vehicle sentry mode provided in this application embodiment can, of course, employ a different execution order of method steps than shown in the figures. That is, Figure 1 The order of the method steps shown does not constitute a limitation on the execution logic order of the visual data management method for vehicle sentry mode provided in this application embodiment. Any other method based on... Figure 1 Reasonable changes to the sequence of steps shown should be included within the protection scope of the visual data management method for vehicle sentry mode provided in the embodiments of this application.

[0082] like Figure 1 As shown, in some embodiments, when a terminal device applies the visual data management method for vehicle sentry mode provided in the embodiments of this application, it may include steps S101 to S104 as shown below.

[0083] Step S101: Acquire visual data in vehicle sentry mode.

[0084] It should be noted that the vehicle sentry mode refers to the vehicle sentry function running on the in-vehicle terminal device. The vehicle sentry function is a passive safety system designed to monitor the vehicle and its surrounding environment to ensure vehicle safety. Specifically, the vehicle sentry function can monitor the surrounding environment and promptly detect abnormalities when the vehicle is unattended. Furthermore, once a potential threat is detected (such as kicking, scratching, or violently striking the vehicle body), the system can automatically start video recording to provide evidence for subsequent accountability. In addition, the vehicle sentry function can also control the vehicle to emit audible and visual alarms and push alarm notifications to the owner via an app when a security threat is detected.

[0085] In addition, visual data can be used for vehicle sentry mode to continuously monitor the vehicle's surroundings by using visual sensors (such as multiple high-definition cameras deployed around the vehicle) configured on the vehicle to capture video streams and images. This results in raw, unstructured video data.

[0086] During the process of the vehicle sentry mode being activated and running to protect the vehicle, the terminal device continuously acquires visual data collected by the vehicle sentry mode to monitor the surrounding environment of the vehicle.

[0087] In some embodiments, when the terminal device acquires visual data collected by the vehicle sentry mode, it can control the vehicle sentry mode to dynamically adjust the frame rate and resolution of the video data acquired by the visual sensor. For example, the terminal device can control the vehicle sentry mode to adopt the following adjustment logic: in a silent state (no person / object appears within the warning range of the pre-calibrated or defined vehicle surrounding environment), the camera uses the lowest frame rate (e.g., 2 frames per second) and the lowest resolution (e.g., 320*320); when a person / object appears within the warning range, the camera uses a medium frame rate (e.g., 30 frames per second) and a medium resolution (e.g., reduced to 320*320 after oversampling); and when a person / object appears within the warning range and a vibration signal is triggered, the camera uses a high frame rate (e.g., 60 frames per second) and the highest resolution for 30 seconds.

[0088] Step S102: Perform feature extraction processing on the visual data based on the self-supervised feature extraction model to obtain the target feature vector of the visual data; the self-supervised feature extraction model is obtained by self-supervised learning training on unlabeled sentry visual sample data based on a general visual feature extraction model.

[0089] It should be noted that the terminal device can pre-train the model using a self-supervised learning approach. This involves training a general visual feature extraction model on a massive, unlabeled sentry visual sample dataset, resulting in a trained self-supervised feature extraction model. Through pre-training, this model can extract features without targeting any specific category (such as "scratches" or "theft") during training and real-world applications. Instead, it autonomously learns a general visual representation of the vehicle's surrounding environment by reconstructing masked image patches or comparing instances, thereby extracting high-density, semantically rich feature vectors from the visual data.

[0090] After acquiring visual data, the terminal device further inputs this raw, unstructured video data into a pre-trained self-supervised feature extraction model to transform it into a structured, machine-readable, high-density feature vector sequence. This model then performs feature extraction on the visual data, yielding the target feature vector. The target feature vector output by the self-supervised feature extraction model efficiently represents the visual and temporal information of the video, providing a strong foundation for downstream tasks such as behavior recognition, video retrieval, and content analysis.

[0091] In some embodiments, when a terminal device performs feature extraction processing on visual data based on a self-supervised feature extraction model, it can capture subtle textures, shapes, and motion pattern changes (such as subtle dents in car paint or specific movements of a pedestrian's arm) from the visual data using this self-supervised feature extraction model, thereby obtaining the target feature vector of the visual data. In this way, by capturing subtle features in the image for subsequent fine-grained classification and recognition, the limitation of traditional solutions that can only perform coarse-grained classification is overcome, thus meeting the accuracy requirements of complex scenarios.

[0092] In some embodiments, when a terminal device performs feature extraction processing on visual data based on a self-supervised feature extraction model, it can extract high-density, semantically rich feature vectors for each frame of the visual data through the self-supervised feature extraction model, thereby obtaining the overall target feature vector of the visual data.

[0093] In other embodiments, the terminal device may also extract high-density, semantically rich feature vectors from only some key frame images in the visual data based on a self-supervised feature extraction model, and then use the features of these extracted key frames as the target feature vector of the entire visual data.

[0094] Step S103: Perform hierarchical clustering processing on the visual data based on the target feature vector to obtain multiple clusters of the visual data.

[0095] After extracting the target feature vector of the visual data, the terminal device further performs hierarchical clustering on the visual data based on the target feature vector, thereby obtaining multiple clusters of the visual data.

[0096] In some embodiments, the terminal device can pre-design a hierarchical clustering pipeline. First, at the frame-level, a pre-trained self-supervised feature extraction model is used to extract the target feature vector of the visual data. Then, based on temporal information, the frame sequence is segmented into short video clips, and the features of all frames within a clip are aggregated to form a clip-level temporal feature representation. Finally, the visual data is clustered at both the frame and clip levels. The clustering results at different levels are mutually verified and complemented to obtain multiple clusters of the visual data.

[0097] Step S104: Based on the implicit identifiers of the multiple clusters, establish a semantic index database to manage the lifecycle of the visual data.

[0098] It should be noted that implicit labeling can be a machine-generated semantic label assigned to each cluster after the terminal device performs hierarchical clustering of visual data to generate multiple clusters. These semantic labels can be obtained by querying external multimodal models or keyword libraries based on cluster center features.

[0099] After the terminal device performs hierarchical clustering of visual data based on the target feature vector to generate multiple clusters and assigns an implicit identifier to each cluster, it can further establish a semantic index database of the visual data based on the implicit identifier, thereby performing lifecycle management such as storage and retrieval of the visual data.

[0100] In some embodiments, when a terminal device builds a semantic index database for visual data based on the implicit identifiers of multiple clusters, it can use the metadata of the clusters with implicit identifiers (semantic tags) and all segments of the visual data (including the unique identifier ID of the video segment, the cluster identifier ID to which it belongs—derived from the mapping relationship between segments and clusters within the cluster, and the pointer to the original data storage location—such as the storage path of the visual data, the Uniform Resource Locator URL, etc.) as input to establish an association index between the video segments and the clusters, semantic tags, and the pointer to the original data storage location, thereby obtaining the semantic index database for the visual data. This semantic index database can store information in the form of records, with each record containing fields such as: unique ID of the video segment, cluster ID, semantic tag, and pointer to the original data storage location. In this way, the terminal device can achieve traceability and rapid querying of visual data based on this semantic index database.

[0101] In this embodiment, during the vehicle's security protection process in vehicle sentry mode, the terminal device continuously acquires visual data collected by monitoring the vehicle's surrounding environment. Then, to convert the raw, unstructured video data into a structured, machine-readable, high-density feature vector sequence, the terminal device inputs this visual data into a pre-trained self-supervised feature extraction model. This model then performs feature extraction processing on the visual data to obtain the target feature vector. Next, the terminal device performs hierarchical clustering based on the target feature vector, resulting in multiple clusters. Finally, the terminal device assigns an implicit identifier to each cluster and further establishes a semantic index database for the visual data based on this implicit identifier, thereby managing the visual data's lifecycle, including storage and retrieval.

[0102] Compared to traditional methods that rely on manual intervention for sentry visual data management, this embodiment utilizes self-supervised learning, which requires no manual intervention and can be trained using only massive amounts of sentry visual sample data. This completely eliminates the need for manual annotation, significantly reducing the manual and time costs of feature extraction from visual data. It overcomes the efficiency bottleneck caused by manual intervention and improves the efficiency of visual data processing. Furthermore, because the self-supervised learning process of the self-supervised feature extraction model is objective and data-driven, it avoids subjective biases caused by manual intervention, thus ensuring the consistency and reliability of the target feature vectors extracted from the visual data.

[0103] Furthermore, this application's embodiments perform hierarchical clustering of visual data based on target feature vectors extracted from visual data. This automatically categorizes massive amounts of visual data without requiring manual intervention to define categories, thereby improving the automation level and efficiency of the overall analysis process. Moreover, by establishing a semantic index database based on the implicit identifiers of multiple clusters obtained through clustering to manage the lifecycle of visual data, similar events within the visual data can be centrally stored. This not only facilitates management but also significantly optimizes subsequent data utilization efficiency and the speed of valuable data retrieval and mining.

[0104] Furthermore, in this embodiment, by abandoning the traditional method of relying on a large amount of manually labeled data to train the model, a pre-trained model based on self-supervised learning is adopted. Since the self-supervised feature extraction model learns the general ability of "how to see the world" rather than the specific ability of "identifying specific things," it can extract meaningful features even for unknown or unseen new event types. This provides the possibility of discovering new categories and makes the system more scalable. Thus, compared to traditional solutions, this embodiment enhances system flexibility by using a pre-trained self-supervised feature extraction model to process visual data. This allows the system to adapt to unknown or evolving scenarios in the vehicle sentry mode for relevant visual data processing. That is, based on the model's "discovery" and "learning" capabilities, it automatically mines unknown, potential, and meaningful new patterns or anomalous events from the data, continuously evolving and expanding its cognitive boundaries without requiring expensive and time-consuming reprogramming each time.

[0105] Please refer to Figure 2 , Figure 2 for Figure 1 A detailed flowchart of step S102.

[0106] like Figure 2 As shown, in some embodiments, the step S102 above, "performing feature extraction processing on the visual data based on a self-supervised feature extraction model", may include steps S201 and S202 as shown below.

[0107] Step S201: Perform keyframe sampling on the target segment of the visual data to obtain a keyframe sequence; the target segment is at least one of the multiple segments of the visual data.

[0108] After receiving visual data generated in vehicle sentry mode, the terminal device can perform video segmentation and keyframe sampling preprocessing on the visual data. Then, it can extract features from the sampled keyframes to obtain high-density feature vectors as the target feature vectors for the visual data. Based on this, in the visual data preprocessing stage, the terminal device can first perform video segmentation to obtain multiple segments of the visual data. Then, for any target segment among these segments, the terminal device can further perform keyframe sampling on that target segment, thereby obtaining a keyframe sequence for each segment.

[0109] In some embodiments, when preprocessing visual data, the terminal device may further perform downsampling processing on each segment after cutting the visual data into shorter segments, thereby extracting the keyframe sequence of each segment.

[0110] For example, assume that the visual data acquired by the terminal device is the original video. The terminal device first processes the original video. Divide evenly into Non-overlapping (or overlapping) segments :

[0111] }

[0112] Then, for each segment The terminal device can sample K keyframes from it:

[0113] }

[0114] in, This represents the k-th sampled frame in the i-th segment.

[0115] Furthermore, the sampling strategy can be represented as a function S:

[0116] } = S( ).

[0117] The strategy can be equal-interval sampling, random sampling, etc.

[0118] Step S202: Perform feature extraction processing on at least one key frame in the key frame sequence based on the self-supervised feature extraction model, and fuse the extracted key frame multidimensional feature vectors into the multidimensional feature vector of the target segment based on the self-supervised feature extraction model for output; the target feature vector of the visual data includes the multidimensional feature vector of the target segment.

[0119] After sampling keyframes from each segment of visual data to obtain a keyframe sequence, the terminal device sequentially treats each segment as a target segment and inputs the keyframe sequence of the target segment into a self-supervised feature extraction model. This model then performs feature extraction processing on the keyframe sequence, capturing subtle texture, shape, and motion pattern variations in each frame. These extracted features are used as the multidimensional feature vectors of each keyframe image. Furthermore, after extracting the multidimensional feature vectors of each frame, the terminal device fuses these vectors to obtain the multidimensional feature vector of the target segment, which is then output. In this way, the terminal device obtains the multidimensional feature vectors of each segment output by the self-supervised feature extraction model and uses these multidimensional feature vectors as the target feature vectors of the visual data.

[0120] For example, assume that the self-supervised feature extraction model is a function , among them These are the parameters for model pre-training, which uses the sampled keyframe sequences. } is used as input, and fine features are extracted for each frame of the image to obtain the features of each frame:

[0121] } = .

[0122] in, This is the D-dimensional high-density feature vector (keyframe multidimensional feature vector) corresponding to the k-th keyframe.

[0123] Then, the self-supervised feature extraction model outputs the global feature Z of the entire segment by fusing the features of all keyframes in the sequence:

[0124] Z = .

[0125] in, It is a global D-dimensional feature vector (multidimensional feature vector) that integrates the feature information of all K frames.

[0126] In some embodiments, the terminal device can average the keyframe multidimensional feature vectors of all frames within the target segment to fuse these keyframe multidimensional feature vectors into a multidimensional feature vector of the target segment. Alternatively, it can use a special token [CLS] in a self-supervised feature extraction model to fuse the keyframe multidimensional feature vectors of all frames within the segment into a multidimensional feature vector of the target segment for output.

[0127] In this embodiment, keyframe sampling is performed on target segments of visual data via a terminal device to obtain a keyframe sequence. Then, a self-supervised feature extraction model is used to extract features from the keyframes in the keyframe sequence. In this way, the self-supervised feature extraction model can autonomously learn from the data to extract multi-dimensional feature vectors without any manual annotation, thus achieving an end-to-end fully automated pipeline from feature extraction to event classification. Furthermore, subsequent unsupervised clustering automatically discovers inherent patterns in the data, enabling the system to autonomously summarize and discover unknown new event types, thereby giving the system the ability to self-evolve and continuously learn.

[0128] In some embodiments, the target feature vector includes a multidimensional feature vector for each frame of video data.

[0129] It should be noted that multidimensional feature vectors can represent subtle features such as texture, shape, and motion pattern changes in an image.

[0130] Please refer to Figure 3 , Figure 3 for Figure 1A detailed flowchart of step S103.

[0131] like Figure 3 As shown, in some embodiments, step S103 above: performing hierarchical clustering processing on the visual data based on the target feature vector to obtain multiple clusters of the visual data may include steps S301 to S303 as shown below.

[0132] Step S301: Perform frame-level clustering processing on the visual data based on the multidimensional feature vector to obtain static visual clusters of the visual data; the multiple clusters of the visual data include the static visual clusters.

[0133] When a terminal device performs hierarchical clustering on visual data based on the target feature vector, it can first perform frame-level clustering on the visual data based on the multi-dimensional feature vector of each frame, thereby obtaining a static visual cluster of the video data. In this way, the terminal device can use this static visual cluster as part of multiple clusters of visual data.

[0134] In some embodiments, when the terminal device performs frame-level clustering processing on the visual data based on the multidimensional feature vector of each frame, it can group frames with similar features in the visual data into the same cluster by similarity measurement. That is, each frame in the visual data is represented by its multidimensional feature vector, and these multidimensional feature vectors are subjected to unsupervised clustering processing by an unsupervised clustering algorithm to obtain cluster centers and cluster division results. Finally, according to the cluster results, frames belonging to the same cluster are grouped together to form a static visual clustering cluster of the visual data.

[0135] In some embodiments, the step of "performing frame-level clustering processing on the visual data based on the multidimensional feature vector" in step S301 above may include the following steps:

[0136] The multidimensional feature vector of each frame of the visual data is subjected to similarity measurement processing to obtain the similarity measurement result;

[0137] Based on the similarity measurement results, unsupervised clustering is performed on the multidimensional feature vectors to obtain the cluster partitioning results of the multidimensional feature vectors; the cluster partitioning results include at least one feature cluster.

[0138] Based on the at least one feature cluster, cluster matching processing is performed on each frame of the visual data to obtain at least one static visual cluster of the visual data.

[0139] When a terminal device groups frames with similar features in visual data into the same cluster to form a static visual cluster, it first performs a similarity measurement on the multidimensional feature vector of each frame of the visual data to obtain the similarity measurement result. This similarity measurement result can be the similarity measurement result between each frame of the visual data and the initial cluster center, which can be the multidimensional feature vector of an image frame randomly selected by the terminal device from multiple frames of the visual data.

[0140] Then, the terminal device uses an unsupervised clustering algorithm to perform unsupervised clustering on the multidimensional feature vector of each frame image, using the similarity measurement result between each frame image and the initial cluster center as the standard. During this process, the cluster centers are updated, thereby obtaining the cluster centers and cluster partitioning results of the multidimensional feature vector. The cluster partitioning results may include one or more feature clusters.

[0141] Finally, the terminal device matches the multidimensional feature vector of each frame in the visual data with the feature clusters in the cluster division results, thereby dividing all frames in the visual data that belong to the same feature cluster into a group, and obtaining one or more static visual clusters of the video data.

[0142] In some embodiments, when the terminal device performs unsupervised clustering on the multidimensional feature vectors of each frame of an image to group frames with similar features into the same feature cluster based on similarity measurement, it can first initialize the multidimensional feature vectors to determine the initial cluster centers (e.g., randomly selecting some feature vectors as initial centers, or generating initial centers based on a preset number of clusters). Then, based on a preset similarity measurement method (e.g., Euclidean distance, cosine similarity, etc.), it calculates the similarity value between the multidimensional feature vector of each frame and each initial cluster center, and assigns the multidimensional feature vector of each frame to the cluster center with the most similarity value. Clusters with similar initial cluster centers are formed into preliminary cluster partitions. Then, for each cluster after the preliminary partitions, the cluster centers are recalculated based on all the multidimensional feature vectors within the cluster (such as updating through aggregation methods like mean and median) to obtain updated cluster centers. Finally, it is determined whether the difference between the current updated cluster centers and the cluster centers of the previous round meets the preset stopping conditions (such as the difference being less than a threshold or the number of iterations reaching the upper limit). If not, the similarity is recalculated based on the updated cluster centers, and the partitioning and updating process is repeated until the preset conditions are met. If they are met, the final cluster centers and the corresponding cluster partitioning results are output.

[0143] In some embodiments, when the terminal device performs cluster matching processing on each frame of video data based on feature clustering, it can parse the clustering results of the multidimensional feature vector, extract the unique frame identifier (such as frame ID, index) corresponding to each feature cluster, form a "cluster ID-frame identifier" mapping table, and then filter out all frames corresponding to each cluster from all frames of visual data based on the mapping table, aggregate them into a cluster-specific frame set, and associate each frame set with the cluster center of its corresponding multidimensional feature vector to form structured data containing "cluster ID, frame set, cluster center", i.e., static visual clustering cluster.

[0144] In some embodiments, the terminal device may use Euclidean distance as a similarity metric:

[0145] .

[0146] in, Let be the multidimensional feature vector of the i-th frame; is the center of the j-th cluster; n is the dimension of the eigenvector.

[0147] Furthermore, the formula for updating the cluster center on the terminal device can be as follows:

[0148] .

[0149] in, Let be the sample set of the j-th cluster in the t-th iteration; For clusters The number of samples in the sample.

[0150] Furthermore, the terminal device can calculate the sum of squared errors as the difference between the updated cluster centers and the previous cluster centers:

[0151] .

[0152] Where k is the number of clusters; Let be the sample set of the j-th cluster.

[0153] Step S302: Perform segmented fusion processing on the multidimensional feature vector based on at least one time segment of the visual data to obtain the frame feature vector fusion result of the at least one time segment.

[0154] When a terminal device performs hierarchical clustering processing on visual data based on the target feature vector of visual data, it can first perform segmented fusion processing on the multidimensional feature vector of each frame of the visual data based on at least one time segment of the visual data. That is, it can aggregate the multidimensional feature vectors of all frames in a time segment of the visual data to form a segment-level temporal feature representation, thereby obtaining the frame feature vector fusion result of at least one time segment.

[0155] Step S303: Perform segment-level clustering processing on the visual data based on the frame feature vector fusion result to obtain dynamic event clusters of the visual data; the multiple clusters of the visual data include the dynamic event clusters.

[0156] After obtaining the frame feature vector fusion results for each time segment of the visual data, the terminal device performs cluster analysis on these results and aggregates the temporal information between each time segment to achieve segment-level clustering of the visual data, thus obtaining a dynamic event cluster of the visual data. In this way, the terminal device can also include this dynamic event cluster as part of multiple clusters of the visual data.

[0157] In some embodiments, the terminal device can segment visual data (video / sequence) into multiple time segments by dividing it into time windows. Then, within each segment, it performs cluster analysis on frame-level features (multi-dimensional feature vectors of each frame of visual data) and aggregates them using the temporal information between each time segment to form dynamic event clusters. In this way, it can capture the evolution of events in the time dimension, which is suitable for scenarios such as video anomaly detection and behavior recognition.

[0158] In some embodiments, step S303 above, "performing segment-level clustering of the visual data based on the frame feature vector fusion result," may include the following steps:

[0159] Cluster analysis is performed on the frame feature vector fusion results to obtain the cluster information of each frame in each time segment of the visual data;

[0160] The cluster information is aggregated based on the temporal relationship of each time segment of the visual data to obtain the dynamic event clusters of the visual data.

[0161] When performing segment-level clustering processing on visual data based on the frame feature vector fusion results of each time segment of the visual data, the terminal device can first perform cluster analysis on the frame feature vector fusion results to obtain the cluster information of each frame within each time segment of the visual data. Then, it can use the temporal relationship of each time segment of the visual data to finally aggregate the cluster information, thereby obtaining the dynamic event cluster of the visual data.

[0162] For example, after the terminal device segments the visual data into multiple time segments, it processes each segment... Use clustering algorithms to obtain cluster centers: } , Cluster tags for frames: Then, the terminal device aggregates cluster information using a time decay factor weighting:

[0163] .

[0164] in, The frame number of cluster k in segment n.

[0165] Finally, the terminal device performs final clustering of the features within each segment based on the aggregated cluster information ClusterScore:

[0166]

[0167] in, The weight is determined by the time and location.

[0168] In this embodiment, the terminal device performs frame-level clustering processing on the visual data based on the multi-dimensional feature vectors of each frame, thereby obtaining static visual clusters of the video data. Furthermore, based on at least one time segment of the visual data, the multi-dimensional feature vectors are fused in segments to obtain frame feature vector fusion results for at least one time segment. Based on these frame feature vector fusion results, segment-level clustering processing is then performed on the visual data to obtain dynamic event clusters of the visual data. Thus, by employing a hierarchical clustering strategy combined with general features, dynamic event sequences and subtle static features can be identified, enabling the system to achieve deep semantic understanding of visual data regarding "who, what, what, and why." In addition, the hierarchical clustering strategy can also understand vision from different dimensions, achieving fine-grained identification and discovery of unknown events in visual data. Specifically, frame-level clustering of visual data can discover similar static elements or key states (such as "the moment the car door handle is pulled" or "tires at different angles"), while segment-level clustering of visual data can discover similar dynamic event patterns (such as "a person pacing around a car," "two cars meeting," or "an electric vehicle brushing past"). In this way, massive amounts of visual data can be automatically categorized into various fine-grained event categories based on frame-level and segment-level clustering of visual data. Complex scenes can be accurately identified and unknown new events can be automatically discovered without manual pre-definition.

[0169] In some embodiments, the implicit identifier of a cluster includes machine semantic tags.

[0170] It should be noted that machine semantic labels can be implemented by the aforementioned terminal devices through cluster center feature queries of external multimodal models (or keyword libraries) and assigned semantic labels to clusters.

[0171] Please refer to Figure 4 , Figure 4 The visual data management method for vehicle sentry mode provided in this application is illustrated in some other embodiments as a step flowchart.

[0172] like Figure 4As shown, in some embodiments, the visual data management method for vehicle sentry mode provided in this application may further include steps S401 and S402 as shown below.

[0173] Step S401: Aggregate the target feature vectors within the target cluster based on the aggregation function to obtain the core feature vector of the target cluster; the target cluster is any one of the plurality of clusters.

[0174] It should be noted that aggregate functions can be the mean, weighted average, median, and / or other forms.

[0175] After performing hierarchical clustering on visual data to obtain multiple clusters, the terminal device uses each cluster as a target cluster. Then, for any target cluster, an aggregation function is used to aggregate the target feature vectors within that target cluster to obtain the core feature vector of that target cluster.

[0176] In some embodiments, the terminal device can extract the commonalities of the target feature vectors within the target cluster through an aggregation function, thereby obtaining the "representative features" of the target cluster as the core feature vector.

[0177] Step S402: Based on the preset visual language big model, perform semantic transformation on the core feature vector to obtain the machine semantic label of the target cluster.

[0178] After obtaining the core feature vector of the target cluster, the terminal device can map the core feature vector into the implicit label of the target cluster. That is, based on the Vision-Language Model (VLM), the core feature vector is semantically transformed to generate the machine semantic label of the target cluster.

[0179] In some embodiments, the terminal device can also perform semantic alignment by mapping visual features (the core feature vectors of the target clusters) onto a linguistic feature space through semantic alignment using a multimodal large model (such as a visual-language large model, VLM), thereby generating implicit labels (machine semantic labels) for each cluster. For example, the terminal device trains a multimodal large model to map visual features onto a linguistic feature space, ensuring that both express similar semantic content in this common space. For a given dataset, it is first divided into multiple clusters using a clustering algorithm. Then, an implicit label is generated for each cluster; this process typically relies on the mapping relationship between features extracted from visual data and linguistic descriptions.

[0180] For example, suppose It represents a set of visual feature vectors (i.e., target feature vectors within a target cluster). This represents the corresponding set of text descriptions (linguistic features). A multimodal model is used. Mapping visual features to a language feature space:

[0181] .

[0182] in, It is the location of the mapped visual feature representation in the language feature space.

[0183] Then, these mapped visual features Mapped to k cluster classes That is, each cluster class implicit tags You can use aggregate functions Calculated using (mean, weighted average, or other forms of aggregation):

[0184] .

[0185] in, Cluster A certain data point within, and Cluster The number of data points in the data.

[0186] Please refer to Figure 5 , Figure 5 for Figure 1 A detailed flowchart of step S104.

[0187] like Figure 5 As shown, in some embodiments, step S104 above: establishing a semantic index database to manage the lifecycle of the visual data based on the implicit identifiers of the multiple clusters, may include steps S501 and S502 as shown below.

[0188] Step S501: Establish the association between the implicit identifiers of the multiple clusters and the metadata of the visual data to obtain a semantic index database; the metadata includes the fragment identifier data of the visual data, the mapping relationship between the visual data and the multiple clusters, and the original storage location data of the visual data.

[0189] It should be noted that the metadata of visual data can be the metadata of each segment of the visual data. That is, the metadata can include the segment identifier data (unique segment ID) of each segment of the visual data, the mapping relationship between each segment of the visual data and multiple clusters (the cluster identifier ID to which each segment belongs), and the original storage location data of each segment of the visual data (original data storage location pointer - the storage path, Uniform Resource Locator URL, etc. of each segment of the visual data).

[0190] When terminal devices manage the lifecycle of visual data by building a semantic index database based on the implicit identifiers of multiple clusters of visual data, they can establish the association between the implicit identifiers of each cluster and the metadata of each segment of visual data, thereby obtaining the semantic index database.

[0191] In some embodiments, when establishing a semantic index database for visual data, the terminal device can first define the creation schema of the semantic index database based on the clusters, the implicit identifiers of the clusters, and the attribute information of the metadata of each segment of visual data. That is, based on the data type of the cluster ID, the format of the machine semantic tag, and the specifications of the video segment metadata (such as the naming rules of the unique ID of the video segment and the format of the original data storage pointer—path / URL / hash value), the field format, data type, and association rules of the semantic index database are clarified, resulting in a schema definition document for the semantic index database. Then, the terminal device uses the schema definition document as a storage template to persistently store the implicit identifiers of the clusters and the metadata of each segment of visual data according to the schema structure, thereby realizing the chain association of "unique ID of video segment → cluster ID → machine semantic tag → original data pointer" (such as the table structure in a relational database or the document collection in a document database), thus establishing a semantic index database for visual data.

[0192] Step S502: Store the visual data in separate compartments based on the semantic index database, and perform semantic retrieval on the visual data stored in separate compartments based on the semantic index database.

[0193] After establishing a semantic index database for visual data, the terminal device can intelligently divide and store various segments of visual data based on the semantic index database. When it receives a query request from a user for visual data, it can respond to the query request and perform semantic retrieval on the visual data stored in the divided warehouse based on the semantic index database.

[0194] In some embodiments, the terminal device can, based on a semantic index database and the correlation between each segment of visual data and its cluster (i.e., high correlation within the same cluster), store highly correlated video segments (data within the same cluster) in the same physical or logical "data bucket," thereby achieving intelligent warehousing of visual data. Each physical or logical "data bucket" corresponds to one or a group of correlated clusters (usually the same cluster), storing the original data of all video segments within that cluster. Furthermore, the terminal device can simultaneously establish a mapping relationship between the "data bucket" and the cluster ID; this mapping relationship can be stored in the semantic index database or independently.

[0195] In this embodiment, machine semantic labels are assigned as implicit identifiers to multiple clusters of visual data via terminal devices. A semantic index database is then established based on these implicit identifiers to intelligently store the visual data in separate warehousing units. This allows for centralized storage of visual data related to similar events, facilitating management and significantly optimizing the efficiency of subsequent applications. For example, when an algorithm team needs a large amount of "vehicle collision" data to optimize the Automatic Emergency Braking (AEB) algorithm, high-quality positive samples can be directly extracted from the "collision event warehouse" in the semantic index database, eliminating the need for extensive manual and time-consuming data collection and greatly improving data utilization efficiency and value extraction speed.

[0196] Please refer to Figure 6 , Figure 6 for Figure 5 A detailed flowchart of step S502.

[0197] like Figure 6 As shown, in some embodiments, the step of “performing semantic retrieval of visual data stored in partitioned warehouses based on the semantic index database” in step S502 above may include steps S601 and S602 as shown below.

[0198] Step S601: Perform semantic similarity calculation on the query intent information corresponding to the visual data query statement and the visual data stored in the semantic index database to obtain the semantic similarity calculation result.

[0199] It should be noted that the visual data query statement can be a query request initiated by the user regarding visual data. For example, based on the user's actual needs for using the visual data, the user can input a query request (such as "all incidents involving electric bicycles scraping the left side door on x year x month x day") into the terminal device through a human-computer interaction interface (such as a graphical user interface). In this way, the terminal device can receive the query request and use the text content as the visual data query statement for querying the visual data stored in the semantic index database.

[0200] Furthermore, the semantic similarity between the query intent information calculated by the terminal device and the visual data can be cosine similarity.

[0201] After intelligently partitioning and storing visual data in a semantic index database, the terminal device can perform precise matching and searching of user queries through semantic retrieval. This allows it to respond to the user's query by returning indexed videos and related information highly relevant to the query intent. Based on this, upon receiving a visual data query, the terminal device first performs deep analysis to obtain the query intent information. Then, it calculates the semantic similarity between this query intent information and the visual data partitioned in the semantic index database. This semantic similarity calculation result can include the semantic similarity between the query intent information and each segment of the visual data (or all segments within each cluster).

[0202] Step S602: If the semantic similarity calculation result indicates that the semantic similarity between the query intent information and the target visual data is greater than or equal to a preset threshold, the target visual data is output as the query result of the visual data query statement; the target visual data is at least one fragment of visual data stored in the semantic index database in separate compartments.

[0203] It should be noted that the target visual data is a fragment of visual data stored in a compartmentalized manner in the semantic index database, or all fragments in a cluster of visual data stored in a compartmentalized manner in the semantic index database.

[0204] After obtaining the semantic similarity calculation result between the query intent information and the visual data, if the result indicates that the semantic similarity between the query intent information and the target visual data in the semantic index database is greater than or equal to a preset threshold, the terminal device will identify the target video data as visual data that is highly related to the query intent (which may include video clips, timestamps, tags, metadata, etc.), and then output the target visual data as the query result of the visual data query statement.

[0205] In some embodiments, when the terminal device outputs target visual data as a query result, it may also output related recommended content (such as video clips in similar scenes, data from different perspectives of the same event, etc.) as part of the query result.

[0206] In this embodiment, after the terminal device intelligently stores the visual data in a semantic index database, it can accurately match and search user queries through semantic retrieval. This responds to the user's query by returning indexed videos and related information highly relevant to the query intent. Thus, by employing semantic indexing and intelligent warehousing to transform unstructured visual data into a structured information database, it can support near-instantaneous retrieval at the natural language level, such as "find all right-side scrape events," allowing access to and extraction of high-value scene data. Furthermore, by understanding the content of each video segment of the visual data through the semantic index database, users can directly use advanced semantic queries such as "find all 'attempts to pull car doors' events" or "find all videos related to 'pets approaching'." The system then quickly locates the relevant "data warehouse" through semantic indexing, achieving millisecond-level retrieval, effectively solving the problem of traditional solutions' inefficient management and retrieval of visual data.

[0207] Next, a complete embodiment of the visual data management method for vehicle sentry mode provided in this application is presented.

[0208] Please refer to Figure 7 , Figure 7 The visual data management method for vehicle sentry mode provided in this application is illustrated in a schematic diagram of the overall system flow in a complete embodiment.

[0209] like Figure 7 As shown, the visual data management method for vehicle sentry mode provided in this application embodiment may include steps S701 to S705 as shown below.

[0210] Step S701: Data preprocessing of the original video: This includes video segmentation and keyframe sampling; feature extraction is performed using a self-supervised pre-trained model to output high-density feature vectors. This process aims to transform raw, unstructured video data into a structured, machine-readable sequence of high-density feature vectors. These feature vectors can efficiently represent the visual and temporal information of the video, providing a strong foundation for downstream tasks such as behavior recognition, video retrieval, and content analysis. The entire process is mainly divided into two core stages: data preprocessing and feature extraction.

[0211] It should be noted that the original video can be the visual data mentioned above, the self-supervised pre-trained model can be the self-supervised feature extraction model mentioned above, and the high-density feature vector can be the target feature vector of the visual data mentioned above.

[0212] Furthermore, the specific operations of data preprocessing and feature extraction are the same as those of the terminal device in the above embodiments for preprocessing visual data and extracting features based on a self-supervised feature extraction model, and the same content will not be described again here.

[0213] Step S702: Perform frame-level clustering calculation, clustering the feature vectors of all frames and outputting the static visual clustering results. That is, frames with similar features are grouped into the same cluster based on similarity metrics. The goal is to represent each frame in the video as a feature vector and obtain cluster centers and cluster partitioning results using an unsupervised feature vector clustering algorithm. Based on the clustering results, frames belonging to the same cluster are grouped together to form a static visual cluster.

[0214] It should be noted that the static visual cluster is the same as the static visual clustering cluster mentioned above. The specific operation of frame-level clustering here is the same as the operation of the terminal device to perform frame-level clustering processing on visual data, and the same content will not be repeated here.

[0215] Step S703: Perform segment-level temporal feature fusion, aggregate cluster information of all frames within the segment time period, and output dynamic event cluster clustering results. That is, divide the video / sequence into multiple time segments by time window division, perform frame-level feature clustering analysis within the segment, and aggregate dynamic event clusters using temporal information.

[0216] It should be noted that the dynamic event cluster is the same as the dynamic event clustering cluster mentioned above. The specific operation of fragment-level clustering here is the same as the operation of the terminal device in performing fragment-level clustering processing on visual data, and the same content will not be repeated here.

[0217] Step S704: Through semantic alignment of the multimodal large model, visual features are mapped onto the language feature space and semantically aligned, generating implicit labels for each cluster; data is stored, along with the mapping relationship between cluster labels and data pointers. That is, each cluster generated after clustering is assigned a machine-generated semantic label. Subsequently, instead of simply storing the original videos by time or vehicle ID, a semantic index database is established. This database records the unique ID of each video segment, its cluster ID, machine semantic label, and a pointer to the original data storage location. This physically stores highly correlated video segments (data in the same cluster) in the same physical or logical "data bucket," achieving "intelligent data warehousing."

[0218] It should be noted that the implicit tags can be the implicit identifiers mentioned above. The specific operations of generating implicit tags and building a semantic index database for intelligent warehousing of video data are the same as the operations of the terminal device generating implicit identifiers for clusters based on the visual language big model and building a semantic index database based on implicit identifiers for warehousing of visual data. Therefore, the same content will not be described again.

[0219] Step S705: Using semantic retrieval technology, achieve precise matching and searching of user queries. If results highly relevant to the query intent are retrieved, the corresponding indexed video and related information are returned. That is, by performing deep semantic parsing on the query statement, the system understands the user's true intent and context, thereby finding semantically similar content in the vector space. This is achieved by calculating the semantic similarity (such as cosine similarity) between the query and candidate content and setting a threshold. If the similarity exceeds the preset threshold, the system returns the most matching indexed video (which may include video clips, timestamps, tags, metadata, etc.). The returned results may be accompanied by recommendations of related content (such as similar scenes, different perspectives on the same event, etc.).

[0220] In this embodiment, through self-supervised feature extraction and unsupervised hierarchical clustering, the self-supervised pre-trained model learns autonomously from the data without any manual annotation, achieving an end-to-end fully automated pipeline from feature extraction to event classification. Unsupervised clustering automatically discovers inherent patterns in the data, enabling the system to autonomously summarize and discover unknown new event types, and possesses the ability to self-evolve and continuously learn. Furthermore, by employing a hierarchical clustering strategy combined with general features, dynamic event sequences and subtle static features can be identified, achieving deep semantic understanding of "who, what, what, and why." Moreover, semantic indexing and intelligent warehousing transform unstructured data into a structured information database, supporting natural language-level, near-instantaneous retrieval, allowing users to efficiently access and extract the high-value scene data they need. Therefore, this embodiment constructs an automated, intelligent, fine-grained, and scalable massive vehicle sentry visual data understanding and management system, overcoming the inherent shortcomings of traditional manual or rule-based solutions in terms of efficiency, objectivity, granularity, and flexibility, thereby transforming raw video data into truly efficient and usable structured information assets.

[0221] Please refer to Figure 8 This application also provides a visual data management device for vehicle sentry mode.

[0222] The visual data management device for vehicle sentry mode provided in this application embodiment can implement the above-described visual data management method for vehicle sentry mode.

[0223] like Figure 8As shown in the embodiments of this application, the visual data management device for vehicle sentry mode may include:

[0224] The acquisition module is used to acquire visual data in vehicle sentry mode;

[0225] The self-supervised feature extraction module is used to perform feature extraction processing on the visual data based on the self-supervised feature extraction model to obtain the target feature vector of the visual data; the self-supervised feature extraction model is obtained by self-supervised learning training on unlabeled sentinel visual sample data based on a general visual feature extraction model.

[0226] The hierarchical clustering module is used to perform hierarchical clustering processing on the visual data based on the target feature vector to obtain multiple clusters of the visual data;

[0227] The semantic indexing module is used to establish a semantic index database based on the implicit identifiers of the multiple clusters to manage the lifecycle of the visual data.

[0228] In some embodiments, the target feature vector includes a multidimensional feature vector for each frame of the visual data;

[0229] The hierarchical clustering module is further configured to perform frame-level clustering processing on the visual data based on the multidimensional feature vectors to obtain static visual clusters of the visual data; the multiple clusters of the visual data include the static visual clusters; perform segmented fusion processing on the multidimensional feature vectors based on at least one time segment of the visual data to obtain frame feature vector fusion results of the at least one time segment; perform segment-level clustering processing on the visual data based on the frame feature vector fusion results to obtain dynamic event clusters of the visual data; the multiple clusters of the visual data include the dynamic event clusters.

[0230] In some embodiments, the hierarchical clustering module is further configured to perform similarity measurement processing on the multidimensional feature vectors of each frame of the visual data to obtain a similarity measurement result; perform unsupervised clustering processing on the multidimensional feature vectors based on the similarity measurement result to obtain a cluster partitioning result of the multidimensional feature vectors; the cluster partitioning result includes at least one feature cluster; and perform cluster matching processing on each frame of the visual data based on the at least one feature cluster to obtain at least one static visual cluster of the visual data.

[0231] In some embodiments, the hierarchical clustering module is further configured to perform cluster analysis on the frame feature vector fusion result to obtain cluster information for each frame within each time segment of the visual data; and to aggregate the cluster information based on the temporal relationship of each time segment of the visual data to obtain dynamic event clusters of the visual data.

[0232] In some embodiments, the implicit identifier includes machine semantic tags. The visual data management device for vehicle sentry mode provided in this application embodiment further includes:

[0233] An implicit label generation module is used to perform aggregation calculations on target feature vectors within a target cluster based on an aggregation function to obtain the core feature vector of the target cluster; the target cluster is any one of the plurality of clusters; and to perform semantic transformation on the core feature vector based on a preset visual language big model to obtain the machine semantic label of the target cluster.

[0234] In some embodiments, the semantic indexing module is further configured to establish the association between the implicit identifiers of the plurality of clusters and the metadata of the visual data to obtain a semantic indexing database; the metadata includes fragment identifier data of the visual data, the mapping relationship between the visual data and the plurality of clusters, and the original storage location data of the visual data; the visual data is stored in separate warehouses based on the semantic indexing database, and semantic retrieval is performed on the visual data stored in separate warehouses based on the semantic indexing database.

[0235] In some embodiments, the semantic indexing module is further configured to perform semantic similarity calculation on the query intent information corresponding to the visual data query statement and the visual data stored in the semantic index database in separate compartments, to obtain a semantic similarity calculation result; and, if the semantic similarity calculation result indicates that the semantic similarity between the query intent information and the target visual data is greater than or equal to a preset threshold, output the target visual data as the query result of the visual data query statement; the target visual data is at least one fragment of the visual data stored in the semantic index database in separate compartments.

[0236] In some embodiments, the acquisition module is further configured to perform keyframe sampling on the target segment of the visual data to obtain a keyframe sequence; the target segment is at least one of multiple segments of the visual data; and to perform feature extraction processing on at least one keyframe in the keyframe sequence based on a self-supervised feature extraction model, and to fuse the extracted multidimensional feature vectors of the keyframes into a multidimensional feature vector of the target segment based on the self-supervised feature extraction model for output; the target feature vector of the visual data includes the multidimensional feature vector of the target segment.

[0237] It should be noted that the specific implementation of the visual data management device for vehicle sentry mode provided in this application is basically the same as the specific implementation of the visual data management method for vehicle sentry mode described above, and will not be repeated here.

[0238] Please see Figure 9 This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-mentioned visual data management method for vehicle sentry mode.

[0239] In some embodiments, the electronic device can be any smart terminal such as a tablet computer, smartphone, in-vehicle hardware platform (e.g., in-vehicle computer), or wearable device.

[0240] like Figure 9 As shown, the electronic device provided in this application embodiment may include:

[0241] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0242] The memory 902 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 to execute the visual data management method of the vehicle sentry mode in the embodiments of this application.

[0243] The input / output interface 903 is used to implement information input and output;

[0244] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0245] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0246] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0247] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described visual data management method for vehicle sentry mode.

[0248] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0249] This application also provides a computer program product, including a computer program. The steps implemented by the computer program when executed by a processor are basically the same as those in the specific embodiments of the visual data management method for vehicle sentry mode described above, and will not be repeated here.

[0250] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0251] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0252] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0253] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0254] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0255] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0256] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0257] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0258] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0259] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0260] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A visual data management method for vehicle sentry mode, characterized in that, The method includes: Acquire visual data in vehicle sentry mode; The visual data is processed by a self-supervised feature extraction model to obtain the target feature vector of the visual data; the self-supervised feature extraction model is trained by self-supervised learning on unlabeled sentinel visual sample data based on a general visual feature extraction model; the target feature vector includes the multi-dimensional feature vector of each frame of the visual data. Based on the multidimensional feature vectors, the visual data is subjected to frame-level clustering to obtain static visual clusters of the visual data. Based on at least one time segment of the visual data, the multidimensional feature vector is segmented and fused to obtain the frame feature vector fusion result of the at least one time segment. Based on the frame feature vector fusion result, the visual data is segment-level clustered to obtain the dynamic event cluster of the visual data. Based on the implicit identifiers of multiple clusters of the visual data, a semantic index database is established to manage the lifecycle of the visual data; the multiple clusters of the visual data include the static visual clusters and the dynamic event clusters; the implicit identifiers include machine semantic tags.

2. The method according to claim 1, characterized in that, The frame-level clustering processing of the visual data based on the multidimensional feature vector includes: The multidimensional feature vector of each frame of the visual data is subjected to similarity measurement processing to obtain the similarity measurement result; Based on the similarity measurement results, unsupervised clustering is performed on the multidimensional feature vectors to obtain the cluster partitioning results of the multidimensional feature vectors; the cluster partitioning results include at least one feature cluster. Based on the at least one feature cluster, cluster matching processing is performed on each frame of the visual data to obtain at least one static visual cluster of the visual data.

3. The method according to claim 1, characterized in that, The segment-level clustering processing of the visual data based on the frame feature vector fusion result includes: Cluster analysis is performed on the frame feature vector fusion results to obtain the cluster information of each frame in each time segment of the visual data; The cluster information is aggregated based on the temporal relationship of each time segment of the visual data to obtain the dynamic event clusters of the visual data.

4. The method according to claim 1, characterized in that, The method further includes: The target feature vectors within the target cluster are aggregated using an aggregation function to obtain the core feature vector of the target cluster; the target cluster is any one of the plurality of clusters. Based on a pre-defined visual language model, the core feature vectors are semantically transformed to obtain machine semantic labels for the target cluster.

5. The method according to claim 1, characterized in that, The implicit identifiers of multiple clusters based on the visual data are used to establish a semantic index database for lifecycle management of the visual data, including: A semantic index database is obtained by establishing the association between the implicit identifiers of each of the multiple clusters and the metadata of the visual data; the metadata includes the fragment identifier data of the visual data, the mapping relationship between the visual data and the multiple clusters, and the original storage location data of the visual data. The visual data is stored in separate compartments based on the semantic index database, and semantic retrieval is performed on the visual data stored in separate compartments based on the semantic index database.

6. The method according to claim 5, characterized in that, The semantic retrieval of visual data stored in separate warehouses based on the semantic index database includes: The semantic similarity is calculated between the query intent information corresponding to the visual data query statement and the visual data stored in the semantic index database in separate compartments, and the semantic similarity calculation result is obtained. If the semantic similarity calculation result indicates that the semantic similarity between the query intent information and the target visual data is greater than or equal to a preset threshold, the target visual data is output as the query result of the visual data query statement; the target visual data is at least one fragment of visual data stored in the semantic index database in separate compartments.

7. The method according to any one of claims 1 to 6, characterized in that, The feature extraction process for the visual data based on the self-supervised feature extraction model includes: Keyframe sampling is performed on the target segment of the visual data to obtain a keyframe sequence; the target segment is at least one of multiple segments of the visual data. The self-supervised feature extraction model is used to extract features from at least one key frame in the key frame sequence, and the extracted multidimensional feature vectors of the key frames are fused into a multidimensional feature vector of the target segment for output. The target feature vector of the visual data includes the multidimensional feature vector of the target segment.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the visual data management method for the vehicle sentry mode according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the visual data management method for the vehicle sentry mode as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the visual data management method for the vehicle sentry mode as described in any one of claims 1 to 7.

Images