Video data processing method, vehicle, equipment and program product

By performing multi-dimensional semantic feature extraction and associated storage of video data, the problem of automatic labeling and mining of video data is solved, high-precision retrieval is realized, suitable for autonomous driving and other video data processing scenarios, and data mining efficiency and resource utilization are improved.

CN118969011APending Publication Date: 2024-11-15BEIJING JIDU TECH CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410947237.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-15
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In the prior art, automatic labeling and mining of video data is difficult to achieve, especially in large amounts of video data, manual labeling efficiency is low and long-tail scene coverage of autonomous driving is difficult. The existing methods are limited to static images or specific target retrieval and cannot be effective. Mining high-value data.

Method used

By extracting video data in a multi-dimensional semantic feature, including object-level, whole-graph-level and video-level semantic features, extracting and splicing based on image modalities, storing and association, diversified retrieval, improving data accuracy, and avoiding additional modalities Model processing.

Benefits of technology

It realizes high-precision multi-dimensional retrieval of video data, improves data mining efficiency, ensures consistency of semantic features, reduces resource investment, and is suitable for autonomous driving and other video data processing scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118969011A_ABST
    Figure CN118969011A_ABST
Patent Text Reader

Abstract

The invention provides a video data processing method, a vehicle, equipment and a program product, and relates to the technical field of computers. According to the method and the device, multi-dimensional semantic feature extraction can be performed on the video data to obtain different-dimensional semantic features such as object-level semantic features, whole-graph-level semantic features and video-level semantic features of the video data. Wherein the object-level semantic feature reflects a target object appearing in the video, the whole-graph-level semantic feature reflects a single video frame in the video data, and the video-level semantic feature reflects a video segment in the video data. In this way, diversified retrieval can be executed through the semantic features of the corresponding dimensions, and therefore the data precision of the video data is improved. Besides, the video-level semantic features can be determined based on the spliced image of the video frame, that is, the semantic features of each dimension can be extracted based on the image, and semantic extraction does not need to be carried out by additionally adopting models of other modalities, so that the semantic consistency of the semantic features of each dimension is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a video data processing method, vehicle, device and program product. Background Art

[0002] With the development of information technology, monitoring equipment is widely present in daily life and can provide assistance for human activities. For example, the vehicle monitoring system can obtain monitoring video data of the vehicle's surrounding environment, so that the processor in the vehicle can perform assisted driving functions based on the monitoring video data.

[0003] In order to mine key information from video data, it is often necessary to manually label the video data in the existing technology. However, with the increase in monitoring equipment, the amount of video data generated also increases. It is difficult to manually label a large amount of video data.

[0004] Therefore, how to analyze and mine video data is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a video data processing method, vehicle, device and program product, which determine a variety of semantic feature video data that are easy to retrieve and mine through semantic recognition of video data.

[0006] In a first aspect, the present application provides a method for processing video data. The method for processing video data includes: obtaining video data to be processed. Performing semantic feature extraction on the video data to obtain multiple semantic features of the video data, wherein the multiple semantic features include object-level semantic features, whole-image-level semantic features, and video-level semantic features. The object-level semantic features reflect the semantic features of the target object included in the video data, the whole-image-level semantic features reflect the semantic features of a single video frame in the video data, and the video-level semantic features reflect the semantic features of a spliced ​​image obtained by splicing multiple video frames in the video data. The multiple semantic features are associated with the video data and stored in a database.

[0007] In a second aspect, the present application provides an electronic device, comprising: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the processor executes the video data processing method described in the first aspect.

[0008] In a third aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a computer program / instruction processor, performs the video data processing method described in the first aspect. Optionally, the computer program may be stored in a readable storage medium of a computer device or in the cloud; the processor of the computer device reads the computer program from the readable storage medium or the cloud.

[0009] In a fourth aspect, the present application provides a vehicle, the vehicle comprising an in-vehicle voice system, at least one processor, and a memory in communication with the at least one processor. The memory stores instructions that can be executed by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is used to obtain video data through the vehicle monitoring system, and execute the video data processing method of any one of claims 1-12 based on the video data.

[0010] The present application provides a video data processing method, vehicle, device and program product. In the present application, multi-dimensional semantic feature extraction can be performed on video data to obtain semantic features of different dimensions such as object-level semantic features, whole-image-level semantic features and video-level semantic features of the video data. Among them, the object-level semantic features reflect the target objects appearing in the video, the whole-image-level semantic features reflect the single video frames in the video data, and the video-level semantic features reflect the video segments in the video data. Therefore, diversified retrieval can be performed through the semantic features of the corresponding dimensions, thereby improving the data accuracy of the video data. In addition, the aforementioned video-level semantic features can be determined based on the spliced ​​images of the video frames, that is, the aforementioned semantic features of each dimension can be extracted based on the image, without the need to use additional models of other modalities for semantic extraction, thereby ensuring the semantic consistency of the semantic features of each dimension. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 It is an application scenario diagram of a video data processing method provided by an exemplary embodiment of the present application.

[0013] Figure 2 It is an exemplary flow chart of a video data processing method provided by an exemplary embodiment of the present application.

[0014] Figure 3 It is an exemplary flow chart of a video data retrieval method provided by an exemplary embodiment of the present application.

[0015] Figure 4 It is an exemplary flow chart of a method for determining video-level semantic features provided by an exemplary embodiment of the present application.

[0016] Figure 5 It is an exemplary flow chart of a tracking semantic feature determination method provided by an exemplary embodiment of the present application.

[0017] Figure 6 It is an exemplary flow chart of a method for determining whole-image-level semantic features provided by an exemplary embodiment of the present application.

[0018] Figure 7 It is an exemplary flow chart of a semantic feature association storage method provided by an exemplary embodiment of the present application.

[0019] Figure 8 It is a system module diagram of a data processing system provided by an exemplary embodiment of the present application.

[0020] Fig. 9 It is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] Application Scenario

[0023] To further illustrate the video data processing method, the present application also provides an application scenario diagram of the video data processing method ( Figure 1 ).

[0024] like Figure 1 As shown, the application scenario 100 mainly includes a data source 110, a database 120 and a user terminal 130. The data source 110, the database 120 and the user terminal 130 may be connected via a network wired / wireless connection.

[0025] The data source 110 may be a signal source for acquiring video data. For example, the data source 110 may be a monitoring device, a camera, a still camera, etc., thereby generating video data in real time. In some embodiments, the data source 110 in the present application may be a vehicle-mounted monitoring system, and the corresponding video data may be monitoring video data of the surrounding area of ​​the vehicle, and may also include monitoring video data inside the vehicle.

[0026] The database 120 may be a collection of devices for storing video data. The video data acquired by the data source 110 may be transmitted to the database 120 for storage. In some embodiments, the database 120 may include a local storage device of the data source 110. For example, the database 120 may include an on-board memory to cache the monitoring video data generated by the data source 110 (on-board monitoring system). In some embodiments, the database 120 may also include a cloud storage device of the data source 110. For example, the database 120 may include a cloud server that periodically receives the monitoring video data generated by the data source 110 (on-board monitoring system).

[0027] The user terminal 130 may be an interactive device used by the staff. In the prior art, the user terminal 130 may be a terminal device including a video annotation system. At this time, the staff may annotate the video data stored in the database 120 through the user terminal 130 .

[0028] However, in reality, with the increase of video data and the complexity of scenes, manual labeling can no longer process video data in a timely manner.

[0029] Take autonomous driving technology (including assisted driving technology) as an example. With the development of artificial intelligence technology, autonomous driving is gradually being implemented. At present, L2 autonomous driving has been mass-produced, but moving towards higher-level autonomous driving requires solving the stability and reliability problems of the system in long-tail scenarios. The iteration of mainstream autonomous driving algorithms based on neural network technology requires a large amount of data (especially video data and the video frames it contains) to train and verify the algorithms. Data-driven has become the underlying logic of automotive intelligence. However, not all data is valuable for the iteration of intelligent driving algorithms, and how to mine high-value data has also become a major issue facing the industry. The classic method is to label the data manually or through algorithms, and then perform label retrieval through the database to obtain high-value data. However, the long-tail scenarios of autonomous driving are too numerous to mention, and the label-based approach requires continuous investment of a large amount of resources to cover these scenarios.

[0030] In order to realize automatic mining of video data, mining methods based on image feature retrieval have emerged in related technologies. That is, the image features of each frame data in the video data can be extracted through the model, and then the image feature retrieval corresponding to the retrieval conditions is determined, and the retrieval is performed through the image features. However, in actual use, this method can generally only retrieve static images or specific targets in the image, which has obvious limitations.

[0031] Based on the above technical problems, the present application provides a method for processing video data. In the present application, multi-dimensional semantic feature extraction can be performed on video data to obtain semantic features of different dimensions such as object-level semantic features, whole-image-level semantic features, and video-level semantic features of video data. Among them, object-level semantic features reflect the target objects appearing in the video, whole-image-level semantic features reflect a single video frame in the video data, and video-level semantic features reflect the video segments in the video data. Thus, diversified retrieval can be performed through the semantic features of the corresponding dimensions, thereby improving the data accuracy of the video data. In addition, the aforementioned video-level semantic features can be determined based on the spliced ​​images of the video frames, that is, the aforementioned semantic features of each dimension can be extracted based on the image, without the need to use additional models of other modalities for semantic extraction, thereby ensuring the semantic consistency of the semantic features of each dimension.

[0032] Specifically, in the aforementioned application scenario 100, based on the video data processing method provided by the present application, the relevant processing device can process the video data in the database 120 to determine the semantic features of multiple dimensions of the video data. Specifically, at least target-level semantic features that reflect the semantics of the target object appearing in the video, whole-image-level semantic features that reflect the content of a single image, and video-level semantic features that reflect the dynamic features of the video can be included. Each semantic feature can be determined based on a semantic extraction model of the same modality so that each feature has consistent semantics.

[0033] The relevant staff can perform subsequent processing on the video data stored in the database 120 through the user terminal 130. The subsequent processing by the staff is generally based on the retrieval of the video data. That is, the staff can input the retrieval conditions, thereby retrieving the videos / images that meet the requirements from the video data based on the feature set, and then perform subsequent processing (such as labeling, specialized training, etc.). For example, the staff can use "collision accident" as the retrieval condition to retrieve the accident clip from the video data.

[0034] Combine the following Figure 2-9 The video data processing method provided in this application is described in detail.

[0035] Exemplary video data processing method

[0036] Figure 2 It is an exemplary flow chart of a video data processing method provided by an exemplary embodiment of the present application. Among them, P200 can be recorded as a video storage method, a video feature mining method, a video feature extraction method, etc.

[0037] In some embodiments, Figure 2The process P200 shown can be executed by a device including a processing device (such as a server) to achieve feature extraction of video data. Figure 2 The process P200 shown can be executed by a vehicle. The vehicle may include an on-board monitoring system, at least one processor, and a memory in communication with the at least one processor. The memory stores instructions that can be executed by the at least one processor. When the instructions are executed by the at least one processor, the at least one processor is used to obtain monitoring video data of the vehicle's surroundings through the on-board monitoring system, and execute the video data processing method described in P200 based on the monitoring video data, so as to train the relevant models of the vehicle's assisted driving function / automatic driving function.

[0038] like Figure 2 As shown, P200 may include the following steps:

[0039] S210: Obtain video data to be processed.

[0040] S220: Perform semantic feature extraction on the video data to obtain multiple semantic features of the video data.

[0041] S230: Associating the plurality of semantic features with the video data and storing them in a database.

[0042] The video data to be processed in the aforementioned S210 (hereinafter referred to as video data) may refer to the execution object of P200, that is, P200 may extract features from the video data to be processed and store them in a database. The video data may refer to a data format composed of multiple continuous video frames, that is, the video data includes multiple continuous frame data.

[0043] In some embodiments, the processing device may directly call the video data from the storage medium or receive the corresponding video data from the data source to implement the aforementioned S210. In some embodiments, considering the accuracy of data processing, the video data may be a sampling result of the original data. For example, the original data may be 2k resolution 144hz original data. To reduce the amount of data during video data processing, the video data may reduce the resolution of the original data (such as converting the 2k resolution to 360p) and the sampling frequency (such as 10hz), thereby reducing the amount of video data.

[0044] In some embodiments, when the video data is surveillance video data of the surroundings of a vehicle, the video data may be segmented according to the driving conditions of the vehicle.

[0045] For example, when acquiring video data, the overall video data of the vehicle during the journey can be determined first. The overall video data is then split based on the road section conditions to determine the road section video data of each road section. The road section video data may include the road section between two intersections and the surveillance video data at the intersection. Thus, when performing data mining, the video data of each road section is used as video data and is stored in association when stored.

[0046] In some embodiments, considering the integrity of the overall video data, each road segment may be sampled when determining the video data, thereby forming overall sampled video data including the sampling results of each road segment as the video data.

[0047] In the aforementioned S220, semantics may refer to elements in language or data that can reflect its meaning and logical relationship. For example, the semantics of text may include information such as the meaning and grammar of the text. The semantics of an image may include the attributes and states of objects in the image and the relationships between them.

[0048] Semantic features can be quantitative descriptions of key semantics in data. That is, semantic features can be used to describe the semantics in the corresponding data. If two data have the same or similar semantic features, it means that the semantics of the two data are similar.

[0049] In some embodiments, to facilitate the subsequent retrieval and processing of video data, the semantic features involved in this application may be cross-modal semantic features determined based on a multimodal model. Among them, modality may refer to different data types, that is, different types of data in this application may be processed by the same multimodal model to determine semantic features. At this time, the semantic features of data in different modalities are consistent. For example, the text "There is a person in a red shirt in the picture" is similar to the semantic features of the picture data containing a person in a red shirt.

[0050] In some embodiments, a multimodal model for semantic feature extraction can generally input data of different modalities and output its semantic features. At the implementation level, a multimodal model can generally include two parts: feature extraction and semantic clustering. When data is input into a multimodal model, it can be processed by a feature extraction module to determine quantified feature data, and then processed by semantic clustering to determine multiple semantics that match the feature data as semantic features. Considering the diverse inputs of the multimodal model, feature extraction can be performed based on classification of data of different modalities.

[0051] In some embodiments, it is considered that when a multimodal model extracts semantic features of different modalities, it is necessary to align their semantics so that the semantic features of the different modal data output by the model reflect the same semantics. In traditional multimodal models, video data and image data are generally regarded as different modalities, and thus their semantic features also need to be aligned.

[0052] To solve this technical problem, the present application creatively proposes that, when determining the overall semantic features of video data, images of multiple video frames of the video data can be spliced ​​to determine a spliced ​​image that can reflect the overall video data, and then semantic features of the spliced ​​image can be extracted in an image modality to determine the semantic features of the spliced ​​image to represent the semantics of the corresponding video data (or video clip).

[0053] Based on the aforementioned semantic feature extraction method, the semantic features of the video data may at least include object-level semantic features, whole-image-level semantic features, and video-level semantic features.

[0054] The object-level semantic features reflect the semantic features of the target objects included in the video data. The target objects may be specific objects (such as people, cars, buildings, etc.) that appear in the video. That is, target recognition may be performed on each video frame of the video data to determine the image data of each target in the video frame, and then semantic recognition may be performed on the image data to determine the corresponding object-level semantic features, so as to facilitate the retrieval of the objects that appear in the video.

[0055] In some embodiments, the target object can be set in advance by a preset type of the target recognition algorithm. For example, a person, a car, and a road sign can be marked as an identification object in the target recognition algorithm, and the corresponding target object is the person, car, and road sign extracted from the video frame.

[0056] In some embodiments, considering that the result of target recognition is generally represented in the video frame by a bounding box surrounding the recognized object, when determining the image corresponding to the target object, the image corresponding to the bounding box can be expanded (for example, the bounding box can be proportionally enlarged to 120%) to completely include the target object and its surrounding environment.

[0057] The whole-image semantic feature reflects the semantic features of a single video frame in the video data. The whole-image semantic feature can be the semantic extraction result of each video frame to reflect the semantics of the corresponding video frame. Considering the content contained in the video frame, the whole-image semantic feature can reflect the static environment semantic features in addition to the target object contained in the video frame.

[0058] Video-level semantic features reflect the semantic features of a spliced ​​image obtained by splicing multiple video frames in the video data. That is, when determining video-level semantic features, the corresponding multiple video frames can be spliced ​​first to determine the spliced ​​image, and then semantic extraction can be performed to determine the semantic features of the corresponding video data or video clips. Therefore, the semantic features of the video can also be semantically extracted in the image mode to avoid the processing requirements of the video mode and reduce the training amount of the relevant model during training (especially semantic alignment).

[0059] In some embodiments, the stitched image can be implemented based on a conventional image stitching algorithm. For example, the video frames can be directly arranged on a large image according to the time sequence of the video frames to determine the stitched image. In order to distinguish the video frames, interval marks can be added to the stitched image to distinguish the video frames. For example, a white border of a certain width exists between the video frames in the stitched image to indicate that the images within each white border in the stitched image are one image frame.

[0060] In some embodiments, to further improve the representation capability of the stitched image, the stitching process of the stitched image can be performed by an image stitching model. The image stitching model can be a trained machine learning model. To further improve the representation capability of the stitched image, the aforementioned image stitching model can be jointly trained with a related model for semantic extraction.

[0061] In some embodiments, the aforementioned splicing process may be performed based on specific video frames in the video data. That is, multiple target video frames for splicing may be selected from the video data first, and then the splicing may be performed. For the specific process, see Figure 4 and its related description.

[0062] It should be noted that the determination of the aforementioned stitched image can also be applied to the determination process of object-level semantic features and whole-image-level semantic features. For example, to determine the motion trajectory of a target object in a video, the video frames containing the target object can be stitched together to determine the tracking semantic features of the target object. For details, see Figure 5 For another example, when the relevant technology is applied in a vehicle, the vehicle can obtain video data from multiple directions. In order to fully analyze the surrounding situation, the video frames obtained by the same type of sensor at the same time can be spliced ​​as a scene splicing image to determine the corresponding semantic features. For details, please refer to Figure 6 and its related description.

[0063] In the aforementioned S230, the associated storage may refer to the stored data being associated with other data for easy calling. For example, when storing data A, the identification information (such as a pointer) of data B may be stored in the database as part of data A, so that when data A is called, the corresponding data B may be read based on its stored identification information.

[0064] In some embodiments, considering that semantic features are generally used as a basis for retrieval to call corresponding video data, the associated storage in the aforementioned S230 can be understood as the associated storage of semantic features and identification information of corresponding video data, so that when the semantic features are called, the corresponding video data can be called based on the stored identification information.

[0065] In some embodiments, the correspondence between semantic features and video data can be determined based on the specific video frame to which they correspond. For example, object-level semantic features and whole-image-level semantic features generally correspond to only one video frame, so a pointer to the video data and a time identifier (such as a timestamp) of the corresponding video frame can be associated and stored with such semantic features. For another example, a video-level semantic feature can correspond to a segment of video data, so a pointer to the segment of video data and a time range identifier can be associated and stored.

[0066] In some embodiments, considering that the subsequent processing of video data generally depends on the retrieval of semantic features, and the amount of video data is large, the semantic features can be classified during retrieval to prioritize the retrieval of semantic features corresponding to multiple video frames, thereby speeding up the retrieval process. Figure 7 and its related description.

[0067] Therefore, based on the relevant steps of the aforementioned P200, multi-dimensional semantic feature extraction can be performed on the video data to obtain semantic features of different dimensions such as object-level semantic features, whole-image-level semantic features, and video-level semantic features of the video data. Among them, the object-level semantic features reflect the target objects appearing in the video, the whole-image-level semantic features reflect the individual video frames in the video data, and the video-level semantic features reflect the video segments in the video data. Therefore, diversified retrieval can be performed through the semantic features of the corresponding dimensions, thereby improving the data accuracy of the video data. In addition, the aforementioned video-level semantic features can be determined based on the spliced ​​images of the video frames, that is, the aforementioned semantic features of each dimension can be extracted based on the image, without the need to use additional models of other modalities for semantic extraction, thereby ensuring the semantic consistency of the semantic features of each dimension.

[0068] Exemplary video data retrieval method

[0069] Considering the huge amount of video data in practical applications, the screening of valid video data and other subsequent processing can realize the retrieval of video data based on the aforementioned semantic features. To further illustrate the process, the present application also provides an exemplary flow chart of a video data retrieval method ( Figure 3 ).

[0070] In some embodiments, Figure 3 The process P300 shown can be executed based on the video data processed by the aforementioned P200. For example, the staff can execute P300 in a database storing the processed video data through a user terminal (such as the user terminal 130).

[0071] like Figure 3 As shown, P300 may include the following steps:

[0072] S310: Obtain search conditions.

[0073] S320: Extract features of the search conditions to determine search semantic features of the search conditions.

[0074] S330 : Determine a target semantic feature matching the search semantic feature from a database based on the similarity with the search semantic feature.

[0075] S340: Output a single video frame corresponding to the target semantic feature, and / or a search result including video data of multiple video frames as the search condition.

[0076] Based on the aforementioned P300, it can be known that the search conditions can be subjected to the same semantic extraction as the video data during retrieval. For example, semantic feature extraction can be performed based on the same multimodal model. Thus, the search semantic features of the search conditions have a high degree of similarity with the multidimensional semantic features of the video data in a corresponding sense, and then matching features can be determined based on the similarity (such as selecting one or more features with high similarity as matching features), thereby using the corresponding video frame / video data as the search result of the search condition.

[0077] Considering that the retrieval condition and the video data share the same semantic extraction algorithm, the data modality of the retrieval condition can correspond to the feature extraction model. That is, the retrieval condition includes at least one of image, video and text. It should be noted that when the retrieval condition is a video, the retrieval condition can be processed using the same splicing technology before semantic extraction.

[0078] As mentioned above, when the search condition is to search for images, it can be understood that the search process is to search for images by images / search for videos by images. When the search condition is to search for videos, it can be understood that the search process is to search for images by videos / search for videos by videos. When the search condition is to search for text, it can be understood that images or videos are searched based on the semantic information of the search text.

[0079] In some embodiments, the search condition may be a composite condition consisting of multiple search conditions. When determining the search result, a Boolean operation may be performed based on the relationship between the various search conditions to determine the search result of the composite condition. For example, the search condition may include both a search image (such as an image of a red vehicle) and a search text (such as requiring a collision in the video) and the search result must meet these two conditions. The two search conditions may be searched separately and their results may be intersected to determine the search result of the composite condition (such as a video of a red vehicle in a search image when a collision occurs).

[0080] In some embodiments, the video data may be subsequently processed based on the aforementioned retrieval process. Considering that video data is generally used for machine learning, the video data may be annotated based on the retrieval process shown in P300.

[0081] That is, the aforementioned P300 may also include:

[0082] S350: Generate annotation information based on the search condition, and index the search result based on the annotation information.

[0083] In some embodiments, the search condition may correspond to the annotation information. For example, when searching for annotated video data, the search condition may be determined based on the corresponding annotation requirements. The annotation information corresponding to the annotation condition may be determined based on the corresponding relationship. After the annotation information is determined, the video data (or part of the video data) corresponding to the search result may be indexed (e.g., adding a variable in a specific form) so as to be used as a training sample for model training.

[0084] In some embodiments, the indexing process shown in P300 above can be automatically executed. For example, the staff can pre-set the annotation requirements to automatically execute the search-annotation process. Exemplarily, the preset annotation requirements can be automatically annotating the scene where the collision occurs. The corresponding search condition can be a collision occurs. The search results determined based on the search condition can add a label that there is a collision.

[0085] Exemplary semantic feature determination method

[0086] In some embodiments, a method for determining semantic features is described, especially a method for determining semantic features based on a spliced ​​image. The present application also provides a variety of methods for determining semantic features. Figure 4 It reflects the process of determining video-level semantic features. Figure 5 It reflects the process of determining the tracking semantic features. Figure 6 It reflects the process of determining the semantic features of the entire image.

[0087] Figure 4It is an exemplary flow chart of a method for determining video-level semantic features provided by an exemplary embodiment of the present application.

[0088] like Figure 4 As shown, P400 may include the following steps:

[0089] S410: Determine a plurality of target video frames from the video data, and perform splicing processing on the plurality of target video frames to determine a spliced ​​image.

[0090] S420: Perform semantic feature extraction on the spliced ​​image to determine video-level semantic features.

[0091] The target video frame may refer to at least a portion of the video frames that need to be spliced, that is, the aforementioned spliced ​​image may include each target video frame.

[0092] In some embodiments, considering the semantic extraction capability of the model, the image size of the spliced ​​image is often fixed, and in order to make the spliced ​​image have a good display effect, the data of the target video frame to be spliced ​​each time is limited. In order to obtain the video-level semantic features of the video data, the video data can be disassembled based on the number of video frames in the spliced ​​image, thereby obtaining multiple video-level semantic features of the video data to characterize the dynamic semantics of the entire video.

[0093] Therefore, the aforementioned S410 may further include:

[0094] S411. Determine at least one time range based on the video duration of the video data.

[0095] S412: Based on a target time range in at least one time range, determine a plurality of target video frames in the target time range from the video data.

[0096] S413: performing stitching processing on a plurality of target video frames to determine a stitched image in a target time range, so as to determine stitched images in each time range.

[0097] In the aforementioned S411, the length of the time range corresponds to the number of video frames in the spliced ​​image, that is, a video time period can be determined according to the video frame data allowed by the spliced ​​image, and the video data can be split based on the video time period to obtain multiple time ranges.

[0098] In some embodiments, the correspondence between the number of video frames and the video time period can be determined based on the video data acquisition situation and the video data sampling method. For example, when the video data is monitoring data of a vehicle, when the vehicle moves at a low speed, the sampling frequency can be lower (such as 5s), so that one video frame corresponds to a 5s video time period. When the vehicle moves at a high speed, the sampling frequency can be lower (such as 1s), so that one video frame corresponds to a 1s video time period.

[0099] Thus, the video time period corresponding to a spliced ​​image can be determined in combination with the number of video frames in the spliced ​​image, so as to divide the video duration based on the video data into at least one time range based on the video time period to cover the entire video data. Each time range can generate a corresponding video-level semantic feature.

[0100] In some embodiments, the aforementioned determination of the target video frame can also be adaptively adjusted according to the actual situation. For example, still taking the vehicle monitoring data as an example, when the road conditions are simple and the environmental changes are less (such as expressways), the aforementioned sampling frequency of the video data can be reduced. When the road conditions are complex and the environmental changes are faster (such as urban roads), the aforementioned sampling frequency of the video data can be increased.

[0101] In some embodiments, considering that in actual applications, data of other modalities (referred to as accompanying frame data) may be collected simultaneously with video data, the accompanying frame data may be mapped into the video frame data during stitching, and then stitching is performed to improve the representation capability of the stitched image.

[0102] That is, the accompanying frame data corresponding to the target video frame can be determined first. Then, the target video frame and the corresponding accompanying frame data are fused to determine the fused video frame, so as to determine the fused video frame of each target video frame. Finally, each fused video frame is spliced ​​to determine the target spliced ​​image. Among them, the accompanying frame data is the sensor data collected by the relevant sensor at the same time and the same scene of the target video frame.

[0103] Taking vehicle monitoring data as an example, the aforementioned accompanying data may include vehicle radar data (such as millimeter wave radar, ultrasonic radar, etc.), and the aforementioned accompanying frame data may refer to the radar data at the acquisition time of each video frame. When the aforementioned fusion is performed, the radar data may be mapped to the video frame in a visual form (such as a radar waveform) to determine the fused video frame.

[0104] It should be noted that the fusion of the accompanying data and the video data is not limited to the corresponding fusion of each frame, and the accompanying data can also be directly presented in the video data in a time sequence. For example, when the video data reflects the movement of a living object, the accompanying data can include the biological information of the living object (such as electrocardiogram information, fluorescence signal, etc.), and then a time series curve can be directly constructed based on the accompanying data in the form of a time series and fused into the spliced ​​image.

[0105] In the aforementioned S420, the splicing process of the target video frames can be implemented based on a conventional algorithm or a machine learning model. That is, the multiple target video frames can be spliced ​​based on the time sequence of the multiple target video frames to determine the target spliced ​​image. The multiple target video frames can also be spliced ​​by an image splicing model to determine the target spliced ​​image, wherein the image splicing model is a trained machine learning model.

[0106] In some embodiments, for a target object, a similar method as described above can also be used to determine the target object-related stitching images, thereby corresponding semantic features. Specifically, various video frames involving the same target object can be called for stitching, thereby determining the semantic features of the target object in each video frame as tracking semantic features. The tracking semantic features can be used as an object-level semantic feature of the target object.

[0107] To further illustrate this semantic feature, Figure 5 It is an exemplary flow chart of a tracking semantic feature determination method provided by an exemplary embodiment of the present application.

[0108] like Figure 5 As shown, P500 may include the following steps:

[0109] S510: Determine a plurality of target image data including a target object from the video data, and perform splicing processing on the plurality of target image data to determine a target spliced ​​image of the target object.

[0110] S520 , performing semantic feature extraction on the target spliced ​​image of the target object to obtain tracking semantic features of the target object.

[0111] Before executing the aforementioned P500, each target object of the video data may be tracked to determine the corresponding relationship between each identified target object in each video frame. For example, the same target object in each video frame may be determined based on the similarity of the image. For another example, a trajectory tracking algorithm may be used to determine the position change of the target object in each video frame based on the change of the target object bounding box, thereby determining the same target object in each video frame.

[0112] The target image data may include at least a portion of the video frame of the target object. In some embodiments, the region of the target object in the video frame may be cropped to determine the target image data.

[0113] In some embodiments, the target image data may be larger than the bounding box of the target object, thereby including environmental information around the target object. In some embodiments, the target image data may also be cropped based on a preset size to normalize the subsequent stitching process. In some embodiments, the target image data may also be the video frame itself, and in order to highlight the target object, a bounding box may be drawn around the target object to highlight the target object.

[0114] Therefore, the aforementioned splicing of target image data can include the temporal changes of the target object in the video data, and the semantic features thereof can reflect the changes of the target object in the video data. The splicing and feature extraction process of the target image data is similar to the aforementioned video-level semantic features, which will not be described in detail here.

[0115] In some embodiments, the whole-image semantic features can also be determined in a similar manner as described above to obtain the corresponding spliced ​​images. Specifically, considering that when the video data is acquired, there may be related video data acquired by other similar sensors, different video frames can be spliced ​​based on the actual spatial relationship to reflect the actual scene situation, and then semantic extraction can be performed to improve the representation ability of the current scene.

[0116] To further illustrate this semantic feature, Figure 6 It is an exemplary flow chart of a method for determining whole-image-level semantic features provided by an exemplary embodiment of the present application.

[0117] like Figure 6 As shown, P600 may include the following steps:

[0118] S610: Determine multiple associated video frames of a single video frame.

[0119] S620: Splice a single video frame and multiple associated video frames based on spatial relationships to determine a scene splicing image.

[0120] S630: Perform semantic feature extraction on the scene stitching image to determine the whole-image semantic features corresponding to a single video frame.

[0121] The associated video frame is the sensor data collected by other similar sensors in the same scene at the same time of a single video frame. The other similar sensors are generally other video acquisition devices (such as cameras) set on the same object.

[0122] For example, for a vehicle, especially an electric vehicle, multiple cameras are generally set around the vehicle to collect environmental information when the vehicle is driving. When archiving data, the video data collected by the front camera is generally recorded, and the associated video frame can reflect the video data collected by other cameras at the same time.

[0123] The spatial relationship reflects the settings of each sensor in the scene. Taking the aforementioned vehicle as an example, the spatial relationship can reflect the internal and external parameters of each camera. For example, the image acquisition range of the camera can be determined based on the internal and external parameters of each camera, and the relationship between the image ranges collected by each camera can be used as the spatial relationship.

[0124] In the aforementioned S620, the scene stitching image determined by stitching based on the aforementioned spatial relationship can reflect the stitching data of the actual scene. For example, for the aforementioned vehicle video data, stitching can be performed based on the image range to obtain an overall image surrounding the vehicle (similar to a panoramic image of the vehicle) as the scene stitching image.

[0125] Therefore, the aforementioned scene stitching image can better reflect the multi-angle data collected at the current moment in the scene than a single image frame, and thus can further reflect the overall situation of the scene. Then, semantic extraction can determine the semantic features of the overall scene.

[0126] Exemplary semantic feature association storage method

[0127] In some embodiments, considering that semantic features are used as feature data of video data for retrieval and matching, in order to speed up the retrieval process, classified storage can be performed based on the correspondence between semantic features and video frames / video data.

[0128] Specifically, the present application Figure 7 An exemplary flow chart of a semantic feature association storage method is also provided. Figure 7 As shown, process P700 may include:

[0129] S710: Divide the plurality of semantic features into first-category semantic features and second-category semantic features.

[0130] S720: Store the first type of semantics in an index area of ​​a database, and store the second type of semantics in a storage area of ​​the database.

[0131] S730: Associating the semantic feature with the storage location of the corresponding portion of video data.

[0132] The first type of semantic features and the second type of semantic features can be a classification of the various semantic features outputted by the video data. Among them, the first type of semantic features can correspond to multiple video frames of the video data, and the second type of semantic features only correspond to a single video frame of the video data. That is, when the first type of semantic features is called, the associated data is the video data of multiple video frames. Then, when the second type of semantic features is called, the associated data is a single video frame.

[0133] Based on the above embodiment, the above first type of semantic features may include the above video level semantic features and tracking semantic features of the target object, while the second type of semantic features may include whole image level semantic features and object level semantic features of the target object in each video frame.

[0134] In some embodiments, in order to facilitate the calling of appropriate target objects and scenes, representative semantic features may be determined based on clustering as the first type of semantic features.

[0135] For the representative semantic features of the target object, multiple object-level semantic features belonging to the same target object can be first determined from multiple semantic features. Multiple object-level semantic features of the same target object are clustered to determine the cluster center of the same target object. Representative semantic features are determined based on the cluster center, wherein the representative semantic features correspond to each video frame containing the same target object.

[0136] For the scene representative semantic features of the whole-image semantic features, the whole-image semantic features of each video frame can be clustered to determine at least one cluster center. Then, at least one scene representative semantic feature is determined based on the at least one cluster center, wherein the scene representative semantic features correspond to the cluster centers one by one and point to the video frames corresponding to each whole-image semantic feature in the cluster center.

[0137] In the process of determining the representative semantic features, clustering can be performed based on a common clustering algorithm to determine multiple clusters. Considering that the semantic features reflect the semantics of the corresponding image, each cluster has similar semantics, and its cluster center can be used as a representative to reflect the semantic features under the cluster.

[0138] To facilitate the calling of the corresponding semantic features during retrieval, the aforementioned first-category semantic features can be stored in the index area of ​​the database. Among them, the data in the index area is called first during retrieval. That is, the retrieval semantic features of the retrieval conditions are matched with each first-category semantic feature first. The storage area of ​​the database can be a regular data storage area of ​​the database, which can perform matching with the retrieval semantic features only after the first-category semantic features are called.

[0139] In some embodiments, further considering the efficiency of retrieval, the first type of semantic features can also be associated with the second type of semantic features. For example, the aforementioned video-level semantic features can be associated with each whole-image-level semantic feature and object-level semantic feature within the time range, and the aforementioned representative semantic features can be associated with each semantic feature of the corresponding cluster.

[0140] Based on the aforementioned index area and associated storage, when searching, the search semantic feature can be first matched with each first-category semantic feature of the index area, thereby determining the target first-category semantic feature from the first-category semantic feature, and then the search semantic feature can be matched with each second-category semantic feature corresponding to the target first semantic feature, thereby determining the search result. In this way, the amount of calculation for matching the second semantic features one by one can be reduced.

[0141] Exemplary Electronic Devices and Systems

[0142] Combination of the above Figures 1 to 7 , describes the method embodiment of the present application in detail, and the device embodiment of the present application is described in detail below. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so the part not described in detail can refer to the previous method embodiment.

[0143] The present application also provides a video data processing system, including a module for implementing the vehicle parameter polishing method provided by the present application. Figure 8 As shown, Figure 8 800 is a system module diagram of a data processing system provided by an exemplary embodiment of the present application. The video data processing system 800 provided by the present application may include a data acquisition module 810, a feature extraction module 820 and a data storage module 830.

[0144] The data acquisition module 810 may be used to acquire video data to be processed.

[0145] The feature extraction module 820 can be used to perform semantic feature extraction on video data to obtain multiple semantic features of the video data, wherein the multiple semantic features include object-level semantic features, whole-image-level semantic features and video-level semantic features, the object-level semantic features reflect the semantic features of the target objects included in the video data, the whole-image-level semantic features reflect the semantic features of a single video frame in the video data, and the video-level semantic features reflect the semantic features of a spliced ​​image obtained by splicing multiple video frames in the video data.

[0146] The data storage module 830 may be used to associate multiple semantic features with video data and store them in a database.

[0147] The present application also provides a vehicle, which includes the data processing system provided by the present application and can execute the video data processing method provided by the present application. The vehicle provided by the present application can be a fully automatic driving vehicle or a semi-automatic driving vehicle. Figure 1 The specific vehicle model provided in this application can be a car, a commercial vehicle, a truck, a bus, a tram, etc.

[0148] The present application also provides an electronic device, such as Fig. 9The electronic device 900 provided in the present application includes a memory 910, a processor 920, and an input / output interface 930. The memory 910, the processor 920, and the input / output interface 930 are connected through an internal connection path, the memory 910 is used to store instructions, and the processor 920 is used to execute the instructions stored in the memory 910 to control the input / output interface 930 to receive input data and information, and output data such as operation results.

[0149] It should be understood that in the embodiment of the present application, the processor 920 can adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs to implement the technical solution provided in the embodiment of the present application.

[0150] The memory 910 may include a read-only memory and a random access memory, and provides instructions and data to the processor 920. A portion of the processor 920 may also include a nonvolatile random access memory. For example, the processor 920 may also store information on the device type.

[0151] In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 920 or the instruction in the form of software. The control method of the vehicle disclosed in the embodiment of the present application can be directly embodied as a hardware processor for execution, or it can be executed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 910, and the processor 920 reads the information in the memory 910 and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0152] Based on the same technical concept, the present application also provides a computer program product, which includes a computer program / instruction. When the computer program / instruction processor is executed, the video data processing method as in any of the above embodiments is implemented.

[0153] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK).

[0154] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0155] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0156] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0157] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0158] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0159] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, a core network device, operation and maintenance management (operation administration and maintenance, OAM) or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that can be read by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)), etc. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0160] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video data processing method, characterized in that: The method comprises: Obtaining video data to be processed; Performing semantic feature extraction on the video data to obtain multiple semantic features of the video data, wherein the multiple semantic features include object-level semantic features, whole-image-level semantic features, and video-level semantic features, the object-level semantic features reflect the semantic features of a target object included in the video data, the whole-image-level semantic features reflect the semantic features of a single video frame in the video data, and the video-level semantic features reflect the semantic features of a spliced ​​image obtained by splicing multiple video frames in the video data; The plurality of semantic features are associated with the video data and stored in a database.

2. The method according to claim 1, characterized in that The performing semantic feature extraction on the video data to obtain a plurality of semantic features of the video data includes: Determine a plurality of target video frames from the video data, and perform splicing processing on the plurality of target video frames to determine a spliced ​​image; Semantic feature extraction is performed on the spliced ​​image to determine the video-level semantic feature.

3. The method according to claim 2, characterized in that The step of performing stitching processing on the multiple target video frames to determine a target stitched image includes: Determine accompanying frame data corresponding to the target video frame, wherein the accompanying frame data is sensor data collected by a related sensor at the same time and the same scene as the target video frame; Performing fusion processing on the target video frame and the corresponding accompanying frame data to determine a fused video frame, so as to determine a fused video frame of each target video frame; Each fused video frame is stitched together to determine the target stitched image.

4. The method according to claim 2, characterized in that: The step of performing stitching processing on the multiple target video frames to determine a target stitched image includes: Performing stitching processing on the multiple target video frames through an image stitching model to determine the target stitching image, wherein the image stitching model is a trained machine learning model; or The target video frames are spliced ​​based on the time sequence of the target video frames to determine the target spliced ​​image.

5. The method according to claim 2, characterized in that: The step of determining a plurality of target video frames from the video data and performing splicing processing on the plurality of target video frames to determine a target spliced ​​image includes: Determining at least one time range based on the video duration of the video data, wherein the length of the time range corresponds to the number of video frames in the stitched image; Based on a target time range in the at least one time range, determining a plurality of target video frames in the target time range from the video data; The plurality of target video frames are spliced ​​to determine a spliced ​​image of the target time range, thereby determining spliced ​​images of each time range.

6. The method according to claim 1, characterized in that The object-level semantic features include tracking semantic features of the target object, and the semantic feature extraction is performed on the video data to obtain a plurality of semantic features of the video data, including: Determining a plurality of target image data including the target object from the video data, and performing splicing processing on the plurality of target image data to determine a target spliced ​​image of the target object; Semantic feature extraction is performed on the target spliced ​​image of the target object to obtain tracking semantic features of the target object, wherein the tracking semantic features reflect changes of the target object in the video data.

7. The method according to claim 1, characterized in that The whole-image-level semantic features include scene semantic features corresponding to the single video frame, and the performing semantic feature extraction on the video data to determine a plurality of semantic features of the video data includes: Determine a plurality of associated video frames of the single video frame, wherein the associated video frames are sensor data collected by other similar sensors at the same time and in the same scene as the single video frame; Performing stitching processing on the single video frame and the multiple associated video frames based on a spatial relationship to determine a scene stitching image, wherein the spatial relationship reflects the setting of each sensor in the scene; Semantic feature extraction is performed on the scene stitching image to determine the whole-image semantic features corresponding to the single video frame.

8. The method according to any one of claims 1 to 7, characterized in that The associating the plurality of semantic features with the video data and storing them in a database comprises: Dividing the multiple semantic features into a first type of semantic features and a second type of semantic features, wherein the first type of semantic features corresponds to multiple video frames of the video data, and the second type of semantic features corresponds to a single video frame of the video data; Storing the first type of semantics in an index area of ​​the database, and storing the second type of semantics in a storage area of ​​the database, wherein the data in the index area is first called during retrieval; The semantic feature is associated with a storage location of the corresponding portion of video data.

9. The method according to claim 8, characterized in that The first type of semantic features includes representative semantic features of the target object, and the method further includes: Determining a plurality of object-level semantic features belonging to the same target object from the plurality of semantic features; Performing clustering processing on multiple object-level semantic features of the same target object to determine a cluster center of the same target object; The representative semantic feature is determined based on the cluster center, wherein the representative semantic feature corresponds to each video frame containing the same target object.

10. The method according to claim 8, characterized in that The first type of semantic features includes scene representative semantic features of the whole-image level semantic features, and the method further includes: Clustering the whole-image semantic features of each video frame to determine at least one cluster center; At least one scene representative semantic feature is determined based on the at least one cluster center, wherein the scene representative semantic feature corresponds to the cluster center one by one and points to the video frame corresponding to each whole-image semantic feature in the cluster center.

11. The method according to any one of claims 1 to 7, characterized in that The method further comprises: Acquire a search condition, wherein the search condition includes at least one of an image, a video, and a text; Performing feature extraction on the search condition to determine the search semantic features of the search condition; Determining a target semantic feature matching the retrieval semantic feature from the database based on a similarity with the retrieval semantic feature; A single video frame corresponding to the target semantic feature and / or video data including multiple video frames are output as a retrieval result of the retrieval condition.

12. The method according to claim 11, characterized in that The method further comprises: The tag information is generated based on the search condition, and the search result is indexed based on the tag information.

13. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the video data processing method according to any one of claims 1 to 12.

14. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction processor is executed, the video data processing method according to any one of claims 1 to 12 is implemented.

15. A vehicle, characterized in that: The vehicle comprises: On-board monitoring system; at least one processor; and a memory communicatively coupled to the at least one processor; Wherein, the memory stores instructions that can be executed by the at least one processor. When the instructions are executed by the at least one processor, the at least one processor is used to obtain video data through the vehicle-mounted monitoring system and execute the video data processing method described in any one of claims 1-12 based on the video data.

Citation Information

Cited By

  • Visual data management method and device in vehicle sentry mode, medium and product

    CN121350596A

  • Visual data management method, device, medium, and product for vehicle sentry mode

    CN121350596B