Interactive video data annotation device and annotation method thereof

Abstract

The present invention discloses an interactive video data annotation device and an annotation method thereof, and belongs to the technical field of data annotation. The present invention solves the problem of low accuracy and efficiency of the existing annotation method, obtains the target video to be annotated through an intelligent camera device, and connects the target data to be annotated into the annotation training model to automatically annotate the video, thereby obtaining the automatically annotated content; manually adds, deletes, modifies and checks the automatically annotated content, and uses the annotation review module to check the annotated content; if the checked content fails, the target video that fails is returned to manual for secondary correction; if the checked content passes, the target video is directly sent to the annotation generation module to generate the final video, and the original target video and the target video with data annotation content are displayed through the video display module; by combining automatic annotation with manual annotation, the accuracy of video annotation content and the efficiency of annotation are improved, and the investment cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of data annotation, and in particular to an interactive video data annotation device and an annotation method thereof. Background Art

[0002] Video data annotation is the process of using machines to automatically generate natural language text to describe video content. It plays a very important bridging role between vision and text. The purpose of video data annotation is to annotate the location, shape, action, color and other related features of the moving targets in the scene; provide a large amount of data for tracking algorithms to use, so as to detect, track, identify the moving targets in the scene, and further analyze behavior and detect events.

[0003] However, in actual application, most of the existing video data labeling methods are manually labeling frame by frame or automatically labeling the target objects; manual labeling is not only slow in efficiency, but also has high error rate and investment cost; although automatic labeling can meet the labeling speed, the accuracy is often limited by the model itself, thereby reducing the accuracy of video labeling.

[0004] Therefore, the existing needs are not met, and we propose an interactive video data annotation device and an annotation method thereof. Summary of the invention

[0005] The object of the present invention is to provide an interactive video data annotation device and an annotation method thereof, which obtains a target video to be annotated through an intelligent camera device, and connects the target data to be annotated to an annotation training model to automatically annotate the video, thereby obtaining automatically annotated content; manually adds, deletes, modifies and checks the automatically annotated content, and uses an annotation review module to check the annotated content; if the checked content fails, the unsuccessful target video is returned to manual for secondary correction; if the checked content passes, the target video is directly sent to an annotation generation module to generate a final video, and the original target video and the target video with data annotation content are displayed through a video display module; by combining automatic annotation with manual annotation, the accuracy of video annotation content and the efficiency of annotation are improved, the investment cost is reduced, and the problems raised in the above-mentioned background technology are solved.

[0006] To achieve the above object, the present invention provides the following technical solution: an interactive video data annotation device, comprising:

[0007] The video acquisition module acquires the target video to be annotated based on the smart camera device, and uploads the target video to be annotated to the video annotation unit for annotation;

[0008] A model training unit, used to create a video feature recognition model and pre-train the created video feature recognition model to obtain a data annotation training model;

[0009] The video annotation unit is used to connect the target video to be annotated to the trained model for automatic annotation, and to review the annotated target video;

[0010] The annotation generation module is based on the content annotated by the video annotation unit and is used to combine the annotated content with the target video, thereby generating a target video with data annotated content;

[0011] The video display module is used to receive and display the original target video acquired by the video acquisition module, as well as the target video with data annotation content.

[0012] Furthermore, the video acquisition module includes:

[0013] The video storage module is used to save the target video to be labeled acquired by the smart camera device in advance as the original target video of the target video to be labeled;

[0014] The video access module is used to configure the transmission address of the target video to be annotated so as to access the real-time target video data to be annotated;

[0015] The video transmission module is used to upload the stored original target video to the video display module for display.

[0016] Furthermore, the video storage module includes:

[0017] The video data extraction module is used to extract the target video to be annotated acquired by the smart camera device within a unit time; wherein the value range of the unit time is 8min-10min;

[0018] A data volume comparison module, used to compare the target video to be labeled acquired by the smart camera device within the unit time with a first data volume threshold and a second data volume threshold to obtain a comparison result;

[0019] A data set division module, configured to classify the target videos to be labeled acquired by the smart camera device within a unit time into a first data set, a second data set and a third data set according to the comparison result;

[0020] The first data set is a video data type whose data volume of the target video to be labeled acquired by the smart camera device within a unit time does not exceed a first data volume threshold;

[0021] The second data set is a type of video data whose data volume of the target video to be labeled acquired by the smart camera device within a unit time exceeds the first data volume threshold but does not exceed the second data threshold;

[0022] The third data set is the type of video data whose data amount of the target video to be labeled acquired by the smart camera device per unit time exceeds the second data threshold;

[0023] a storage space setting module, configured to set a cache space of a data cache area corresponding to the first data set, the second data set and the third data set according to the first data set, the second data set and the third data set;

[0024] The cache space of the data cache area corresponding to the first data set, the second data set and the third data set is obtained by the following formula:

[0025]

[0026] Among them, C k1 represents the cache space of the data cache area corresponding to the first data set; C0 represents the storage capacity corresponding to the preset reference storage space; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; n indicates the number of unit times that have passed; C 01i represents the amount of data belonging to the video data type corresponding to the first data set in the video data acquired in the i-th unit time; C d1i represents the amount of video data acquired per unit time; C 01ci represents the amount of data belonging to the video data type corresponding to the first data set in the video data actually uploaded in the i-th unit time; C d2i Indicates the amount of video data actually uploaded per i-th unit time;

[0027]

[0028] Among them, C k2 represents the cache space of the data cache area corresponding to the second data set; C0 represents the storage capacity corresponding to the preset reference storage space; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; n indicates the number of unit times that have passed; C 02i represents the amount of data belonging to the video data type corresponding to the second data set in the video data acquired in the i-th unit time; C d1irepresents the amount of video data acquired per unit time; C 02ci represents the amount of data belonging to the video data type corresponding to the second data set in the video data actually uploaded in the i-th unit time; C d2i Indicates the amount of video data actually uploaded per i-th unit time;

[0029]

[0030] Among them, C k3 represents the cache space of the data cache area corresponding to the third data set; C0 represents the storage capacity corresponding to the preset reference storage space; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; n indicates the number of unit times that have passed; C 03i represents the amount of data belonging to the video data type corresponding to the third data set in the video data acquired in the i-th unit time; C d1i represents the amount of video data acquired per unit time; C 03ci represents the amount of data belonging to the video data type corresponding to the third data set in the video data actually uploaded in the i-th unit time; C d2i Indicates the amount of video data actually uploaded per i-th unit time.

[0031] Furthermore, the video transmission module includes:

[0032] A first real-time monitoring module, used for monitoring in real time the space occupancy of the cache space of the data cache area corresponding to the first data set;

[0033] The first data transmission rate setting module is used to set a first data transmission rate per unit time for the data cache area corresponding to the first data set according to the space occupation of the cache space of the data cache area corresponding to the first data set, wherein the first data transmission rate is obtained by the following formula:

[0034]

[0035] Among them, C s1 represents the first data transmission volume; C s0 Indicates the preset data transmission volume; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; C z1 represents the space occupancy of the cache space of the data cache area corresponding to the first data set; C k1represents the cache space of the data cache area corresponding to the first data set; e represents a constant;

[0036] A second real-time monitoring module, used for monitoring in real time the space occupancy of the cache space of the data cache area corresponding to the second data set;

[0037] The second data transmission rate setting module is used to set a second data transmission rate per unit time for the data cache area corresponding to the second data set according to the space occupation of the cache space of the data cache area corresponding to the second data set, wherein the second data transmission rate is obtained by the following formula:

[0038]

[0039] Among them, C s2 represents the second data transmission volume; C s0 Indicates the preset data transmission volume; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; C z2 represents the space occupancy of the cache space of the data cache area corresponding to the second data set; C k2 Represents a cache space of a data cache area corresponding to a second data set;

[0040] A third real-time monitoring module, used for monitoring in real time the space occupancy of the cache space of the data cache area corresponding to the third data set;

[0041] The third data transmission volume setting module is used to set a third data transmission volume per unit time for the data cache area corresponding to the third data set according to the space occupation of the cache space of the data cache area corresponding to the third data set, wherein the third data transmission volume is obtained by the following formula:

[0042]

[0043] Among them, C s3 Indicates the third data transmission volume; C s0 Indicates the preset data transmission volume; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; C z3 represents the space occupancy of the cache space of the data cache area corresponding to the third data set; C k3 Represents the cache space of the data cache area corresponding to the third data set.

[0044] Furthermore, the model training unit includes:

[0045] A model creation module is used to create a video feature recognition model to serve as a training model for the target video to be labeled;

[0046] A sample acquisition module is used to acquire video data samples and annotate the annotated points in the video data samples based on a manual annotation method, thereby obtaining annotated data samples;

[0047] The model training module performs data annotation training on the video feature recognition model based on the labeled data samples, thereby obtaining a training model of the target video to be labeled.

[0048] Furthermore, the video annotation unit includes:

[0049] The model loading module is used to preload the trained data annotation training model to ensure the stability of access to the target video to be annotated;

[0050] The automatic annotation module, based on the data annotation training model, automatically annotates the target video to be annotated by connecting it to the trained data annotation training model, and saves the automatically annotated content;

[0051] The manual annotation module is used to add, delete, modify and check the automatically annotated content in the target video to be annotated, and save the corrected annotated content for a second time;

[0052] The annotation review module is used to check the automatically annotated content with the manually annotated content to ensure the accuracy of the data annotation in the target video.

[0053] Furthermore, the model training module performs data annotation training on the video feature recognition model by annotating data samples, specifically:

[0054] Manually select the annotation points in the video data sample. For data that requires complex annotation, annotation rules need to be formulated in advance, including: specifying the classification of objects in the image and the location of the bounding box;

[0055] Use annotation tools to annotate video data samples and verify the quality of the annotated data;

[0056] Organize the annotation results into a format that can be used by the machine learning model, including: converting the image annotation results into bounding box coordinates and classification labels;

[0057] The labeled data is connected to the video feature recognition model for training and learning, thereby obtaining a training model for the target video to be labeled.

[0058] Furthermore, the annotation review module checks the automatically annotated content with the manually annotated content, specifically:

[0059] The annotation review module is used to check the automatic and manual annotation contents in the target video to be annotated, including the classification of objects in the image, the position of the bounding box, and whether the annotation text is accurate. If the verification is successful, the annotated target video is sent to the annotation generation module to generate the classification of objects in the image and the position of the bounding box. If the verification is unsuccessful, the target video that fails the verification is returned to the manual annotation module for secondary correction until the target video passes the verification.

[0060] Furthermore, the annotation review module includes:

[0061] The data storage module is used to store the video data that has passed the review into a designated database.

[0062] The present invention also provides another technical solution: a labeling method of an interactive video data labeling device, comprising the following steps:

[0063] S1. Create a video feature recognition model through a model creation module, and train the model using labeled data samples, thereby obtaining a training model of the target video to be labeled;

[0064] S2, obtaining a video of a target to be annotated through a smart camera device, and connecting the target data to be annotated to an annotation training model for automatic annotation, thereby obtaining automatic annotation content;

[0065] S3, adding, deleting, modifying and checking the automatically annotated content through the manual annotation module, and checking the automatically annotated and manually annotated content through the annotation review module;

[0066] If the verification fails in S4 and S3, the target video that fails the verification is returned to the manual annotation module for secondary correction until the target video passes the verification; if the verification passes, the target video is directly sent to the annotation generation module to generate the final video;

[0067] S5. Display the original target video and the target video with data annotation content through the video display module.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] The present invention creates a video feature recognition model and trains the model with labeled data samples, thereby obtaining a video training model; obtains a target video to be labeled through an intelligent camera device, and connects the target data to be labeled into a labeling training model for automatic labeling, thereby obtaining automatic labeling content; manually adds, deletes, modifies and checks the automatically labeled content, and uses a labeling review module to check the labeled content; if the checked content fails, the failed target video is returned to manual for secondary correction; if the checked content passes, the target video is directly sent to a labeling generation module to generate a final video, and the original target video and the target video with data labeling content are displayed through a video display module; through the above method, the automatic labeling method is combined with the manual labeling method, the error of the video labeling content is reduced, and the video labeling content has higher accuracy; at the same time, the efficiency of video labeling is improved, and the investment cost is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a composition diagram of the interactive video data annotation device of the present invention;

[0071] Figure 2 This is a flow chart of the interactive video data annotation method of the present invention. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] In order to solve the technical problem that most existing video data annotation methods use manual frame-by-frame annotation or automatic object annotation, manual annotation is not only slow, but also has high error rate and investment cost; although automatic annotation can meet the annotation speed, the accuracy is often limited by the model itself, thus reducing the accuracy of video annotation. Figure 1-Figure 2 , this embodiment provides the following technical solutions:

[0074] An interactive video data annotation device, comprising:

[0075] The video acquisition module obtains the target video to be labeled based on the smart camera device, and uploads the target video to be labeled to the video labeling unit for labeling; specifically, the video to be labeled is shot by the smart camera device, which is used as the target video to be labeled, and the captured target video to be labeled is uploaded to the video labeling unit for labeling through the transmission function of the smart camera device; in actual application, the optional models of smart camera devices are: Maumer-CX series industrial cameras.

[0076] The model training unit is used to create a video feature recognition model and pre-train the created video feature recognition model to obtain a data labeling training model; specifically, by creating a video feature recognition model and using previously labeled data as samples for training, a data labeling training model is obtained to facilitate automatic labeling of the target video to be labeled.

[0077] The video annotation unit is used to connect the target video to be annotated to the trained model for automatic annotation, and to review the annotated target video; specifically, by importing the target video to be annotated into the data annotation training model, and thereby automatically annotating the target annotation content in the target video to be annotated; manually correcting the automatically annotated content, and then finally reviewing the corrected content, thereby ensuring the accuracy of the annotated content in the target video to be annotated, and thereby reducing the error rate of the annotated content in the target video to be annotated.

[0078] The annotation generation module is based on the content annotated by the video annotation unit and is used to combine the annotation content with the target video, thereby generating a target video with data annotation content; specifically, the annotation generation module saves the verified annotation content and the target video, thereby annotating the annotation content in the target video to form the annotated target video, and the annotated target video is uploaded to the video display module for display.

[0079] The video display module is used to receive and display the original target video obtained by the video acquisition module, as well as the target video with data annotation content; specifically, the original unlabeled target video and the target video with annotation content are displayed separately through the video display module for users to view, compare and retrieve related content; in actual application, the video display module is specifically: the user's smart phone, computer and its display screen.

[0080] The working principle of the above content is as follows: by creating a video feature recognition model and training the model with labeled data samples, a video training model is obtained; the target video to be labeled is obtained through an intelligent camera device, and the target data to be labeled is connected to the labeling training model for automatic labeling, thereby obtaining automatic labeling content; the automatically labeled content is manually added, deleted, modified and checked, and the labeled content is verified; if the verified content fails, the target video that fails is returned to manual for secondary correction; if the verified content passes, the target video is directly sent to the labeling generation module to generate the final video, and the original target video and the target video with data labeling content are displayed through the video display module; through the above method, the automatic labeling method is combined with the manual labeling method, the error of the video labeling content is reduced, and the video labeling content has a higher accuracy; at the same time, the efficiency of video labeling is improved and the investment cost is reduced.

[0081] The video acquisition module includes:

[0082] The video storage module is used to save the target video to be labeled obtained by the smart camera device in advance as the original target video of the target video to be labeled; specifically, the captured target video is saved through the video storage module as a backup of the target video, mainly used to prevent the target video from being lost or accidentally deleted during the labeling process or after the labeling is completed; at the same time, the saved original video needs to be transmitted to the video display module and displayed side by side with the labeled target video for users to view, compare and retrieve related content.

[0083] The video access module is used to configure the transmission address of the target video to be labeled so as to access the real-time target video data to be labeled. Specifically, the video access module configures the smart camera device and the external receiving device with addresses that can be used for transmission. Based on the address, the external receiving device can receive the target video data to be labeled taken by the smart camera device in real time, thereby improving the timeliness of the transmission of the smart camera device and further improving the efficiency of data labeling.

[0084] The video transmission module is used to upload the stored original target video to the video display module for display; specifically, the video transmission module provides a wireless communication path between the smart camera device and the video display module, and based on the communication path, the smart camera device transmits the captured target video to the video display module, and displays it side by side with the marked target video.

[0085] Specifically, the video storage module includes:

[0086] The video data extraction module is used to extract the target video to be annotated acquired by the smart camera device within a unit time; wherein the value range of the unit time is 8min-10min;

[0087] A data volume comparison module, used to compare the target video to be labeled acquired by the smart camera device within the unit time with a first data volume threshold and a second data volume threshold to obtain a comparison result;

[0088] A data set division module, configured to classify the target videos to be labeled acquired by the smart camera device within a unit time into a first data set, a second data set and a third data set according to the comparison result;

[0089] The first data set is a video data type whose data volume of the target video to be labeled acquired by the smart camera device within a unit time does not exceed a first data volume threshold;

[0090] The second data set is a type of video data whose data volume of the target video to be labeled acquired by the smart camera device within a unit time exceeds the first data volume threshold but does not exceed the second data threshold;

[0091] The third data set is the type of video data whose data amount of the target video to be labeled acquired by the smart camera device per unit time exceeds the second data threshold;

[0092] a storage space setting module, configured to set a cache space of a data cache area corresponding to the first data set, the second data set and the third data set according to the first data set, the second data set and the third data set;

[0093] The cache space of the data cache area corresponding to the first data set, the second data set and the third data set is obtained by the following formula:

[0094]

[0095] Among them, C k1 represents the cache space of the data cache area corresponding to the first data set; C0 represents the storage capacity corresponding to the preset reference storage space; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; n indicates the number of unit times that have passed; C 01i represents the amount of data belonging to the video data type corresponding to the first data set in the video data acquired in the i-th unit time; C d1i represents the amount of video data acquired per unit time; C 01ci represents the amount of data belonging to the video data type corresponding to the first data set in the video data actually uploaded in the i-th unit time; C d2i Indicates the amount of video data actually uploaded per i-th unit time;

[0096]

[0097] Among them, C k2 represents the cache space of the data cache area corresponding to the second data set; C0 represents the storage capacity corresponding to the preset reference storage space; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; n indicates the number of unit times that have passed; C 02i represents the amount of data belonging to the video data type corresponding to the second data set in the video data acquired in the i-th unit time; C d1i represents the amount of video data acquired per unit time; C 02ci represents the amount of data belonging to the video data type corresponding to the second data set in the video data actually uploaded in the i-th unit time; C d2i Indicates the amount of video data actually uploaded per i-th unit time;

[0098]

[0099] Among them, C k3 represents the cache space of the data cache area corresponding to the third data set; C0 represents the storage capacity corresponding to the preset reference storage space; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; n indicates the number of unit times that have passed; C 03i represents the amount of data belonging to the video data type corresponding to the third data set in the video data acquired in the i-th unit time; C d1i represents the amount of video data acquired per unit time; C 03ci represents the amount of data belonging to the video data type corresponding to the third data set in the video data actually uploaded in the i-th unit time; C d2i Indicates the amount of video data actually uploaded per i-th unit time.

[0100] The technical effects of the above technical solution are as follows: Dynamic storage management: This solution can flexibly manage the storage requirements of video data by adaptively adjusting the cache space of the data cache area according to the video data type with different data volumes. This helps to avoid unnecessary storage waste and make full use of available storage resources.

[0101] Optimize storage costs: By dynamically adjusting storage space, storage costs can be minimized without sacrificing data quality. This is particularly useful for application scenarios that require long-term storage of large amounts of video data.

[0102] Data type classification: Video data is divided into different data sets so that the data can be managed and retrieved according to their type and importance. This helps to organize and use video data more efficiently.

[0103] Real-time adaptability: By monitoring the amount of data per unit time and adjusting the storage space in real time, the system can adapt to changes in data volume in different situations and ensure the effectiveness and availability of data storage.

[0104] Data transmission control: By considering the receiving capacity per unit time (Cp2) of the video display module and the average amount of data acquired per unit time (Cp1), control can be performed on data transmission to prevent data transmission overload.

[0105] In general, this technical solution helps smart camera devices manage video data more intelligently to improve the efficiency and economy of data storage while ensuring the integrity and availability of data. This has important practical significance for applications that require large-scale processing and storage of video data, such as surveillance systems and video analysis.

[0106] Specifically, the video transmission module includes:

[0107] A first real-time monitoring module, used for monitoring in real time the space occupancy of the cache space of the data cache area corresponding to the first data set;

[0108] The first data transmission rate setting module is used to set a first data transmission rate per unit time for the data cache area corresponding to the first data set according to the space occupation of the cache space of the data cache area corresponding to the first data set, wherein the first data transmission rate is obtained by the following formula:

[0109]

[0110] Among them, C s1 represents the first data transmission volume; C s0 Indicates the preset data transmission volume; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; C z1 represents the space occupancy of the cache space of the data cache area corresponding to the first data set; C k1 represents the cache space of the data cache area corresponding to the first data set; e represents a constant;

[0111] A second real-time monitoring module, used for monitoring in real time the space occupancy of the cache space of the data cache area corresponding to the second data set;

[0112] The second data transmission rate setting module is used to set a second data transmission rate per unit time for the data cache area corresponding to the second data set according to the space occupation of the cache space of the data cache area corresponding to the second data set, wherein the second data transmission rate is obtained by the following formula:

[0113]

[0114] Among them, C s2 represents the second data transmission volume; C s0 Indicates the preset data transmission volume; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; C z2 represents the space occupancy of the cache space of the data cache area corresponding to the second data set; C k2 Represents a cache space of a data cache area corresponding to a second data set;

[0115] A third real-time monitoring module, used for monitoring in real time the space occupancy of the cache space of the data cache area corresponding to the third data set;

[0116] The third data transmission volume setting module is used to set a third data transmission volume per unit time for the data cache area corresponding to the third data set according to the space occupation of the cache space of the data cache area corresponding to the third data set, wherein the third data transmission volume is obtained by the following formula:

[0117]

[0118] Among them, C s3 Indicates the third data transmission volume; C s0 Indicates the preset data transmission volume; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; C z3 represents the space occupancy of the cache space of the data cache area corresponding to the third data set; C k3 Represents the cache space of the data cache area corresponding to the third data set.

[0119] The technical effects of the above technical solution are: Real-time monitoring and management: By real-time monitoring of the space occupied by the data cache area corresponding to different data sets, the system can dynamically set the data transmission volume according to the actual storage requirements. This helps to avoid overflow or waste of the data cache area and ensure timely data transmission.

[0120] Adaptive data transmission: By setting the data transmission volume per unit time for different data sets according to the space occupied by the data cache area, the system can adaptively control the data transmission rate of different data sets. This ensures that the transmission rate can be appropriately adjusted for data sets of different importance.

[0121] Data transmission smoothness: By using a formula to calculate the amount of data transmitted, the system can smoothly adjust the transmission rate according to the actual situation. This helps prevent excessive jitter or instability in data transmission, thereby maintaining the stability of data transmission.

[0122] Minimum data volume guarantee: Taking into account the minimum data receiving capability (Cp2) of the video display module, the system can ensure that the amount of data transmitted to the video display module is not less than a certain minimum requirement. This helps to avoid video display freezes or data loss.

[0123] Save bandwidth and resources: By adjusting the data transmission rate according to actual needs, the system can effectively save network bandwidth and storage resources to improve overall performance and efficiency.

[0124] In general, this technical solution helps to achieve effective management and transmission of video data in smart camera devices to ensure that video data can be transmitted to the display module on demand while making full use of available resources and providing a smooth data transmission experience. This is of great significance for application scenarios that require real-time video monitoring and display.

[0125] The model training unit includes:

[0126] The model creation module is used to create a video feature recognition model to serve as a training model for the target video to be labeled.

[0127] The sample acquisition module is used to obtain video data samples and annotate the annotation points in the video data samples based on manual annotation, thereby obtaining annotated data samples; specifically, previous videos and annotation contents are obtained through intelligent camera devices, and contents with clear annotation contents and high annotation accuracy are selected from previous videos as training samples, thereby providing training data for the video feature recognition model.

[0128] The model training module performs data annotation training on the video feature recognition model based on the labeled data samples, thereby obtaining a training model of the target video to be labeled; specifically, the training model of the target video to be labeled is obtained by importing the above training samples into the created video feature recognition model for training, so as to facilitate the target video to be labeled to be connected to the training model for automatic labeling.

[0129] The model training module performs data annotation training on the video feature recognition model by annotating data samples, specifically:

[0130] By manually selecting the annotation points in the video data samples, for data that requires complex annotation, annotation rules need to be formulated in advance, including: specifying the classification of objects in the image and the location of the bounding box; using annotation tools to annotate the video data samples and verifying the quality of the annotated data; organizing the annotation results into a format that can be used by the machine learning model, including: converting the image annotation results into the coordinates of the bounding box and the classification label; connecting the annotated data to the video feature recognition model for training and learning, thereby obtaining a training model for the target video to be annotated.

[0131] The video annotation unit includes:

[0132] The model loading module is used to pre-load the trained data annotation training model to ensure the stability of access to the target video to be labeled; specifically, the data annotation training model is pre-loaded offline through the model loading module, and then the target video to be labeled is imported into the data annotation training model for labeling processing, thereby improving the stability of transmission between the target video to be labeled and the data annotation training model, and then ensuring the integrity of the target video to be labeled.

[0133] The automatic labeling module, based on the data labeling training model, connects the target video to be labeled into the trained data labeling training model for automatic labeling, and saves the automatic labeling content; specifically, the target video to be labeled imported into the data labeling training model is trained by the automatic labeling module, thereby automatically labeling the target labeling points in the target video to be labeled; in one embodiment, such as: shooting the target video to be labeled by an intelligent camera device, the target video can specifically be: a video of the surrounding environment of the community; the labeling requirements are: labeling all parking spaces in the surrounding environment of the community, and re-labeling parking spaces with charging piles; by pre-training the data labeling training model with the same type of labeled data samples, and importing the above target video into the data labeling training model for automatic labeling, the parking spaces around the community in the target video are marked, and the parking spaces with charging piles are separately labeled, so as to understand the number of parking spaces around the community and the number of parking spaces where new energy vehicles can be parked.

[0134] The manual annotation module is used to add, delete, modify and check the automatically annotated content in the target video to be annotated, and to save the corrected annotation content for a second time; in accordance with the above embodiment, for example, the parking spaces around the community and the parking spaces with charging piles installed in the target video are distinguished and marked through the data annotation training model, and the parking spaces with charging piles installed are annotated with text using the manual annotation module to improve the visibility of the two types of data annotations in the target video.

[0135] The annotation review module is used to check the automatically annotated content with the manually annotated content to ensure the accuracy of the data annotation in the target video; following the above embodiment, for example, after the parking spaces with charging piles are annotated with text by the manual annotation module, the annotation review module is used to verify the manually annotated text content, and the accuracy of the automatically annotated parking space location and whether the charging piles are installed is verified, thereby improving the accuracy of the data annotation; during the verification process, if it is found that one of the parking spaces is equipped with a charging pile, and the automatically annotated content and the manually annotated text content both mark the parking space as an ordinary parking space, that is, a parking space without a charging pile installed, indicating that the content is incorrectly annotated; the content is returned to the manual for secondary correction and saved until all the annotation points are verified, and then the verified video is uploaded to the annotation generation module to generate the annotated video, and the original target video and the target video with data annotation content are displayed through the video display module for user viewing.

[0136] The labeling and review module includes: a data storage module, which is used to store the video data that has passed the review into a designated database for subsequent review at any time; at the same time, it can be used as a data sample to train the data labeling training model in the later stage.

[0137] The annotation review module checks the automatically annotated content with the manually annotated content, specifically:

[0138] The annotation review module is used to check the automatic and manual annotation contents in the target video to be annotated, including the classification of objects in the image, the position of the bounding box, and whether the annotation text is accurate. If the verification is successful, the annotated target video is sent to the annotation generation module to generate the classification of objects in the image and the position of the bounding box. If the verification is unsuccessful, the target video that fails the verification is returned to the manual annotation module for secondary correction until the target video passes the verification.

[0139] In order to better demonstrate the annotation process of the interactive video data annotation device, this embodiment proposes an annotation method of the interactive video data annotation device, and the annotation method includes the following steps:

[0140] S1. Create a video feature recognition model through a model creation module, and train the model using labeled data samples, thereby obtaining a training model of the target video to be labeled;

[0141] S2, obtaining a video of a target to be annotated through a smart camera device, and connecting the target data to be annotated to an annotation training model for automatic annotation, thereby obtaining automatic annotation content;

[0142] S3, adding, deleting, modifying and checking the automatically annotated content through the manual annotation module, and checking the automatically annotated and manually annotated content through the annotation review module;

[0143] If the verification fails in S4 and S3, the target video that fails the verification is returned to the manual annotation module for secondary correction until the target video passes the verification; if the verification passes, the target video is directly sent to the annotation generation module to generate the final video;

[0144] S5. Display the original target video and the target video with data annotation content through the video display module.

[0145] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0146] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An interactive video data annotation device, characterized in that: include: The video acquisition module acquires the target video to be annotated based on the smart camera device, and uploads the target video to be annotated to the video annotation unit for annotation; The video acquisition module comprises: The video storage module is used to save the target video to be labeled acquired by the smart camera device in advance as the original target video of the target video to be labeled; the video storage module includes: The video data extraction module is used to extract the target video to be annotated acquired by the smart camera device within a unit time; wherein the value range of the unit time is 8min-10min; A data volume comparison module, used to compare the target video to be labeled acquired by the smart camera device within the unit time with a first data volume threshold and a second data volume threshold to obtain a comparison result; A data set division module is used to classify the target videos to be labeled acquired by the smart camera device in a unit time into a first data set, a second data set and a third data set according to the comparison result; wherein the first data set is the type of video data whose data volume of the target videos to be labeled acquired by the smart camera device in a unit time does not exceed a first data volume threshold; the second data set is the type of video data whose data volume of the target videos to be labeled acquired by the smart camera device in a unit time exceeds the first data volume threshold, but does not exceed the second data threshold; the third data set is the type of video data whose data volume of the target videos to be labeled acquired by the smart camera device in a unit time exceeds the second data threshold; A storage space setting module is used to set the cache space of the data cache area corresponding to the first data set, the second data set and the third data set according to the first data set, the second data set and the third data set; wherein the cache space of the data cache area corresponding to the first data set, the second data set and the third data set is obtained by the following formula: Among them, C k1 represents the cache space of the data cache area corresponding to the first data set; C0 represents the storage capacity corresponding to the preset reference storage space; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; n indicates the number of unit times that have passed; C 01i represents the amount of data belonging to the video data type corresponding to the first data set in the video data acquired in the i-th unit time; C d1i represents the amount of video data acquired per unit time; C 01ci represents the amount of data belonging to the video data type corresponding to the first data set in the video data actually uploaded in the i-th unit time; C d2i Indicates the amount of video data actually uploaded per i-th unit time; Among them, C k2 represents the cache space of the data cache area corresponding to the second data set; C0 represents the storage capacity corresponding to the preset reference storage space; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; n indicates the number of unit times that have passed; C 02i represents the amount of data belonging to the video data type corresponding to the second data set in the video data acquired in the i-th unit time; C d1i represents the amount of video data acquired per unit time; C 02ci represents the amount of data belonging to the video data type corresponding to the second data set in the video data actually uploaded in the i-th unit time; C d2i Indicates the amount of video data actually uploaded per i-th unit time; Among them, C k3 represents the cache space of the data cache area corresponding to the third data set; C0 represents the storage capacity corresponding to the preset reference storage space; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; n indicates the number of unit times that have passed; C 03i represents the amount of data belonging to the video data type corresponding to the third data set in the video data acquired in the i-th unit time; C d1i represents the amount of video data acquired per unit time; C 03ci represents the amount of data belonging to the video data type corresponding to the third data set in the video data actually uploaded in the i-th unit time; C d2i Indicates the amount of video data actually uploaded per i-th unit time; A model training unit, used to create a video feature recognition model and pre-train the created video feature recognition model to obtain a data annotation training model; The video annotation unit is used to connect the target video to be annotated to the trained model for automatic annotation, and to review the annotated target video; The annotation generation module is based on the content annotated by the video annotation unit and is used to combine the annotated content with the target video, thereby generating a target video with data annotated content; The video display module is used to receive and display the original target video acquired by the video acquisition module, as well as the target video with data annotation content.

2. The interactive video data labeling device according to claim 1, characterized in that: The video acquisition module comprises: The video access module is used to configure the transmission address of the target video to be annotated so as to access the real-time target video data to be annotated; The video transmission module is used to upload the stored original target video to the video display module for display.

3. The interactive video data labeling device according to claim 2, characterized in that: Video transmission module, including: A first real-time monitoring module, used for monitoring in real time the space occupancy of the cache space of the data cache area corresponding to the first data set; The first data transmission rate setting module is used to set a first data transmission rate per unit time for the data cache area corresponding to the first data set according to the space occupation of the cache space of the data cache area corresponding to the first data set, wherein the first data transmission rate is obtained by the following formula: Among them, C s1 represents the first data transmission volume; C s0 Indicates the preset data transmission volume; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; C z1 represents the space occupancy of the cache space of the data cache area corresponding to the first data set; C k1 Represents a cache space of a data cache area corresponding to the first data set; A second real-time monitoring module, used for monitoring in real time the space occupancy of the cache space of the data cache area corresponding to the second data set; The second data transmission rate setting module is used to set a second data transmission rate per unit time for the data cache area corresponding to the second data set according to the space occupation of the cache space of the data cache area corresponding to the second data set, wherein the second data transmission rate is obtained by the following formula: Among them, C s2 represents the second data transmission volume; C s0 Indicates the preset data transmission volume; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; C z2 represents the space occupancy of the cache space of the data cache area corresponding to the second data set; C k2 Represents a cache space of a data cache area corresponding to a second data set; A third real-time monitoring module, used for monitoring in real time the space occupancy of the cache space of the data cache area corresponding to the third data set; The third data transmission volume setting module is used to set a third data transmission volume per unit time for the data cache area corresponding to the third data set according to the space occupation of the cache space of the data cache area corresponding to the third data set, wherein the third data transmission volume is obtained by the following formula: Among them, C s3 Indicates the third data transmission volume; C s0 Indicates the preset data transmission volume; C p1 Indicates the average amount of video data acquired per unit time; C p2 Indicates the minimum amount of video data that the video display module can receive per unit time; C z3 represents the space occupancy of the cache space of the data cache area corresponding to the third data set; C k3 Represents the cache space of the data cache area corresponding to the third data set.

4. The interactive video data labeling device according to claim 1, characterized in that: The model training unit comprises: A model creation module is used to create a video feature recognition model to serve as a training model for the target video to be labeled; A sample acquisition module is used to acquire video data samples and annotate the annotated points in the video data samples based on a manual annotation method, thereby obtaining annotated data samples; The model training module performs data annotation training on the video feature recognition model based on the labeled data samples, thereby obtaining a training model of the target video to be labeled.

5. The interactive video data labeling device according to claim 1, characterized in that: The video annotation unit comprises: The model loading module is used to preload the trained data annotation training model to ensure the stability of access to the target video to be annotated; The automatic annotation module, based on the data annotation training model, automatically annotates the target video to be annotated by connecting it to the trained data annotation training model, and saves the automatically annotated content; The manual annotation module is used to add, delete, modify and check the automatically annotated content in the target video to be annotated, and save the corrected annotated content for a second time; The annotation review module is used to check the automatically annotated content with the manually annotated content to ensure the accuracy of the data annotation in the target video.

6. The interactive video data labeling device according to claim 4, characterized in that: The model training module performs data annotation training on the video feature recognition model by annotating data samples, specifically: Manually select the annotation points in the video data sample. For data that requires complex annotation, annotation rules need to be formulated in advance, including: specifying the classification of objects in the image and the location of the bounding box; Use annotation tools to annotate video data samples and verify the quality of the annotated data; Organize the annotation results into a format that can be used by the machine learning model, including: converting the image annotation results into bounding box coordinates and classification labels; The labeled data is connected to the video feature recognition model for training and learning, thereby obtaining a training model for the target video to be labeled.

7. The interactive video data labeling device according to claim 5, characterized in that: The annotation review module verifies the automatically annotated content with the manually annotated content, specifically: The annotation review module is used to check the automatic and manual annotation contents in the target video to be annotated, including the classification of objects in the image, the position of the bounding box, and whether the annotation text is accurate. If the verification is successful, the annotated target video is sent to the annotation generation module to generate the classification of objects in the image and the position of the bounding box. If the verification is unsuccessful, the target video that fails the verification is returned to the manual annotation module for secondary correction until the target video passes the verification.

8. The interactive video data labeling device according to claim 5, characterized in that: The annotation review module includes: The data storage module is used to store the video data that has passed the review into a designated database.

9. A labeling method according to any one of claims 1 to 8, characterized in that: The following steps are involved: S1. Create a video feature recognition model through a model creation module, and train the model using labeled data samples, thereby obtaining a training model of the target video to be labeled; S2, obtaining a video of a target to be annotated through a smart camera device, and connecting the target data to be annotated to an annotation training model for automatic annotation, thereby obtaining automatic annotation content; S3, adding, deleting, modifying and checking the automatically annotated content through the manual annotation module, and checking the automatically annotated and manually annotated content through the annotation review module; If the content in S4 and S3 fails to pass the verification, the failed target video will be returned to the manual annotation module for secondary correction until the target video passes the verification; If the verification content is passed, the target video will be directly sent to the annotation generation module to generate the final video; S5. Display the original target video and the target video with data annotation content through the video display module.

Images