Video tag generation method, system and device, electronic device and storage medium

Abstract

The invention relates to a video tag generation method, system and device, an electronic device and a storage medium, and the method comprises the steps: constructing a video tag library, and converting tags in the video tag library into tag vectors; extracting a video frame of the target video, and converting the video frame into a video frame vector; and calculating the vector similarity between the tag vector and the video frame vector, and determining the tag corresponding to the tag vector of which the vector similarity meets the preset standard as the tag of the target video corresponding to the video frame vector, thereby improving the accuracy and efficiency of the video tag.

Description

Technical Field

[0001] The present application relates to the field of video, and in particular to a method, system, device, electronic device and storage medium for generating video tags. Background Art

[0002] With the rapid development of the media industry and the rise of short videos, the types and subdivisions of integrated media videos are increasing, and growing faster and faster. In order to quickly retrieve in various subdivisions, it is necessary to classify and mark the integrated media videos. With the classification tags, people can develop resources on this basis, combined with the current technological advances in big data, to make these video media data generate more value.

[0003] Video tags refer to specific phrases used to describe the characteristics of a video. Tagging videos can help users quickly and efficiently retrieve video content. Currently, the main method for generating video tags is manual tagging. For online video tag generation methods, they are mainly based on the understanding of images, videos, or speech text. From the image perspective, it is mainly to extract frames from the video to obtain pictures, then label the pictures, and finally integrate the image tags of the video to obtain video tags. From the video perspective, it is mainly to use video understanding methods to obtain video tags. From the speech text perspective, it is mainly to first extract information such as the voice or subtitles of the video, and then use the analysis method of speech text to obtain video tags.

[0004] The existing technology mainly uses the metadata self-labeling method. When the integrated media video is stored in the database, it comes with the video-related metadata information, which includes classification, label and other data. Label separation and matching are performed based on the label data, but this method is relatively rough and incomplete, the label data is relatively simple, and many new labels cannot be automatically adapted and associated after they come out.

[0005] Alternatively, manual review can be used, where administrators of the video playback platform manually add labels. However, due to the limited number of administrators and the uncertainty of manual review, manual review and labeling takes a long time and requires more manpower. In addition, each person has different review standards, and there are large differences in the labels prepared. If new labels are to be labeled again, they must be reviewed again, which is time-consuming, labor-intensive and slow.

[0006] Alternatively, the video content can be understood through a CNN neural network machine, and then the video can be labeled, such as a conventional neural network such as ResNet50. However, this method has problems such as low recognition error rate and low precision, and new labels require retraining of the neural network model.

[0007] There is currently no effective solution to the problems of low accuracy and efficiency of video labeling in related technologies. Summary of the invention

[0008] In this embodiment, a video tag generation method, system, device, electronic device and storage medium are provided to solve the problems of low accuracy and efficiency of video tags in related technologies.

[0009] In a first aspect, a video tag generation method is provided in this embodiment, including: Build a video tag library and convert the tags in the video tag library into tag vectors; Extracting video frames of the target video and converting the video frames into video frame vectors; The vector similarity between the label vector and the video frame vector is calculated, and the label corresponding to the label vector whose vector similarity meets the preset standard is identified as the label of the target video corresponding to the video frame vector.

[0010] In some of the embodiments, extracting a video frame of a target video includes: Segment the target video into unit plot video segments; Video frames are extracted based on unit plot video segments.

[0011] In some of the embodiments, extracting a video frame of a target video includes: The video frames of the target video are extracted according to a predetermined time interval; or the video frames are extracted by averaging frames according to the total length of the target video.

[0012] In some of the embodiments, the vector similarity is obtained by calculating the Euclidean distance between the label vector and the video frame vector.

[0013] In some embodiments, calculating the vector similarity between the label vector and the video frame vector, and identifying the label corresponding to the label vector whose vector similarity meets the preset standard as the label of the target video corresponding to the video frame vector includes: According to the size of the calculated vector similarity, the labels corresponding to the label vectors whose vector similarity meets the preset standard are sorted, and the top parameter is used to select the label in the front order and identify it as the label of the target video corresponding to the video frame vector.

[0014] In some of the embodiments, when a new tag is added, the new tag is added to the video tag library, and the new tag is converted into a new tag vector; The vector similarity between the new label vector and the video frame vector is calculated, and the label corresponding to the new label vector whose vector similarity meets the preset standard is identified as the label of the target video corresponding to the video frame vector.

[0015] In a second aspect, a video tag generation system is provided in this embodiment, and the video tag generation method described in the first aspect is applied. The system includes: an access terminal, an API access layer, a business service layer, a data processing layer, a database, and an operating environment layer, wherein: The access terminal is used to collect the target video and input the target video into the business service layer through the API access layer; The API access layer includes API gateway and load balancing, which are used to unify the entry and balance the traffic; The business service layer is used to label the target video with a corresponding label according to the video label generation method; Data layer, used for data processing, storage and management; The database includes message queues, cache data, and relational storage data; The runtime environment layer is built using Python language and FastAPI framework.

[0016] In a third aspect, a video tag generation device is provided in this embodiment, including: a first conversion module, a second conversion module and a tag module, wherein: The first conversion module is used to build a video tag library and convert the tags in the video tag library into tag vectors; A second conversion module, used for extracting a video frame of a target video and converting the video frame into a video frame vector; The label module is used to calculate the vector similarity between the label vector and the video frame vector, and identify the label corresponding to the label vector whose vector similarity meets the preset standard as the label of the target video corresponding to the video frame vector.

[0017] In a fourth aspect, an electronic device is provided in this embodiment, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the video tag generation method described in the first aspect is implemented.

[0018] In a fifth aspect, a storage medium is provided in this embodiment, on which a computer program is stored, and when the program is executed by a processor, the video tag generation method described in the first aspect is implemented.

[0019] Compared with the related art, the video tag generation method provided in this embodiment, by constructing a video tag library, converts the tags in the video tag library into tag vectors; extracts the video frames of the target video, and converts the video frames into video frame vectors; calculates the vector similarity between the tag vector and the video frame vector, and identifies the tag corresponding to the tag vector whose vector similarity meets the preset standard as the tag of the target video corresponding to the video frame vector, thereby improving the accuracy and efficiency of video tags.

[0020] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 4 is a hardware structure block diagram of a terminal of the video tag generation method of this embodiment.

[0022] Figure 2 4 is a flowchart of the video tag generation method of this embodiment.

[0023] Figure 3 This is a flowchart of another method for generating video tags according to this embodiment.

[0024] Figure 4 4 is a diagram of the video tag generation system architecture of this embodiment.

[0025] Figure 5 It is a structural block diagram of the video tag generating device of this embodiment. DETAILED DESCRIPTION

[0026] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0027] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0028] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of the video tag generation method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0029] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the video tag generation method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, a method for generating a video tag is provided. Figure 2 is a flow chart of the video tag generation method of this embodiment. Figure 2 As shown, the process includes the following steps: Step S201, construct a video tag library, and convert the tags in the video tag library into tag vectors.

[0032] Specifically, a video tag library is first constructed, which may include all tags that need to be annotated in existing videos, for example, theme tags: travel, food, technology, education, scene tags: beach, city, forest, family, character tags: celebrities, children, athletes, actors, action tags: running, dancing, cooking, speech, etc. Specifically, for a video scene of a little boy riding a bicycle to a park, the tags that may need to be annotated include little boy, bicycle, park, etc. The data in the video tag library is first preprocessed, including cleaning data, removing invalid or erroneous tags, and converting tags to a format acceptable to the model, such as text segmentation, standardization, etc. The neural network model is used to extract features from the preprocessed tag data. In this embodiment, the video encoder in the clip4clip neural network model is used to extract features from the tag data. By inputting the tag data text into the model, the tag vector corresponding to the tag data text is obtained. Among them, the neural network model can also be used in other ways, and this embodiment does not make specific restrictions on this.

[0033] Step S202, extracting video frames of the target video and converting the video frames into video frame vectors.

[0034] Specifically, after obtaining the target video, the input target video is first decoded and sampled into a series of frame images. These frame images are used as the basic units of the target video content. The scene content is understood by extracting frames, and features of these frame images are extracted through the clip4clip neural network model and converted into feature vectors, thereby realizing the conversion of the target video into video frame vectors.

[0035] Step S203, calculating the vector similarity between the label vector and the video frame vector, and identifying the label corresponding to the label vector whose vector similarity meets the preset standard as the label of the target video corresponding to the video frame vector.

[0036] Specifically, after the label vectorization of the video label library and the video frame vectorization, the similarity between the label vector and the video frame vector is calculated, wherein the similarity can be calculated by some commonly used vector similarity comparison methods, such as cosine similarity, normalized cosine similarity, weighted cosine similarity, Euclidean distance, Manhattan distance, Jaccard similarity coefficient, Hamming distance, Pearson correlation coefficient, Spearman rank correlation coefficient, Mahalanobis distance and inner product, etc. It can be selected based on the specific application scenario. In this embodiment, Euclidean distance is mainly used to calculate the similarity. According to the preset similarity threshold, the label corresponding to the label vector whose similarity meets the standard is identified as the label of the target video corresponding to the video frame vector, thereby realizing the mapping from the video frame to the label, and the target video corresponding to the video frame is marked with a corresponding label for understanding, retrieval and other applications of the video content. Among them, the threshold of the similarity can be determined according to the specific similarity comparison tool used, and this embodiment does not specifically limit this.

[0037] Through the above steps 201 to S203, a video tag library is constructed, and the tags in the video tag library are converted into tag vectors; video frames of the target video are extracted, and the video frames are converted into video frame vectors; the vector similarity between the tag vector and the video frame vector is calculated, and the tag corresponding to the tag vector whose vector similarity meets the preset standard is identified as the tag of the target video corresponding to the video frame vector. Compared with the method of metadata self-labeling and manual review in the prior art, this embodiment converts the target video into a video frame vector first, converts the tag of the video tag library into a tag vector, and then compares the video frame vector with the tag vector for similarity, thereby obtaining the tag corresponding to the video frame vector, and then obtaining the tag of the target video, and obtains the video tag by comparing the similarity between the vectors, thereby improving the accuracy of the video tag, and by processing the video tag library and the target video separately, reducing the coupling between the two, and being more flexible when performing video tagging, and finally, by performing vector conversion and similarity comparison through a set program, the efficiency of automatic matching is improved, and the problem of low efficiency of video tagging caused by manual review is solved.

[0038] In some of the embodiments, extracting a video frame of a target video includes: The target video is divided into unit plot video segments; and video frames are extracted according to the unit plot video segments.

[0039] Specifically, existing target videos, such as those in film and television works, are often relatively long. After obtaining a relatively long target video, the target video is segmented into multiple video clips according to the video plot through a shot cutting algorithm. Each video clip consists of continuous frames, where a video clip can be a plot or a solution to a problem. Each segmented video clip is a target sub-video, which can be labeled separately, thereby making the label of the target video more detailed and accurate. The target video is cut under the ffmpeg multimedia framework by calculating the difference between the pixels of two video frame images. If the difference exceeds a certain threshold, it is considered that a shot switch has occurred. The shot cutting algorithm uses the following formula: ; in, k is the total number of pixels in the video frame, p is the number of lens blocks, h m (i, j)-h n (i, j) Represents two video frames h m (i, j) and h n (i, j) exist (i, j) The distance above this histogram unit; X 2 Represents the difference between two video frames. X 2 When it is 0, it means that the previous and next video frames belong to the same video plot and are not video segmentation points; when X 2 When it is not 0, it means that there is a large difference between the previous and next video frames, and the video can be segmented at this point.

[0040] The starting point and the end point of the segmented video are obtained through the shot cutting algorithm, and then the segmented video is obtained. Then, the clip4clip neural network model is used to extract a preset number of video frames from the description of all frames in each segmented video. The extracted video frames in each segmented video are converted into video frame vectors, and all video frame vectors are saved in the vector database. One target video corresponds to multiple segmented videos, and multiple segmented videos correspond to multiple video frame vectors.

[0041] In another embodiment, extracting a video frame of a target video includes: The video frames of the target video are extracted according to a predetermined time interval; or the video frames are extracted by averaging frames according to the total length of the target video.

[0042] Specifically, if the target video is short, the target video may not be segmented, and video frames may be directly extracted. If the target video is long, the target video may be segmented according to the video segmentation scheme of the above embodiment to obtain segmented videos, and video frames may be extracted from the segmented videos. The video frames may be extracted according to a predetermined time interval, for example, one frame is extracted every 2 seconds as a video frame, or the video may be extracted in an average manner according to the total length of the target video, for example, the target video may be segmented into 15 segmented videos and segmented using an average algorithm. The specific time interval and the number of segmentations may be determined according to actual needs, and this embodiment does not specifically limit this.

[0043] In some embodiments, the vector similarity is obtained by calculating the Euclidean distance between the label vector and the video frame vector.

[0044] Specifically, in this embodiment, the similarity comparison between the label vector and the video frame vector is obtained by Euclidean distance. By judging whether the calculated Euclidean distance between the label vector and the video frame vector meets the preset distance threshold, the label corresponding to the label vector that meets the distance threshold is used as the label of the corresponding video frame vector, and then mapped to the target video label corresponding to the video frame. Among them, the smaller the Euclidean distance, the closer the label vector is to the video frame vector, and the more accurate the label corresponding to the video frame is. The preset distance threshold can be set according to the actual situation. The smaller the distance threshold, the higher the label accuracy. Among them, a certain error range can also be allowed to avoid the situation where the video frame vector cannot match the label vector. This embodiment does not make specific limitations on this.

[0045] In another embodiment, calculating the vector similarity between the label vector and the video frame vector, and identifying the label corresponding to the label vector whose vector similarity meets a preset standard as the label of the target video corresponding to the video frame vector, includes: According to the size of the calculated vector similarity, the labels corresponding to the label vectors whose vector similarity meets the preset standard are sorted, and the top parameter is used to select the label in the front order and identify it as the label of the target video corresponding to the video frame vector.

[0046] Specifically, a similarity calculation is performed on the converted label vector and the video frame vector to obtain a similarity calculation value. First, label vectors that meet the standard are screened out according to the set similarity standard. When there are many label vectors that meet the standard, these label vectors that meet the standard are sorted from large to small according to the similarity calculation value. By setting a Top parameter, such as TOP-n, the top n labels are selected as labels of the video frame vector, and these labels are used as representatives to describe the target video as labels of the target video.

[0047] In some other embodiments, when a new tag is added, the new tag is added to the video tag library, and the new tag is converted into a new tag vector; The vector similarity between the new label vector and the video frame vector is calculated, and the label corresponding to the new label vector whose vector similarity meets the preset standard is identified as the label of the target video corresponding to the video frame vector.

[0048] Specifically, when new things appear in the video and need to be marked, the video label library and the target video are decoupled, so that the video description features are automatically separated from the label features. There is no need to repeatedly use the labels to train the neural network model. After adding the labels directly to the video label library, the input target video can be directly matched to the corresponding new labels through the vector similarity matching method described in the above embodiment, so as to achieve the effect of one-time storage and repeated use, fast matching speed and high accuracy.

[0049] This embodiment also provides a method for generating video tags. Figure 3 is a flowchart of another method for generating video tags in this embodiment. Figure 3 As shown, the process includes the following steps: Step S301, constructing a video tag library, and converting tags in the video tag library into tag vectors; Step S302, dividing the target video into unit plot video segments; extracting a preset number of video frames according to the unit plot video segments; Step S303, converting the video frame into a video frame vector; Step S304, calculating the Euclidean distance between the label vector and the video frame vector; Step S305 , sorting the labels corresponding to the label vectors whose Euclidean distances meet the preset standard according to the calculated Euclidean distances, and selecting the labels with the highest sorting by using the Top parameter, and identifying them as the labels of the target video corresponding to the video frame vectors.

[0050] Through the above steps S301 to S305, compared with the method of metadata self-labeling and manual review in the prior art, this embodiment converts the labels in the preset label library into label vectors, and then segments the target video to obtain a segmented video; extracts a preset number of video frames in the segmented video, and converts the video frames into video frame vectors; calculates the Euclidean distance between the video frame vector and the label vector, sorts the label vectors whose Euclidean distance meets the preset standard, and takes the label vector arranged in front as the label of the target video corresponding to the video frame vector to complete the video labeling. Among them, by converting both the video frame and the label into vectors, and matching and obtaining the corresponding video label by calculating the Euclidean distance between the vectors, automatic labeling is achieved, and the accuracy and efficiency of video labeling are improved.

[0051] In this embodiment, a video tag generation system is also provided. Figure 4 This is the architecture diagram of the video tag generation system of this embodiment. The system applies the video tag generation method described in any of the above embodiments. The video tag generation system includes: an access terminal 41, an API access layer 42, a business service layer 43, a data layer 44, a database 45, and an operating environment 46, wherein the access terminal 41 is used to collect the target video and input the target video into the business service layer through the API access layer. The API access layer 42 includes an API gateway and a load balancer for unifying the entrance and balancing the traffic; the business service layer 43 is used to label the target video with the corresponding label according to the video tag generation method; the data layer 44 is used for data processing, storage, and management; the database 45 includes a message queue, task data, cache data, and relational storage data; the operating environment 46 is built using the Python language and the FastAPI framework.

[0052] Specifically, the system business functions include: Role management: adding, modifying, and deleting roles such as user administrator, system administrator, and operator; Organization management: add, modify, and delete the organization and department to which the user belongs; Tag library management: add, modify, and delete tags used to associate videos; Video management: used for storage video management, adding, modifying, deleting, marking and maintaining; Task scheduling: used for video storage scheduling, asynchronous video file pulling, video shot segmentation scheduling, shot frame cutting scheduling, shot feature extraction scheduling, label understanding scheduling, and video shot and label automatic identification scheduling.

[0053] The access terminal 41 is used to collect the target video and input the target video to the business service layer through the API access layer. The access terminal 41 includes: PC terminal, mobile client, applet and other access parties.

[0054] The API access layer 42 includes an API gateway and a load balancer, which are used to unify the entry and balance the traffic. The main functions are as follows: 1. Unified entry: API Gateway serves as a single entry point for all client requests. It receives all API calls from front-end applications, mobile applications, third-party systems, etc., and then routes them to the corresponding back-end microservices based on the characteristics of the request.

[0055] 2. Protocol conversion: API gateway can handle request protocols of different clients, such as converting HTTP requests into gRPC or RESTful API suitable for internal communication of microservices, and vice versa, thereby achieving protocol unification and compatibility.

[0056] 3. Security control: Centrally implement security policies such as authentication, authorization, rate limiting, and API key management to protect backend services from malicious attacks and ensure the security of services.

[0057] 4. Traffic management: API gateway can achieve load balancing of requests and dynamically allocate traffic according to the health status and capacity of microservices. It also supports strategies such as circuit breaking and degradation to enhance the stability and reliability of the system.

[0058] 5. Monitoring and logging: Provides centralized API monitoring, logging, and analysis capabilities to facilitate tracking of the entire request chain, timely identification and resolution of problems, and also provides data support for performance tuning and capacity planning.

[0059] 6.API version management: supports the coexistence of different versions of API, facilitating smooth transition between old and new versions.

[0060] 7. Caching and optimization: Cache frequently accessed and infrequently changing data to reduce direct requests to backend services, improve response speed and reduce system pressure.

[0061] The business service layer 43 is used to label the target video according to the video label generation method, including various business functions and management, which are divided into business control and business end. Business control specifically includes: API interface, task scheduling, role architecture and organizational structure; the business end includes: asynchronous file pulling, lens video frame extraction, label library management, label and lens content comparison, lens scene segmentation, lens frame picture understanding, label library understanding and video lens automatic identification. Its main functions are: 1. Business logic processing: The business service layer carries the main business processes and rules of the system, decomposing complex business requirements into a series of fine-grained services. Each service is responsible for handling specific business functions, such as video library management, tag library processing, identification and separation control, etc.

[0062] 2. Service orchestration: responsible for coordinating and orchestrating the interactions between multiple microservices to implement cross-service business processes.

[0063] 3. Domain model implementation: Based on the principles of domain-driven design (DDD), the business service layer implements the domain model, encapsulates entities, value objects, and business rules in the business domain, and maintains the purity of business logic and centralized management of domain knowledge.

[0064] 4. Service decoupling: Clear service boundaries are defined through interfaces, making each business service independent of other services and able to be developed, tested, deployed and expanded separately, reducing the coupling between services and improving the maintainability and scalability of the system.

[0065] 5. API exposure: Some business services may be directly facing external clients or serve as dependencies for other services, exposing necessary interfaces through RESTful API, gRPC, etc. to achieve communication and integration between services.

[0066] The data processing layer is used for data processing, storage and management; the database includes message queues, task data, cache data and relational storage data; the data processing layer includes the data layer and the database. The data layer includes: data cache, data transactions and file servers; the database includes: Kafka, redis and mysql; the main functions of the data layer are as follows: 1. Data storage and management: The data layer is responsible for storing various data required by the application, including but not limited to user data, business data, configuration information, etc. It may include a variety of database technologies, such as relational databases (MySQL), key-value storage (Redis), etc., to meet the needs of different business scenarios.

[0067] 2. Data Sharding and Replication: In order to cope with large-scale data storage and high concurrent access, the data layer usually uses sharding technology to distribute data to multiple database nodes, and data replication to ensure high data availability and disaster recovery capabilities. This helps improve data access performance and system scalability.

[0068] 3. Transaction processing and consistency: In a distributed environment, transaction consistency needs to be maintained. The data layer needs to ensure data consistency and integrity through distributed transactions, transaction compensation mechanisms, or the use of eventual consistency strategies.

[0069] 4. Cache strategy: In order to improve data access speed and reduce the direct burden on the database, the data layer often integrates cache services, such as Redis or Memcached, to store hot data. A reasonable cache strategy can significantly improve the response speed of the system.

[0070] 5. Data access abstraction: Through the data access layer (DAO, Data Access Object) or ORM (Object-Relational Mapping) framework, a unified data access interface is provided for the business service layer, hiding the specific implementation details of the underlying data storage, so that the business logic and data access logic are decoupled, which is easy to maintain and expand.

[0071] 6. Data integration and synchronization: In a microservice architecture, different services may have their own data storage. The data layer also needs to handle data integration and synchronization issues between services to ensure data consistency and timeliness.

[0072] The data layer 44 is used for data processing, storage and management, including data cache, data transactions and file servers; the database 45 includes message queues, cached data and relational storage. The database 45 contains kafka message queues and task data, redis cache and fast reading data, and mysql relational storage database.

[0073] The operating environment 46 of the system is built with Python language and FastAPI framework, runs in uvicorn container, and supports docker container.

[0074] The characteristics of the system's layered distributed microservice architecture are: 1. Clear hierarchy: The system is divided into multiple layers, such as access layer, business service layer, data layer, etc. Each layer consists of a group of related services and is responsible for a specific functional area.

[0075] 2. Loose coupling: Despite the layered structure, the services between layers remain loosely coupled. The services of each layer only communicate with the layers directly adjacent to it, reducing cross-layer dependencies and facilitating independent development, testing, and deployment of services at each layer.

[0076] 3. Standardized interface: In order to ensure efficient and consistent communication between layers, clear API interface specifications are usually defined, such as RESTful API or gRPC protocol, as the standard for interaction between layers.

[0077] 4. Enhanced reusability: Layered design enables the reuse of services, especially services in the infrastructure layer (such as authentication, logging, and monitoring) and the data access layer, which can be called by services in multiple business logic layers, reducing duplicate code and improving development efficiency.

[0078] 5. High scalability: The system can easily add or reduce service instances according to business needs, and each service can be scaled independently, thereby linearly improving the processing capacity of the entire system and better coping with traffic peaks.

[0079] 6. Fault tolerance and high availability: Since services are relatively independent, the failure of a single service will not directly affect the entire system. The high availability of the system can be achieved through service redundancy and fast failover mechanism.

[0080] 7. Flexible technology selection: Each service can independently select the most suitable technology stack based on its business needs, without being restricted by the unified standards of the entire system, which promotes technological innovation and optimization.

[0081] 8. Better resource utilization: Under the microservice architecture, resource allocation is more refined, and resources can be dynamically adjusted according to the actual needs of the service, avoiding over-allocation or waste of resources.

[0082] 9. Easy to maintain and upgrade: The separation of services makes problem location clearer, and local modifications and upgrades do not affect the entire system, reducing maintenance costs and risks.

[0083] The system uses a layered distributed microservice architecture. The lens scene segmentation microservice unit breaks down the target video into unit plot video segments according to the shots. The lens video frame extraction microservice unit extracts the key parts of the segments frame by frame. The neural network model microservice unit understands and obtains the expression description vector data of the segment video. The video is subdivided into plot understandings and then converted into video vector data and saved in the database. The label management module is used to manage the label library data. The neural network model microservice unit task understands the label library to obtain the label vector, compares the label vector with the video vector, and then uses the video lens automatic identification microservice unit to automatically tag the plot segments in the video with corresponding labels, thereby associating the video and improving the accuracy and efficiency of video labeling.

[0084] In this embodiment, a video tag generation device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware of predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0085] Figure 5 is a structural block diagram of the video tag generating device of this embodiment. Figure 5 As shown, the device 50 includes: a first conversion module 51, a second conversion module 52 and a label module 53, wherein: A first conversion module 51 is used to construct a video tag library and convert tags in the video tag library into tag vectors; A second conversion module 52, used for extracting a video frame of a target video and converting the video frame into a video frame vector; The label module 53 is used to calculate the vector similarity between the label vector and the video frame vector, and identify the label corresponding to the label vector whose vector similarity meets the preset standard as the label of the target video corresponding to the video frame vector.

[0086] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0087] In this embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0088] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0089] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program: S1, build a video tag library and convert the tags in the video tag library into tag vectors; S2, extracting the video frames of the target video and converting the video frames into video frame vectors; S3, calculating the vector similarity between the label vector and the video frame vector, and identifying the label corresponding to the label vector whose vector similarity meets the preset standard as the label of the target video corresponding to the video frame vector.

[0090] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0091] In addition, in combination with the video tag generation method provided in the above embodiment, a storage medium can also be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any of the video tag generation methods in the above embodiment is implemented.

[0092] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0093] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0094] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0095] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0096] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A video tag generation method, characterized in that: include: Building a video tag library, and converting tags in the video tag library into tag vectors; Extracting a video frame of a target video and converting the video frame into a video frame vector; The vector similarity between the label vector and the video frame vector is calculated, and the label corresponding to the label vector whose vector similarity meets a preset standard is identified as the label of the target video corresponding to the video frame vector.

2. The video tag generation method according to claim 1, characterized in that: Extract the video frames of the target video, including: Segmenting the target video into unit plot video segments; The video frames are extracted according to the unit plot video segment.

3. The video tag generation method according to claim 1 or claim 2, characterized in that: The step of extracting a video frame of a target video includes: The video frames of the target video are extracted according to a predetermined time interval; or the video frames are extracted by averaging frames according to the total length of the target video.

4. The video tag generation method according to claim 1, characterized in that: The vector similarity is obtained by calculating the Euclidean distance between the label vector and the video frame vector.

5. The video tag generation method according to claim 1, characterized in that: Calculating the vector similarity between the label vector and the video frame vector, and identifying the label corresponding to the label vector whose vector similarity meets a preset standard as the label of the target video corresponding to the video frame vector, including: According to the magnitude of the calculated vector similarity, the labels corresponding to the label vectors whose vector similarity meets the preset standard are sorted, and the label in the front order is selected using the Top parameter and identified as the label of the target video corresponding to the video frame vector.

6. The video tag generation method according to claim 1, characterized in that: The method further comprises: When a new tag is added, the new tag is added to the video tag library, and the new tag is converted into a new tag vector; The vector similarity between the new label vector and the video frame vector is calculated, and the label corresponding to the new label vector whose vector similarity meets a preset standard is identified as the label of the target video corresponding to the video frame vector.

7. A video tag generation system, using the video tag generation method according to any one of claims 1 to 6, characterized in that: The system includes: an access terminal, an API access layer, a business service layer, a data layer, a database, and an operating environment layer, wherein: The access terminal is used to collect the target video and input the target video into the business service layer through the API access layer; The API access layer includes an API gateway and a load balancer, which are used to unify the entrance and balance the traffic; The business service layer is used to label the target video with a corresponding label according to the video label generation method; The data layer is used for data processing, storage and management; The database includes a message queue, cache data and relational storage data; The operating environment layer is built using Python language and FastAPI framework.

8. A video tag generation device, characterized in that: include: A first conversion module, a second conversion module and a label module, wherein: The first conversion module is used to build a video tag library and convert the tags in the video tag library into tag vectors; The second conversion module is used to extract video frames of the target video and convert the video frames into video frame vectors; The label module is used to calculate the vector similarity between the label vector and the video frame vector, and identify the label corresponding to the label vector whose vector similarity meets a preset standard as the label of the target video corresponding to the video frame vector.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the video tag generating method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the video tag generating method according to any one of claims 1 to 6 are implemented.

Images