A label extraction method, device and computer readable storage medium

Abstract

The application provides a label extraction method, device and computer readable storage medium; the method comprises the following steps: receiving a video to be extracted and a title text corresponding to the video to be extracted; extracting a to-be-identified image from the video to be extracted; the to-be-identified image is a cover image of the video to be extracted or a video frame of the video to be extracted; identifying the to-be-identified image to obtain an image category label corresponding to the to-be-identified image; the image category label represents an object contained in the to-be-identified image; performing word extraction on the title text to obtain at least one candidate label corresponding to the title text; the at least one candidate label is a noun word in the title text; based on the image category label, selecting a label corresponding to the video to be extracted from the at least one candidate label. Through the application, the extraction efficiency of the label of the video can be improved through artificial intelligence.

Description

Technical Field

[0001] This application relates to artificial intelligence technology, and more particularly to a tag extraction method, device, and computer-readable storage medium. Background Technology

[0002] When watching short videos, users typically only click to play those that interest them. To help users select videos they are interested in from a vast library, it's necessary to extract tags from the videos, allowing the system to select videos that might interest the user based on those tags.

[0003] In related technologies, short video tags are mainly assigned manually or extracted from keywords in the video's title. However, manual tagging has low coverage and the titles contain limited information, resulting in poor tag extraction efficiency. Summary of the Invention

[0004] This application provides a tag extraction method, device, and computer-readable storage medium, which can improve the tag extraction efficiency of videos.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a tag extraction method, including:

[0007] Receive the video to be extracted and the title text corresponding to the video to be extracted;

[0008] Extract the image to be identified from the video to be extracted; the image to be identified is the cover image of the video to be extracted or a video frame of the video to be extracted;

[0009] The image to be identified is identified to obtain the image category label corresponding to the image to be identified; the image category label represents the object contained in the image to be identified;

[0010] Word extraction is performed on the title text to obtain at least one candidate tag corresponding to the title text; the at least one candidate tag is a noun word in the title text.

[0011] Based on the image category label, the label corresponding to the video to be extracted is selected from the at least one candidate label.

[0012] This application provides a tag extraction device, including:

[0013] The video receiving module is used to receive the video to be extracted and the title text corresponding to the video to be extracted;

[0014] An image extraction module is used to extract an image to be identified from the video to be extracted; the image to be identified is the cover image of the video to be extracted or a video frame of the video to be extracted;

[0015] An image recognition module is used to recognize the image to be recognized and obtain an image category label corresponding to the image to be recognized; the image category label represents the object contained in the image to be recognized;

[0016] The candidate tag extraction module is used to extract words from the title text to obtain at least one candidate tag corresponding to the title text; the at least one candidate tag is a noun word in the title text.

[0017] The label determination module is used to select the label corresponding to the video to be extracted from the at least one candidate label based on the image category label.

[0018] In some embodiments of this application, when the image to be identified is the cover image of the video to be extracted, the image category label is the cover category label corresponding to the cover image; the label determination module is further configured to calculate the similarity between the cover category label and each of the at least one candidate labels to obtain a first word similarity corresponding to each candidate label; compare the first word similarity corresponding to each candidate label with a preset similarity threshold to obtain at least one comparison result corresponding to the at least one candidate label; the comparison result indicates whether the first word similarity is greater than the preset similarity threshold; and select the label corresponding to the video to be extracted from the at least one candidate label according to the at least one comparison result.

[0019] In some embodiments of this application, the tag determination module is further configured to select a target comparison result from the at least one comparison result, which indicates that the similarity of the first word is greater than the preset similarity threshold; and select a tag corresponding to the video to be extracted from the at least one candidate tag.

[0020] In some embodiments of this application, when the image to be identified is a video frame of the video to be extracted, the image category label is the video category label corresponding to the video frame; the label determination module is further configured to perform similarity calculation on each of the at least one candidate label and the video category label corresponding to the video frame, respectively, to obtain at least one second word similarity corresponding to the at least one candidate label; and select the label corresponding to the video to be extracted from the at least one candidate label based on the at least one second word similarity.

[0021] In some embodiments of this application, the tag determination module is further configured to select the largest preset number of second word similarities from the at least one second word similarity as the target word similarity; and select the tag corresponding to the video to be extracted from the at least one candidate tag that corresponds to the target word similarity.

[0022] In some embodiments of this application, the candidate tag extraction module is further configured to perform word segmentation on the title text to obtain multiple word segmentation results corresponding to the title text; select a first candidate word segment that hits a preset entity dictionary from the multiple word segmentation results; perform part-of-speech tagging on one or more other word segmentation results other than the first candidate word segmentation result to obtain one or more tagging results of the one or more other word segmentation results; the one or more tagging results characterize the part of speech of the one or more other word segmentation results; select a second candidate word segment from the one or more other word segmentation results according to the one or more tagging results; and integrate the first candidate word segmentation and the second candidate word segmentation to form the at least one candidate tag corresponding to the title text.

[0023] In some embodiments of this application, the candidate tag extraction module is further configured to select noun tagging results representing nouns from the one or more tagging results; and to determine other word segmentation results corresponding to the noun tagging results from the one or more other word segmentation results as the second candidate word segmentation.

[0024] In some embodiments of this application, the image recognition module is further configured to extract image features of the image to be recognized using an acquired preset image recognition model; and to recognize the image features to obtain the image category label corresponding to the image to be recognized.

[0025] This application provides a server, including:

[0026] Memory, used to store executable tag extraction instructions;

[0027] The processor is configured to execute executable tag extraction instructions stored in the memory to implement the tag extraction method provided in the embodiments of this application.

[0028] This application provides a computer-readable storage medium storing executable tag extraction instructions, which are used to implement the tag extraction method provided in this application when executed by a processor.

[0029] The embodiments of this application have the following beneficial effects:

[0030] The server can extract the image to be identified from the received video to be extracted and identify the image category label corresponding to the image to be identified, thereby obtaining the objects contained in the image to be identified. Then, it extracts nouns from the title text of the received video to be extracted, that is, initially determines at least one candidate label. Finally, based on the image category label, it further selects the label corresponding to the video to be extracted from at least one candidate label. In this way, it realizes the automatic extraction of labels from the title with less information by combining the screen content of the video to be extracted, thus improving the efficiency of video label extraction. Attached Figure Description

[0031] Figure 1 An example diagram of artificially labeled short videos in related technologies is shown;

[0032] Figure 2 This is an optional architecture diagram of the tag extraction system 100 provided in this application embodiment;

[0033] Figure 3 This is provided by the embodiments of this application. Figure 2 A structural diagram of server 200 in the diagram;

[0034] Figure 4 This is an optional flowchart illustrating the tag extraction method provided in the embodiments of this application. Figure 1 ;

[0035] Figure 5 This is an optional flowchart illustrating the tag extraction method provided in the embodiments of this application. Figure 2 ;

[0036] Figure 6 This is an optional flowchart illustrating the tag extraction method provided in the embodiments of this application. Figure 3 ;

[0037] Figure 7 This is an optional flowchart illustrating the tag extraction method provided in the embodiments of this application. Figure 4 ;

[0038] Figure 8 This is an example diagram illustrating the process of extracting at least one candidate label provided in an embodiment of this application;

[0039] Figure 9 This is an example diagram of the preset image recognition model provided in the embodiments of this application;

[0040] Figure 10 This is an example image illustrating the classification and recognition of short video covers using a trained VGGNet model in a real-world application scenario provided in this application embodiment. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0043] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0045] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0046] 1) Artificial Intelligence (AI) is the theory, methods, technology, and application systems by which digital computers or machines controlled by digital computers simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and generate new intelligent machines that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0047] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0048] 2) Computer Vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to using cameras and computers to replace human eyes in recognizing and measuring targets, and then performing image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and other technologies such as facial recognition and fingerprint recognition.

[0049] 3) Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close relationship with linguistic research. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0050] 4) Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, and inductive learning.

[0051] 5) Short videos, usually referring to videos with a duration of less than 5 minutes.

[0052] 6) Candidate tags refer to the entity words that have been preliminarily identified as having the potential to be used as tags for short videos. In other words, the tags for short videos are selected from the candidate tags.

[0053] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0054] Tagging short videos is an application of artificial intelligence. When watching short videos, users often only click to play those that interest them. To help users select videos of interest from a vast library, it's necessary to extract tags from the videos, thus selecting potentially engaging content based on these tags. Furthermore, short video tags can be used for grouping and expanding the reach of short videos beyond their initial target audience.

[0055] In related technologies, users are typically guided to manually tag short videos through product means, or based on the principles of information theory, the most informative title keywords are extracted from the largest to the smallest by calculating word frequency * inverse document frequency.

[0056] For example, Figure 1 An example diagram of artificially labeled short videos in related technologies is shown. See also Figure 1 The short video content is 1-1, and the title of the short video is 1-2: The next episode is here! How close can you get to the animals in the wildlife park? Watch the video and you'll find out! After the title of the short video 1-2, there are also artificial tags 1-3: #WonderlandShakeItUp##Travel#.

[0057] However, in practical applications, due to various reasons, many users do not actively tag short videos, resulting in low coverage of manual tagging, meaning that effective tags cannot be extracted for most short videos. Furthermore, extracting title keywords based on information theory principles is generally applied to longer texts, while short video titles are typically shorter, with fewer words and less information; most words may appear only once, leading to inaccurate title keyword extraction. Therefore, it is evident that the tag extraction efficiency for short videos is relatively poor among related technologies.

[0058] This application provides a tag extraction method, device, and computer-readable storage medium, which can improve the tag extraction efficiency of videos. The following describes exemplary applications of the tag extraction device provided in this application. The tag extraction device provided in this application can be implemented as various types of user terminals or as a server. The server can be a standalone server, a server cluster or distributed system composed of multiple servers, or a cloud server providing basic cloud computing servers such as cloud servers, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein. The following describes exemplary applications when the device is implemented as a server.

[0059] See Figure 2 , Figure 2 This is an optional architecture diagram of the tag extraction system 100 provided in this application embodiment. To support a tag extraction application, terminal 400 (terminals 400-1 and 400-2 are shown as examples) connects to server 200 through network 300. Network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of both. Server 200 is also connected to database 500, which provides data services to server 200.

[0060] Terminal 400-1 is used to record and upload the video to be extracted. That is, after the user has recorded the video to be extracted and added a cover image and title text to the video, after clicking the upload button on the graphical interface 400-11, terminal 400-1 will upload the video to be extracted and the corresponding title text to the server 200 through network 300.

[0061] Server 200 is used to receive the video to be extracted and the title text corresponding to the video to be extracted; extract the image to be identified from the video to be extracted, wherein the image to be identified is the cover image of the video to be extracted or a video frame of the video to be extracted; identify the image to be identified to obtain the image category label corresponding to the image to be identified, wherein the image category label represents the object contained in the image to be identified; extract words from the title text to obtain at least one candidate label corresponding to the title text, wherein the at least one candidate label is a noun in the title text; and select the label corresponding to the video to be extracted from the at least one candidate label based on the image category label.

[0062] After server 200 identifies the tags corresponding to the videos to be extracted, it can push the videos to the user based on the user's interest tags.

[0063] Terminal 400-2 is used to receive the video to be extracted pushed by the server and display the video to be extracted on the graphical interface 400-21.

[0064] See Figure 3 , Figure 3 This is provided by the embodiments of this application. Figure 2 A structural diagram of server 200 in the diagram. Figure 3 The server 200 shown includes at least one processor 210, memory 250, at least one network interface 220, and a user interface 230. The various components in server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 3 The general labeled all buses as Bus System 240.

[0065] Processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0066] User interface 230 includes one or more output devices 231 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 230 also includes one or more input devices 232, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0067] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 250 may optionally include one or more storage devices physically located away from the processor 210.

[0068] Memory 250 may include volatile memory or non-volatile memory, or both. Non-volatile memory may be read-only memory (ROM), and volatile memory may be random access memory (RAM). The memory 250 described in this application embodiment is intended to include any suitable type of memory.

[0069] In some embodiments, memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0070] Operating system 251 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0071] The network communication module 252 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220, such as Bluetooth, Wi-Fi, and Universal Serial Bus (USB).

[0072] Presentation module 253 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 231 (e.g., a display screen, a speaker, etc.) associated with user interface 230;

[0073] The input processing module 254 is used to detect and translate one or more user inputs or interactions from one or more input devices 232.

[0074] In some embodiments, the tag extraction device provided in this application can be implemented in software. Figure 2 A tag extraction device 255 stored in memory 250 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a video receiving module 2551, an image extraction module 2552, an image recognition module 2553, a candidate tag extraction module 2554, and a tag determination module 2555. These modules are logically connected and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0075] In other embodiments, the tag extraction device provided in this application can be implemented in hardware. As an example, the tag extraction device provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the tag extraction method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0076] For example, an embodiment of this application provides a server, including:

[0077] Memory, used to store executable tag extraction instructions;

[0078] The processor is configured to execute executable tag extraction instructions stored in the memory to implement the tag extraction method provided in the embodiments of this application.

[0079] The tag extraction method provided in this application will be described in conjunction with exemplary applications and implementations of the server provided in the embodiments of this application.

[0080] See Figure 4 , Figure 4 This is an optional flowchart illustrating the tag extraction method provided in the embodiments of this application. Figure 1 , will combine Figure 4 The steps shown are explained.

[0081] S101. Receive the video to be extracted and the title text corresponding to the video to be extracted.

[0082] This application embodiment is implemented in a scenario where video tags are extracted and used to annotate the videos. For example, tags are extracted from user-uploaded short videos to annotate them, or tags are extracted from video clips extracted from movies to annotate them. During operation, the server monitors whether any terminals are uploading videos. When a video upload is detected, the server receives the uploaded video and uses it as the video to be extracted. That is, the server receives the video to be extracted sent by the terminal via the network. Since users often add titles and other content to their videos to clearly express the theme, the terminal uploads the title text of the video to be extracted along with the video itself. Therefore, the server receives both the video and its corresponding title text in addition to the video itself.

[0083] It is understood that the video to be extracted refers to any video uploaded by the user. The video to be extracted can be a short video recorded by the user, a video created by the user through editing software or other applications, or a movie video, TV series video, variety show video, etc. on the Internet. This application embodiment does not make specific limitations here.

[0084] It is understood that the title text of the video to be extracted can be a title edited by the user for the video to be extracted, a title automatically generated by the machine based on the content of the video to be extracted, or a title obtained through other forms. This application embodiment does not limit this.

[0085] S102. Extract the image to be identified from the video to be extracted; the image to be identified is the cover image of the video to be extracted or a video frame of the video to be extracted.

[0086] In practical applications, both the video's visuals and title encapsulate its content, and video tags are determined based on this content. Therefore, tagging a video requires analyzing its visuals. After receiving the video to be extracted, the server needs to extract certain visual content. This content can be the video's cover image or individual video frames. The server uses this extracted content as the image to be identified, which is then used to determine the video's tag.

[0087] Understandably, the cover image of the video to be extracted refers to a still frame of the video before it is played. The cover image often reveals the essence of the video, such as the main character or the main event of the video. Therefore, the cover image can be used to determine the corresponding tags of the video to be extracted.

[0088] Similarly, the video content of the video to be extracted is presented in video frames. Therefore, in addition to the cover image, the server can also extract one or more video frames from the video to be extracted, and use each extracted video frame as the image to be identified, thereby using the video frames to determine the tag corresponding to the video to be extracted.

[0089] In some embodiments of this application, the server may also use the cover image of the video to be extracted, as well as one or more video frames from the video to be extracted, as images to be identified, thereby enriching the image content of the images to be identified and making the tags extracted from the video to be extracted more accurate.

[0090] It should be noted that the cover image of the video to be extracted can be extracted by the user from the video frames of the short video they recorded. For example, the user can extract the most exciting video frame from the short video as the cover image. The cover image can also be selected by the user from their favorite still image. For example, the user can use a photo of a scenic spot as the cover image of a video clip about a scenic spot they created. The cover image can also be an existing movie poster, a still from a TV series, or a promotional photo from a variety show. Of course, in some other embodiments of this application, the cover image can also be in other forms, which are not limited here.

[0091] Understandably, the server can extract one or more video frames from the video to be extracted based on a preset time interval, such as 2 seconds or 1 second. The server can also analyze all video frames of the video to be extracted, identify key video frames from the video to be extracted, and extract the key video frames. For example, the server can identify the objects contained in each video frame in the video to be extracted, and select the video frame containing the preset key object or the video frame containing the most objects as the key video frame.

[0092] S103. Recognize the image to be recognized and obtain the image category label corresponding to the image to be recognized.

[0093] After receiving the image to be identified, the server first classifies and identifies the objects contained in the image to determine their categories. Then, the categories of all contained objects are used as the image category label corresponding to the image to be identified. In other words, the image category label represents the objects contained in the image to be identified.

[0094] It should be noted that the image category label can be the name or category of a dynamic object, such as an animal, a car, or a pedestrian, or it can be the name of a static object, such as a mountain, a building, or a car parked on the roadside. This application embodiment does not limit the specific category.

[0095] It is understood that, in some embodiments of this application, the server can recognize the image to be recognized using a preset image recognition model. In this case, after obtaining the image to be recognized, the server will first acquire the preset image recognition model, then input the image to be recognized into the preset image recognition model, and use the category output by the preset image recognition model as the image category label.

[0096] Furthermore, the preset image recognition model is pre-trained. That is, before starting the label extraction process, the server uses a large number of video covers and video frames of specific categories as training data to train the preset image recognition model, so that it is convenient to use the preset image recognition model for classification and recognition in the future.

[0097] S104. Extract words from the title text to obtain at least one candidate label corresponding to the title text.

[0098] The server also needs to segment the title text to obtain the segmentation results, and then extract words from the segmented results to extract all nouns in the title text. These extracted nouns are then used as candidate tags. Since there may be more than one noun in the title text, the server will extract at least one candidate tag. Therefore, at least one candidate tag is a noun in the title text.

[0099] It should be noted that at least one candidate label refers to the nouns or words initially identified as potential labels for the video to be extracted. However, although these nouns or words are extracted from the title text and describe key information about the video to be extracted, in order to ensure a higher relevance between the labels of the video and its content, it is necessary to further filter the most suitable candidate labels from at least one candidate label using the image category labels of the image to be identified, and then use these as the labels for the video to be extracted.

[0100] It is understood that in some embodiments of this application, the server can use a word segmentation tool to segment the title text, thereby obtaining segmentation results. The server can perform part-of-speech tagging on the segmentation results, extracting all nouns from the segmentation results to obtain at least one candidate tag. In other embodiments of this application, while extracting all nouns from the title text, the server can also match the segmentation results with an entity dictionary, extracting all words that match the entity dictionary, and using all words that match the entity dictionary, along with all extracted nouns, to form at least one candidate tag.

[0101] S105. Based on the image category label, select the label corresponding to the video to be extracted from at least one candidate label.

[0102] After identifying the image category label of the image to be identified, the server determines the category of objects contained in the content of the video to be extracted, thus clarifying which objects the video focuses on. Next, the server calculates the word relevance between the image category label and each of the at least one candidate label. Then, using the word similarity between the image category label and each candidate label, the server selects candidate labels with high relevance to the image category label from the at least one candidate label list. These selected candidate labels are used as the corresponding labels for the video to be extracted. In this way, the server completes the extraction of labels closely related to the video from the title text based on the video's content, facilitating the use of these labels for video classification and other purposes.

[0103] It is understood that in some embodiments of this application, when the image to be identified is the cover image of the video to be extracted, the corresponding image category label is the category label corresponding to the cover image, that is, the cover category label. At this time, the server can select the label corresponding to the video to be extracted from at least one candidate label based on the cover category label.

[0104] In some other embodiments of this application, the image to be identified can be a video frame of the video to be extracted. In this case, the image category label is the category label corresponding to the video frame, that is, the video category label. At this time, the server can select the label corresponding to the video to be extracted from at least one candidate label based on the video category label.

[0105] In this embodiment, the server can extract the image to be identified from the received video to be extracted and identify the image category label corresponding to the image to be identified, thereby obtaining the object contained in the image to be identified. Then, it extracts nouns from the title text of the received video to be extracted, that is, initially determines at least one candidate label. Finally, based on the image category label, it further selects the label corresponding to the video to be extracted from at least one candidate label. In this way, it realizes the automatic extraction of labels from the title with less information by combining the screen content of the video to be extracted, thereby improving the efficiency of video label extraction.

[0106] See Figure 5 , Figure 5 This is an optional flowchart illustrating the tag extraction method provided in the embodiments of this application. Figure 2 In some embodiments of this application, when the image to be identified is the cover image of the video to be extracted, the image category label is the cover category label corresponding to the cover image. In this case, the server selects the label corresponding to the video to be extracted from at least one candidate label based on the image category label. The specific implementation process of S105 may include: S1051-S1053, as follows:

[0107] S1051. Calculate the similarity between the cover category label and each candidate label in at least one candidate label to obtain the first word similarity corresponding to each candidate label.

[0108] When the image to be identified is the cover image of the video to be extracted, the image category label obtained by the server is the cover category label. At this time, the server will use the cover category label to pair with each candidate label in at least one candidate label to obtain the word pair corresponding to each candidate label. Then, the server will calculate the word similarity between the cover category label and the candidate label in each word pair, thereby obtaining the word similarity of each word pair, which is the word similarity between each candidate label and the cover category label. This word similarity is recorded as the first word similarity.

[0109] It is understood that in this embodiment of the application, the server converts the cover category label into a semantic vector and converts each candidate label into a semantic vector. Then, based on calculation methods such as cosine similarity and Euclidean distance, the server calculates the vector similarity between the semantic vector corresponding to the cover category label and the semantic vector corresponding to each candidate label. The calculated vector similarity is the first word similarity.

[0110] In some embodiments of this application, the server may convert the cover category label and each candidate label into 50-dimensional vectors respectively. For example, word2vec can be used to convert the cover category label into a 50-dimensional vector and each candidate label into a 50-dimensional vector.

[0111] S1052. Compare the first word similarity corresponding to each candidate label with a preset similarity threshold to obtain at least one comparison result corresponding to at least one candidate label.

[0112] After calculating the similarity between the cover category label and each candidate label, and obtaining the first word similarity for each candidate label, the server acquires a preset similarity threshold. Next, the server compares the first word similarity threshold for each candidate label with the acquired preset similarity threshold to determine the relationship between the first word similarity and the preset similarity threshold, thus obtaining a comparison result for each candidate label. Once at least one first word similarity for at least one candidate label has been compared with the preset similarity threshold, the server obtains at least one comparison result. Each comparison result corresponds one-to-one with at least one candidate label.

[0113] Understandably, the comparison result indicates whether the first word similarity is greater than the preset similarity threshold. At this time, the server compares the first word similarity of each candidate tag with the preset similarity threshold, which is essentially determining whether the first word similarity of each candidate tag is greater than the preset similarity threshold.

[0114] In this embodiment, the preset similarity threshold can be set according to the actual situation. For example, the preset similarity threshold can be set to 0.75 or 0.6, etc. This embodiment does not limit the setting.

[0115] S1053. Based on at least one comparison result, select the tag corresponding to the video to be extracted from at least one candidate tag.

[0116] After receiving at least one comparison result, the server parses it to determine the relationship between the first word similarity score and a preset similarity threshold for each candidate tag. This relationship indicates the relevance between the candidate tag and the cover category tag. A high relevance between a candidate tag and the cover category tag indicates that the content of the video to be extracted is similar to the content described by that candidate tag; that is, the content closely matches the video. Using this candidate tag as the corresponding tag for the video allows the server to accurately determine the tag for the video even with limited information in the title text, thus improving the efficiency of video tag extraction.

[0117] In this embodiment, the server can first calculate the first word similarity between each candidate tag and the cover category tag, then compare the first word similarity with a preset similarity threshold to determine the relevance between each candidate tag and the cover category tag, and then select the candidate tags with higher relevance to the cover category tag as tags for the video to be extracted, thereby improving the accuracy of tag extraction for the video to be extracted and thus improving the efficiency of tag extraction for the video.

[0118] In some embodiments of this application, the specific implementation process of selecting the tag corresponding to the video to be extracted from at least one candidate tag based on at least one comparison result, i.e., S1053, may include: S1053a-S1053b, as follows:

[0119] S1053a. From at least one comparison result, select the target comparison result that indicates that the similarity of the first word is greater than a preset similarity threshold.

[0120] The server reads at least one comparison result to determine whether each result indicates a first similarity greater than or less than a preset similarity threshold. Then, the server selects the comparison results indicating a first similarity greater than the preset similarity threshold and records these results as the target comparison results.

[0121] S1053b: From at least one candidate label, select the label corresponding to the target comparison result of the video to be extracted.

[0122] Since there is a one-to-one correspondence between at least one candidate label and at least one comparison result—meaning each candidate label has a corresponding comparison result—there is a direct correspondence between candidate labels and comparison results. Therefore, after selecting the target comparison result, the server can select the candidate labels corresponding to the target comparison result based on the direct correspondence between candidate labels and comparison results, and use the selected candidate labels as target candidate labels. Because the target candidate label corresponds to a target comparison result indicating that the first word similarity is greater than a preset similarity threshold—that is, the first word similarity of the target candidate label is greater than the preset similarity threshold—the target candidate label has a high correlation with the cover category label. Therefore, the server will use the target candidate label as the label for the video to be extracted, thereby achieving label extraction.

[0123] For example, if there are three candidate tags among at least one candidate tag, and the first word similarity of these three candidate tags is 0.1, 0.5 and 0.8 respectively, when the preset similarity threshold is 0.7, the server obtains the relationship between the first word similarity of these three candidate tags and 0.7, that is, it obtains the comparison results corresponding to these three candidate tags respectively, and then selects the candidate tag corresponding to 0.8 as the tag corresponding to the video to be extracted based on the comparison results.

[0124] In this embodiment, the server first selects the target comparison result from at least one comparison result, and then uses the candidate tags corresponding to the target comparison result as the final tags of the video to be extracted. This achieves the goal of selecting the tags corresponding to the video to be extracted based on the comparison results, i.e., the relevance between the candidate tags and the cover category tags, thereby improving the efficiency of tag extraction.

[0125] See Figure 6 , Figure 6 This is an optional flowchart illustrating the tag extraction method provided in the embodiments of this application. Figure 3In some embodiments of this application, when the image to be identified is a video frame of the video to be extracted, the image category label is the video category label corresponding to the video frame. In this case, the server selects the label corresponding to the video to be extracted from at least one candidate label based on the image category label. The specific implementation process of S105 may include: S1054-S1055, as follows:

[0126] S1054. For each candidate label in at least one candidate label, calculate the similarity with the video category label corresponding to the video frame to obtain at least one second word similarity corresponding to at least one candidate label.

[0127] When the image to be identified is a video frame from the video to be extracted, the image category label identified by the server is essentially the video category label corresponding to the video frame. In this case, when the server selects the label corresponding to the video to be extracted from at least one candidate label, it first forms a word pair with each candidate label and the video category label, thus obtaining a word pair corresponding to each candidate label. Then, it calculates the word similarity between the candidate label and the video category label in each word pair, and records this calculated word similarity as the second word similarity. When the similarity calculation for each word pair corresponding to each candidate label is completed, at least one second word similarity is obtained, where at least one second word similarity corresponds one-to-one with at least one candidate label.

[0128] In some embodiments of this application, more than one video frame may be extracted from the video to be extracted. However, for each video frame, the server identifies a video category label, thus obtaining multiple video category labels. The server then combines these multiple video category labels with at least one candidate label in pairs, resulting in multiple word pairs corresponding to each candidate label. This leads to multiple second word similarities for each candidate label, which are then used to select the label corresponding to the video to be extracted from the at least one candidate label.

[0129] S1055. Based on at least one second word similarity, select the tag corresponding to the video to be extracted from at least one candidate tag.

[0130] After obtaining at least one second word similarity, the server compares the similarity of each second word in the at least one second word similarity to determine the size relationship of each second word similarity. Then, based on the size relationship of each second word similarity, it selects some candidate tags that are most relevant to the video category tag from at least one candidate tag and uses the selected candidate tags as the tags corresponding to the video to be extracted.

[0131] It should be noted that when there is more than one video frame, the server compares the similarity of each second word corresponding to all candidate tags, and then selects the tag corresponding to the video to be extracted from at least one candidate tag based on the relationship between the similarity of each second word of all candidate tags.

[0132] In this embodiment, the server can calculate the similarity between each candidate tag and the video category tag, thereby obtaining at least one second word similarity corresponding to at least one candidate tag. Then, by using at least one second word similarity, the server determines the relevance between at least one candidate tag and the video category tag, thereby selecting the tag corresponding to the video to be extracted. This improves the accuracy of tag extraction from the video to be extracted and increases the efficiency of tag extraction from the video.

[0133] In some embodiments of this application, the specific implementation process of selecting the tag corresponding to the video to be extracted from at least one candidate tag based on at least one second word similarity, i.e., S1055, may include: S1055a-S1055b, as follows:

[0134] S1055a. From at least one second word similarity, select the largest preset number of second word similarities as the target word similarity.

[0135] When selecting tags for videos based on second-word similarity, the server sorts at least one second-word similarity in descending order to determine the order of each second-word similarity among all second-word similarities. Then, the server extracts a predetermined number of second-word similarities sequentially from beginning to end, and these extracted similarities are recorded as the target similarity.

[0136] It is understood that the preset quantity can be set according to the actual situation. For example, the preset quantity can be set to 1, 3, or other values. This application embodiment does not limit this.

[0137] S1055b: From at least one candidate tag, select the tag corresponding to the video to be extracted that corresponds to the similarity of the target word.

[0138] After obtaining the target word similarity, the server will select the candidate label corresponding to the target word similarity from at least one candidate label based on the correspondence between the second word similarity and the candidate label, and use the selected candidate label as the label corresponding to the video to be extracted.

[0139] For example, when at least one second word similarity is 0.1, 0.5, and 0.8 respectively, the server sorts these second word similarities from largest to smallest. When the preset number is 2, the server will use 0.8 and 0.5 as target word similarities, and then select the candidate tags corresponding to 0.8 and 0.5 as tags for the video to be extracted.

[0140] In this embodiment, the server first sorts at least one second word similarity in descending order, then extracts the largest preset number of second word similarities, and uses the candidate tags corresponding to these second similarities as tags for the video to be extracted. Thus, the server can extract some candidate tags with the highest relevance to the video category tag from at least one candidate tag as tags, thereby improving the accuracy of tag extraction for the video to be extracted and improving the efficiency of video tag extraction.

[0141] See Figure 7 , Figure 7 This is an optional flowchart illustrating the tag extraction method provided in the embodiments of this application. Figure 4 In some embodiments of this application, word extraction is performed on the title text to obtain at least one candidate tag corresponding to the title text. The specific implementation process of S104 may include: S1041-S1045, as follows:

[0142] S1041. Perform word segmentation on the title text to obtain multiple word segmentation results corresponding to the title text.

[0143] When the server extracts words from the title text, it first uses a word segmentation tool to divide the title text into multiple individual words. These segmented words are the word segmentation results corresponding to the title text.

[0144] S1042. Select the first candidate word that matches the preset entity dictionary from multiple word segmentation results.

[0145] After receiving multiple word segmentation results, the server immediately retrieves a preset entity dictionary. It then matches these preset entity words with each of the multiple segmentation results to determine which segmentation results match entities in the preset entity dictionary—that is, which segmentation results hit the preset entity dictionary. The server selects the segmentation results that match the preset entity dictionary and uses these as the first candidate segmentation words.

[0146] Understandably, the pre-defined entity dictionary contains multiple pre-collected entity words, such as "video" and "animal." The server's determination of which segmentation results match the pre-defined entity dictionary essentially involves selecting the pre-collected entity words from these multiple segmentation results.

[0147] S1043. Perform part-of-speech tagging on one or more other word segmentation results besides the first candidate word segmentation result, and obtain one or more tagging results of one or more other word segmentation results.

[0148] After obtaining the first candidate word segment, the server will remove it from the multiple segmentation results, treating all remaining segmentation results as other segmentation results. Since there may be more than one segmentation result remaining after removing the first candidate word, the server will obtain one or more other segmentation results. Next, the server will call a part-of-speech tagging tool to perform part-of-speech tagging on each other segmentation result, obtaining the tagging result corresponding to each other segmentation result. When the server has completed part-of-speech tagging for one or more other segmentation results, it will obtain one or more tagging results.

[0149] It should be noted that one or more tagging results represent the part of speech of one or more other word segmentation results. That is, the server determines the tagging result corresponding to each other word segmentation result, which is essentially determining the part of speech of each other word segmentation result, that is, determining whether each other word segmentation result belongs to a noun, verb, or adjective, etc.

[0150] S1044. Based on one or more annotation results, select a second candidate word from one or more other word segmentation results.

[0151] After obtaining one or more tagging results, the server has determined the part of speech of each of the other segmentation results. At this point, the server will select the other segmentation results that belong to nouns from the one or more other segmentation results and use these selected other segmentation results as the second candidate segmentation.

[0152] S1045. Using the first and second candidate word segments, integrate them into at least one candidate tag corresponding to the title text.

[0153] After obtaining the second candidate word segment, the server will integrate the first and second candidate words, that is, use the first and second candidate words to form a set. Each word included in the set is a candidate tag. In this way, the server obtains at least one candidate tag.

[0154] For example, this application provides a process illustration for extracting at least one candidate label, see [link to illustration]. Figure 8 The server segments the title text: "How close can you get to the animals in the wildlife park?" into multiple segments (8-2), resulting in the following segments: wildlife park (8-21), inside (8-22), in the end (8-23), can (8-24), away (8-25), animals (8-26), have (8-27), and how close (8-28). Then, using a pre-defined entity dictionary, the server selects "animals" (8-26) from these segments as the first candidate segment (8-3). Next, from the remaining segments (8-4) excluding "animals" (8-26), the server selects "wildlife park" (8-21) as the second candidate segment (8-5). Finally, the first and second candidate segments (8-3 and 8-5) are integrated to obtain at least one candidate tag (8-6). These multiple segments (8-6) contain both "wildlife park" (8-21) and "animals" (8-26).

[0155] In this embodiment, the server first uses a preset entity dictionary to select entity words as first candidate words from multiple segmentation results of the title text. Then, from one or more other segmentation results besides the first candidate words, it selects nouns as second candidate words. Finally, it combines the first and second candidate words to obtain at least one candidate tag. In this way, the server achieves the process of extracting candidate tags from the title text without omission.

[0156] In some embodiments of this application, the specific implementation process of selecting a second candidate word from one or more other word segmentation results based on one or more annotation results, i.e., S1044, may include: S1044a-S1044b, as follows:

[0157] S1044a. Select the noun tagging results that represent nouns from one or more tagging results.

[0158] S1044b: Among one or more other word segmentation results, the other word segmentation results corresponding to the noun tagging results are determined as the second candidate word segmentation.

[0159] When selecting the second candidate word segment, the server first reads each of the one or more tagging results to determine whether each tag represents a noun, a verb, or another part of speech. Next, the server extracts the noun tagging results from the one or more tagging results. Then, based on the correspondence between the tagging results and other word segmentation results, the server extracts the other word segmentation results corresponding to the noun tagging results from the one or more other word segmentation results. These extracted other word segmentation results are the second candidate word segments.

[0160] In this embodiment, the server can first select a noun tagging result from one or more tagging results, and then select a second candidate word segment corresponding to the noun tagging result from one or more other word segmentation results. In this way, the server can select a second candidate word segment from one or more other word segmentation results based on its part of speech.

[0161] In some embodiments of this application, the process of recognizing the image to be recognized and obtaining the image category label corresponding to the image to be recognized, i.e., S103, may include: S1031-S1032, as follows:

[0162] S1031. Using the obtained preset image recognition model, extract the image features of the image to be recognized.

[0163] In this embodiment, the server first obtains a trained preset image recognition model, then inputs the image to be recognized into the preset image recognition model, and the preset image recognition model extracts the image features of the image to be recognized.

[0164] It is understandable that image features can be in vector form; for example, the server extracts a vector with the same dimension as the number of image categories from a pre-defined image recognition model. Image features can also be in feature map form; for example, the server extracts a feature map that best represents the image to be recognized from a pre-defined image recognition model.

[0165] It should be noted that the preset image recognition model can be a pre-trained deep learning model, such as a convolutional neural network model, etc., and this application embodiment does not limit it.

[0166] For example, this application provides example diagrams of a preset image recognition model. (See also...) Figure 9 As shown, the preset image recognition model can have 22 layers, namely an input layer with a size of 224×244×3 and two convolutional layers with a size of 224×244×64. Figure 9 Midpoint cloud filling part), pooling layer ( Figure 9 The layers consist of: a centrally located oblique grid filled area, two convolutional layers and pooling layers with dimensions of 112×112×128, three convolutional layers and pooling layers with dimensions of 56×56×256, three convolutional layers and pooling layers with dimensions of 28×28×512, three convolutional layers and pooling layers with dimensions of 14×14×512, and two fully connected layers with dimensions of 1×1×4096. Figure 9 (The middle grid filling part), a 1×1×1000 fully connected layer, thus, a 1×1×1000 vector can be obtained, which is the image feature.

[0167] S1032. Recognize the image features to obtain the image category label corresponding to the image to be recognized.

[0168] After obtaining image features, the server identifies and classifies them, thus determining the category label to which each image feature belongs. Since the image features correspond to the image to be identified, the category label to which the server determines the image features is the image category label corresponding to the image to be identified.

[0169] In this embodiment, the server first calls a preset image recognition model to extract image features from the image to be recognized, then classifies and recognizes the image features to determine the image category label. In this way, the server completes the recognition process for the image to be recognized.

[0170] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0171] The embodiments of this application are implemented in the scenario of determining tags for short videos created by users.

[0172] Before the backend (server) receives and tags the short videos recorded by users, it trains a convolutional neural network model, namely the VGGNet model (pre-set image recognition model), to facilitate the subsequent classification and recognition of images in the short videos.

[0173] When training the VGGNet model, the backend first acquires the ImageNet dataset (a large visualization database for user data recognition software research, containing over 14 million manually labeled images, encompassing over 20,000 categories, each with hundreds of images). Then, the backend translates the English labels in the ImageNet dataset into Chinese labels. Because VGGNet has a large number of parameters, training directly on a small dataset would result in poor model performance. Therefore, the backend first trains VGGNet using the translated ImageNet dataset to obtain a preliminary trained model. Then, it fine-tunes the preliminary model using video frames or video covers corresponding to the label types required in the real-world scenario, thus constructing a VGGNet model suitable for this scenario. The video frames or video covers corresponding to the label types required in the real-world scenario are manually collected and labeled.

[0174] Next, the backend will receive the short videos created by users (videos to be extracted), and then use the trained VGGNet model to classify and recognize the cover images of the short videos to obtain the corresponding tags (cover category tags) of the short video covers.

[0175] For example, this application provides example diagrams showing the classification and recognition of short video cover images using a trained VGGNet model in a real-world application scenario. Figure 10 As shown, the backend inputs cover 10-1 into VGGNet model 10-2. Through convolution and pooling in the convolutional and pooling layers of VGGNet model 10-2, features are obtained. Figure 10-2 2, thereby defining the features Figure 10-2 2. Input to the fully connected layer 10-23 for dimensionality reduction, obtain the score of each category 10-24, and then pass through the classifier 10-25 to obtain the final category "tiger" for the cover 10-1.

[0176] After obtaining the tags corresponding to the short video's cover image, the backend will also perform word segmentation on the short video's title (title text), and then extract all nouns (at least one candidate tag) based on a pre-collected entity dictionary (preset entity dictionary) and part-of-speech tagging tools. For example, from Figure 1 The titles of the short videos shown (1-2: Part 2 is here! How close can you get to the animals at the wildlife park? Watch the video and you'll find out!) extract "wildlife park," "animals," and "video." Then, the backend uses word2vec to convert "tiger," "wildlife park," "animals," and "video" into 50-dimensional vectors V. The cosine similarity (first word similarity) between these vectors is then calculated, resulting in (V(tiger), V(wildlife park)) = 0.72, (V(tiger), V(animals)) = 0.85, and (V(tiger), V(video)) = 0.3. Next, the backend sets the threshold K (preset similarity threshold) to 0.7, thus allowing the tags #wildlifepark# and #animals# to be selected from "wildlife park," "animals," and "video."

[0177] It should be noted that, in addition to VGGNet, the backend can also use similar training methods and data to train AlexNet, and then use the trained AlexNet to classify and recognize the cover or video frame of the short video to obtain the label corresponding to the cover or the label corresponding to the video frame. Then, based on the obtained labels, the labels of the short video are selected from the nouns extracted from the title of the short video.

[0178] The above method allows for the first extraction of nouns from the title, yielding candidate nouns that can become tags for the short video. Then, by using the categories obtained from recognizing the cover of the short video, tags for the short video can be further selected from these nouns. This achieves the goal of extracting tags from the title text by combining the image content of the short video, thus improving the efficiency of tag extraction for short videos.

[0179] The following continues to describe an exemplary structure of the tag extraction device 255 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 3 As shown, the software module stored in the tag extraction device 255 in the memory 250 may include:

[0180] Video receiving module 2551 is used to receive the video to be extracted and the title text corresponding to the video to be extracted;

[0181] Image extraction module 2552 is used to extract an image to be identified from the video to be extracted; the image to be identified is the cover image of the video to be extracted or a video frame of the video to be extracted;

[0182] Image recognition module 2553 is used to recognize the image to be recognized and obtain the image category label corresponding to the image to be recognized; the image category label represents the object contained in the image to be recognized;

[0183] The candidate tag extraction module 2554 is used to extract words from the title text to obtain at least one candidate tag corresponding to the title text; the at least one candidate tag is a noun word in the title text.

[0184] The label determination module 2555 is used to select the label corresponding to the video to be extracted from the at least one candidate label based on the image category label.

[0185] In some embodiments of this application, when the image to be identified is the cover image of the video to be extracted, the image category label is the cover category label corresponding to the cover image; the label determination module 2555 is further configured to calculate the similarity between the cover category label and each of the at least one candidate labels to obtain a first word similarity corresponding to each candidate label; compare the first word similarity corresponding to each candidate label with a preset similarity threshold to obtain at least one comparison result corresponding to the at least one candidate label; the comparison result indicates whether the first word similarity is greater than the preset similarity threshold; and select the label corresponding to the video to be extracted from the at least one candidate label according to the at least one comparison result.

[0186] In some embodiments of this application, the tag determination module 2555 is further configured to select a target comparison result from the at least one comparison result, which indicates that the similarity of the first word is greater than the preset similarity threshold; and select a tag corresponding to the video to be extracted from the at least one candidate tag.

[0187] In some embodiments of this application, when the image to be identified is a video frame of the video to be extracted, the image category label is the video category label corresponding to the video frame; the label determination module 2555 is further configured to perform similarity calculation on each of the at least one candidate label and the video category label corresponding to the video frame, respectively, to obtain at least one second word similarity corresponding to the at least one candidate label; and select the label corresponding to the video to be extracted from the at least one candidate label based on the at least one second word similarity.

[0188] In some embodiments of this application, the tag determination module 2555 is further configured to select the largest preset number of second word similarities from the at least one second word similarity as the target word similarity; and select the tag corresponding to the video to be extracted from the at least one candidate tag that corresponds to the target word similarity.

[0189] In some embodiments of this application, the candidate tag extraction module 2554 is further configured to perform word segmentation on the title text to obtain multiple word segmentation results corresponding to the title text; select a first candidate word segmentation that matches a preset entity dictionary from the multiple word segmentation results; perform part-of-speech tagging on one or more other word segmentation results other than the first candidate word segmentation to obtain one or more tagging results of the one or more other word segmentation results; the one or more tagging results characterize the part of speech of the one or more other word segmentation results; select a second candidate word segmentation from the one or more other word segmentation results according to the one or more tagging results; and integrate the first candidate word segmentation and the second candidate word segmentation to form the at least one candidate tag corresponding to the title text.

[0190] In some embodiments of this application, the candidate tag extraction module 2554 is further configured to select a noun tagging result representing a noun from the one or more tagging results; and to determine the other word segmentation results corresponding to the noun tagging result from the one or more other word segmentation results as the second candidate word segmentation.

[0191] In some embodiments of this application, the image recognition module 2553 is further configured to extract image features of the image to be recognized using the acquired preset image recognition model; and to recognize the image features to obtain the image category label corresponding to the image to be recognized.

[0192] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the tag extraction method described above in this application.

[0193] This application provides a computer-readable storage medium storing executable instructions, including executable tag extraction instructions. When the executable tag extraction instructions are executed by a processor, the processor will execute the tag extraction method provided in this application. For example, ... Figure 4 , Figure 5 , Figure 6 as well as Figure 7 The method shown.

[0194] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0195] In some embodiments, executable tag extraction instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0196] As an example, executable tag extraction instructions may, but do not necessarily, correspond to files in the file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborative files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0197] As an example, executable tag extraction instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0198] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A tag extraction method, characterized in that, include: Receive the video to be extracted and the title text corresponding to the video to be extracted; Extract the image to be identified from the video to be extracted; The image to be identified is either the cover image of the video to be extracted or a video frame of the video to be extracted. The image to be identified is identified to obtain the image category label corresponding to the image to be identified; the image category label represents the object contained in the image to be identified; The title text is segmented into words to obtain multiple segmentation results. From the multiple segmentation results, entity words that match a preset entity dictionary are selected as first candidate words. Part-of-speech tagging is performed on the other segmentation results besides the first candidate words to obtain the tagging results of the other segmentation results. Based on the tagging results, noun words are selected from the other segmentation results as second candidate words. The first candidate words and the second candidate words are summed to obtain at least one candidate tag corresponding to the title text. Calculate the similarity between the image category label and each of the at least one candidate label; Based on the similarity between the image category label and each candidate label, the label corresponding to the video to be extracted is selected from the at least one candidate label.

2. The method according to claim 1, characterized in that, When the image to be identified is the cover image of the video to be extracted, the image category label is the cover category label corresponding to the cover image; The calculation of the similarity between the image category label and each of the at least one candidate labels includes: The similarity between the cover category tag and each of the at least one candidate tags is calculated to obtain the first word similarity corresponding to each candidate tag; The step of selecting the tag corresponding to the video to be extracted from the at least one candidate tag based on the similarity between the image category tag and each candidate tag includes: The similarity of the first word corresponding to each candidate label is compared with a preset similarity threshold to obtain at least one comparison result corresponding to the at least one candidate label; the comparison result indicates whether the similarity of the first word is greater than the preset similarity threshold. Based on the at least one comparison result, the tag corresponding to the video to be extracted is selected from the at least one candidate tag.

3. The method according to claim 2, characterized in that, The step of selecting the tag corresponding to the video to be extracted from the at least one candidate tag based on the at least one comparison result includes: From the at least one comparison result, select the target comparison result that indicates that the similarity of the first word is greater than the preset similarity threshold; From the at least one candidate tag, select the tag corresponding to the video to be extracted that corresponds to the target comparison result.

4. The method according to claim 1, characterized in that, When the image to be identified is a video frame of the video to be extracted, the image category label is the video category label corresponding to the video frame; The calculation of the similarity between the image category label and each of the at least one candidate labels includes: For each candidate tag in the at least one candidate tag, a similarity calculation is performed with the video category tag corresponding to the video frame to obtain at least one second word similarity corresponding to the at least one candidate tag; The step of selecting the tag corresponding to the video to be extracted from the at least one candidate tag based on the similarity between the image category tag and each candidate tag includes: Based on the similarity of at least one second word, the tag corresponding to the video to be extracted is selected from the at least one candidate tag.

5. The method according to claim 4, characterized in that, The step of selecting the tag corresponding to the video to be extracted from the at least one candidate tag based on the at least one second word similarity includes: From the at least one second word similarity, select the largest preset number of second word similarities as the target word similarity; From the at least one candidate tag, select the tag corresponding to the video to be extracted that corresponds to the similarity of the target word.

6. The method according to claim 1, characterized in that, The step of selecting noun words from the other word segmentation results based on the annotation results includes: From the annotation results, select the noun annotation results that represent nouns; The other word segmentation results that correspond to the noun tagging results are identified as the noun words.

7. The method according to any one of claims 1 to 6, characterized in that, The step of identifying the image to be identified to obtain the image category label corresponding to the image to be identified includes: Using the acquired preset image recognition model, the image features of the image to be recognized are extracted; The image features are identified to obtain the image category label corresponding to the image to be identified.

8. A server, characterized in that, include: Memory, used to store executable tag extraction instructions; A processor, when executing executable tag extraction instructions stored in the memory, implements the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, It stores executable tag extraction instructions, which, when executed by a processor, implement the method described in any one of claims 1 to 7.

Images