Method, apparatus and storage medium for video recognition

Abstract

According to an embodiment of the present disclosure, a method, apparatus, device and storage medium for video recognition are provided. The method comprises: classifying a video clip into at least one classification label in a target classification label set related to a target scene by using a target video recognition model related to the target scene, the target video recognition model being trained on the target classification label set; and performing a post-processing operation for the target scene on each classification label in the at least one classification label based on a type in which each classification label in the target classification label set is divided in the target scene, to obtain at least one target classification label of the video clip in the target scene. Thus, an accurate classification label can be determined for the video, and the accuracy of video recognition in a specific scene is improved.

Description

TECHNICAL FIELD

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and in particular, to a method, an apparatus, an electronic device, and a computer-readable storage medium for video recognition. BACKGROUND

[0002] With the rapid development of Internet technology, more and more applications and websites provide services related to video recognition. In many application scenarios related to videos, it can be necessary to recognize the content involved in the videos, including classifying the videos to label the categories of the videos. The recognition result of the video categories can be used for subsequent video-related processing, including classified storage of the videos, adding classification labels on the videos when editing the videos, and expanding more text description content using the classification labels when editing the videos. It can be understood that the accuracy of the recognition result will affect the subsequent video processing. People expect to be able to simply and conveniently obtain accurate video recognition results. SUMMARY

[0003] In a first aspect of the present disclosure, a method for video recognition is provided. The method comprises: classifying, by a target video recognition model related to a target scene, a video segment into at least one classification label in a target classification label set related to the target scene, the target video recognition model being trained on the target classification label set; and performing, for each classification label in the at least one classification label, a post-processing operation for the target scene based on a type in which the classification label is divided in the target scene, to obtain at least one target classification label of the video segment in the target scene.

[0004] In a second aspect of the present disclosure, an apparatus for video recognition is provided. The apparatus comprises: a video classification module configured to classify, by a target video recognition model related to a target scene, a video segment into at least one classification label in a target classification label set related to the target scene, the target video recognition model being trained on the target classification label set; and a label post-processing module configured to perform, for each classification label in the at least one classification label, a post-processing operation for the target scene based on a type in which the classification label is divided in the target scene, to obtain at least one target classification label of the video segment in the target scene.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device comprises at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. The instructions, when executed by the at least one processing unit, cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of the disclosure, a computer-readable storage medium is provided. The computer-readable storage medium has stored thereon a computer program, the computer program being executable by a processor to implement the method of the first aspect.

[0007] It should be understood that the description in the summary section is not intended to define key or essential features of embodiments of the disclosure or to limit the scope of the disclosure. Other features of the disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0008] The above and other features, advantages and aspects of embodiments of the present disclosure will become more apparent by describing in detail some embodiments thereof with reference to the attached drawings in which:

[0009] Figure 1 A schematic diagram illustrating an example environment in which embodiments of the present disclosure can be implemented is shown;

[0010] Figure 2 A schematic diagram illustrating an example architecture for video recognition according to some embodiments of the present disclosure is shown;

[0011] Figure 3 A schematic diagram illustrating one example of label set determination according to some embodiments of the present disclosure is shown;

[0012] Figure 4 A schematic diagram illustrating an example architecture for video recognition according to further embodiments of the present disclosure is shown;

[0013] Figure 5 A schematic diagram illustrating an example process for model training according to some embodiments of the present disclosure is shown;

[0014] Figure 6 A flow diagram illustrating a process for video recognition according to some embodiments of the present disclosure is shown;

[0015] Figure 7 A block diagram of an apparatus for video recognition according to some embodiments of the present disclosure is shown; and

[0016] Figure 8 A block diagram of an electronic device capable of implementing a number of embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0017] It can be understood that, before using the technical solutions disclosed by the embodiments of the present disclosure, the type of personal information involved in the present disclosure, the scope of use, the scenario of use, etc. should be informed to the user and the authorization of the user should be obtained in accordance with relevant laws and regulations.

[0018] For example, in response to receiving an active request of a user, a prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed by the user will require obtaining and using personal information of the user. Thus, the user can autonomously select whether to provide the personal information to the software or hardware, such as an electronic device, an application program, a server or a storage medium, performing the operation of the technical solution of the present disclosure according to the prompt information.

[0019] As an optional but non-limiting implementation, in response to receiving an active request of a user, the prompt information can be sent to the user in the form of a pop-up window, in which the prompt information can be presented in the form of text. In addition, the pop-up window can also carry selection controls for the user to select “agree” or “disagree” to provide personal information to the electronic device.

[0020] It can be understood that the above notification and obtaining user authorization process is only illustrative and does not limit the implementation of the present disclosure, and other ways that meet the relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0021] It can be understood that the data involved in the technical solution (including but not limited to the data itself, the obtaining or use of the data) should comply with the requirements of the relevant laws and regulations and the relevant provisions.

[0022] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms, and should not be interpreted as being limited to the embodiments described herein, rather, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes, and are not intended to limit the scope of protection of the present disclosure.

[0023] It should be noted that the titles of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and any type of embodiment can be included under any section / subsection. Furthermore, embodiments described in any section / subsection can be combined with any other embodiment described in the same section / subsection and / or in a different section / subsection in any manner.

[0024] In the description of embodiments of the present disclosure, the term “comprising” and similar terms are to be understood as open-ended, i.e., “including but not limited to”. The term “based on” is to be understood as “based at least in part on”. The term “one embodiment” or “the embodiment” is to be understood as “at least one embodiment”. The term “some embodiments” is to be understood as “at least some embodiments”. Other explicit and implicit definitions can also be included below. The terms “first”, “second”, etc. can refer to different or the same objects. Other explicit and implicit definitions can also be included below.

[0025] As used in this paper, the term "model" refers to a system that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. In this paper, "model" may also be referred to as a "machine learning model," a "machine learning network," or simply a "network," and these terms are used interchangeably. A model can also include different types of processing units or networks.

[0026] As used herein, a “unit,” “operation unit,” or “subunit” can consist of any suitable machine learning model or network. As used herein, a set of elements or similar expressions can include one or more such elements. For example, “a set of convolutional units” can include one or more convolutional units.

[0027] Machine learning typically comprises three phases: training, testing, and application (also known as inference). In the training phase, a given model is trained using a large amount of training data, iteratively updating its parameter values ​​until the model can consistently generate inferences that meet the expected goals from the training data. Through training, the model can be considered to have learned the relationship between inputs and outputs (also known as the input-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether it can provide the correct output, thus determining the model's performance. In the application phase, the model can be used to process actual inputs based on the trained parameter values ​​to determine the corresponding output.

[0028] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. For example... Figure 1 As shown, environment 100 may include electronic device 110.

[0029] Electronic device 110 can acquire video clip 102 and determine a category tag 112 that matches video clip 102. Category tag 112 may include one or more category tags. Electronic device 110 can acquire video clip 102 in any suitable manner. For example, electronic device 110 can capture video clip 102 in real time using a camera. Alternatively, electronic device 110 can also acquire video clip 102 stored locally on electronic device 110. Furthermore, electronic device 110 can also acquire video clip 102 from other electronic devices and / or the cloud. The video clip here can be any suitable video clip of any length, any format, including any subject, etc.

[0030] The electronic device 110 may, for example, utilize the video recognition model 120 to determine the classification label 112 that matches the video segment 102. The video recognition model 120 may, for example, include, without limitation, a Transformer model, a convolutional neural network (CNN), a recurrent neural network (RNN), a deep neural network (DNN), a multi-layer perceptron (MLP), or any other suitable model. The video recognition model 120 can be a model that is local to the electronic device 110 or a model that is installed on another electronic device (e.g., installed in a remote device).

[0031] The electronic device 110 may, for example, also utilize the label set 130 to determine the classification label 112 that matches the video segment 102. The label set 130 may, for example, include one or more label sets, each of which includes a large number of labels. The electronic device 110 can determine one or more labels from the label set 130 that match the video segment 102 and determine the one or more labels as the classification label 112 that matches the video segment 102. In cases where the classification label 112 is determined with the aid of the video recognition model 120, the electronic device 110 can utilize the video recognition model 120 to classify the video segment 102 into at least one classification label in the label set 130.

[0032] The electronic device 110 can include any computing system having computing capability, such as various computing devices / systems, end devices, server devices, and the like. The end device can be any type of mobile terminal, fixed terminal, or portable terminal including a mobile handset, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media player, a multimedia tablet, a palmtop computer, a portable gaming terminal, a VR / AR device, a Personal Communication System (PCS) device, a personal navigation device, a Personal Digital Assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a game device, or any combination thereof, including accessories and peripherals of these devices or any combination thereof.

[0033] The server device can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content distribution network, and big data and artificial intelligence platform.

[0034] It should be understood that the structure and function of the various elements in the environment 100 are described for illustrative purposes only, without implying any limitation on the scope of the present disclosure.

[0035] As briefly mentioned above, in a video-related application scenario, video recognition can be performed on an obtained video clip, and a subsequent operation can be performed based on the recognition result, including generating a text description (also referred to as a "script") that matches the video content. With the development of machine learning technology, it is currently proposed that a video recognition model can be trained to classify a video clip into at least one classification label in a label set.

[0036] Conventionally, a video recognition model is usually trained with a large amount of labeled training data. The labeled training data includes video clips in various scenarios and classification labels annotated for the video clips. The video recognition model trained in this way is referred to as a general video recognition model, which can classify a video clip into at least one general classification label in a general classification label set.

[0037] However, in many application scenarios, the classification requirements for video clips related to different scenarios are different. Therefore, when a video clip in a specified scenario needs to be recognized, the subject recognized by the general video model and the at least one general classification label determined can not be accurate enough. For example, if the specified scenario is a manicure product scenario, the video clip can present a pair of hands and a necklace. If the at least one general classification label determined by the general video recognition model can include "necklace". It can be seen that the general classification label determined by the electronic device using the general video recognition model is not accurate enough. The text description obtained by the electronic device based on such a general classification label can be "XX set is too good! The necklace is beautiful and affordable, very delicate, come and try it out! ". Such a text description is irrelevant to the specified manicure product scenario, i.e., the text description is also not accurate. Therefore, for the specified scenario, the accuracy of the at least one general classification label determined by the electronic device using the general video recognition model is poor.

[0038] In view of this, embodiments of the present disclosure propose an improved scheme for video recognition. According to various embodiments of the present disclosure, a target video recognition model associated with a target scene is used to classify video segments into at least one category label in a target category label set associated with the target scene. The target video recognition model is trained on the target category label set. Based on the type into which each category label in the target category label set is classified in the target scene, post-processing operations specific to the target scene are performed on each of the at least one category label to obtain at least one target category label for the video segment in the target scene. Thus, by utilizing a target scene-specific model and performing scene-specific post-processing on the obtained category labels, target category labels matching the target scene can be obtained. This allows for the determination of accurate category labels for the video, improving the accuracy of video recognition in specific scenes. This helps improve the accuracy of subsequently generated text descriptions of the target scene for video segments, thereby improving the quality of services related to video recognition.

[0039] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0040] Figure 2 A schematic diagram of an example architecture 200 for video recognition according to some embodiments of the present disclosure is shown. Architecture 200 can be implemented at an electronic device 110. For ease of discussion, reference will be made to... Figure 1 The environment 100 is used to describe the architecture 200. For example... Figure 2 As shown, the architecture 200 generally includes a target video recognition model 210, a post-processing unit 220, and a text generation model 230.

[0041] After acquiring the video segment 202, the electronic device 110 can use a target video recognition model 210 related to the target scene to classify the video segment 202 into at least one classification label 212 in a target classification label set related to the target scene. The target video recognition model 210 can be trained on the target classification label set. The electronic device 110 can also perform post-processing operations on each classification label in the at least one classification label 212 based on the type in which each classification label in the target classification label set is classified in the target scene, to obtain at least one target classification label 222 for the video segment 202 in the target scene. The electronic device 110 can also determine a text description 235 that matches the video segment 202 based on at least one target classification label 222.

[0042] Specifically, the electronic device 110 can utilize a trained target video recognition model 210 related to the target scene to classify video segments into at least one classification label 212 in a target classification label set related to the target scene (for example, it may include classification labels 212-1, 212-2, ..., 212-N, where N is a positive integer; hereinafter, one or more classification labels can be collectively referred to as classification label 212). The target scene here can be any suitable scene, such as a location introduction scene, a product recommendation scene, etc. It should be noted that the video segments 202 here may include one or more video segments with different subjects, and the electronic device 110 can use the target video recognition model 210 to classify each video segment into at least one corresponding classification label 212 in the target classification label set.

[0043] In some embodiments, the target video recognition model 210 can extract one or more of the following features from the video segment 202: visual features, speech features, and text features, as video features. For example, the target video recognition model 210 can extract the visual features of each video frame in the video segment 202. The target video recognition model 210 can extract the speech features of dialogue or narration appearing in the video segment 202. The target video recognition model 210 can extract the text features of subtitles appearing in the video segment 202. For example, the audio corresponding to the video segment can be converted into text based on speech recognition technology, and the text features of the text can be extracted. The target video recognition model 210 can then classify the video segment 202 into at least one corresponding classification label 212 in the target classification label set based on the video features, including multimodal features, extracted from the video segment 202.

[0044] The target classification label set here can be a set of labels related to the target scene obtained by the electronic device 110 processing the label set 130. Figure 3 A schematic diagram of an example 300 of tag set determination according to some embodiments of the present disclosure is shown. Figure 3 As shown, the electronic device 110 can acquire sample video segments 302 related to the target scene. The electronic device 110 can also acquire a trained general video recognition model and a general classification label set 312. The general video recognition model can be trained on the general classification label set.

[0045] The general classification label set 312 can be obtained directly by the electronic device 110, or can be obtained by the electronic device 110 processing the label set 130. In some embodiments, the electronic device 110 can obtain the label set 130. The electronic device 110 can perform de-duplication, de-noising, etc. on the label set 130 to obtain the general classification label set 312. It can be understood that in the case that there is no duplicate label, noise, etc. in the label set 130 (i.e. no processing is required), the electronic device 110 can also directly determine the label set 130 as the general classification label set 312.

[0046] The electronic device 110 can determine the confidence of the sample video clip 302 being classified into each classification label in the general classification label set 312 by using the trained general video recognition model. The confidence corresponding to each classification label can indicate the probability of the sample video clip 302 being classified into the classification label. For example, if the general classification label set 312 includes 6000 classification labels, the electronic device 110 can determine the confidence of the sample video clip 302 being classified into each of the 6000 classification labels (i.e. determine 6000 confidences).

[0047] The electronic device 110 can further determine the target classification label set from the general classification label set 312 based on the ordering of the confidence of the sample video clip being classified into each classification label in the general classification label set 312. Specifically, the general classification label set 312 and the confidence of each classification label in the general classification label set 312 will be provided to the label screening unit 320. The label screening unit 320 can select a plurality of candidate classification labels related to the target scene from the general classification label set 312 based on the ordering of the confidence of the sample video clip 302 being classified into each classification label in the general classification label set.

[0048] For example, the label screening unit 320 can select a predetermined number of classification labels with the highest confidence based on the ordering of the confidence of each classification label (e.g. if the ordering of the confidence of each classification label is in descending order, the label screening unit 320 can select a plurality of classification labels located in the top predetermined number of positions), which are the plurality of candidate classification labels related to the target scene. For example, if the general classification label set 312 includes 6000 classification labels, and the predetermined number is 35, the label screening unit 320 can select the 35 classification labels with the highest confidence from the 6000 classification labels based on the ordering of the confidence of the 6000 classification labels, and determine the 35 classification labels as the 35 candidate classification labels related to the target scene.

[0049] In some embodiments, the electronic device 110 can obtain a plurality of sample video clips 302 related to the target scene, and determine, by using the trained general video recognition model, a confidence that each of the plurality of sample video clips 302 is classified into each of the general classification label set 312. For each sample video clip 302, the electronic device 110 can select a plurality of classification labels related to the target scene from the general classification label set 312 based on an ordering of the confidence that the sample video clip is classified into each of the general classification label set 312. That is, for each sample video clip 302, the electronic device 110 can determine a plurality of classification labels. The electronic device 110 may, for example, perform de-duplication and ordering on the plurality of classification labels corresponding to the plurality of sample video clips 302, and determine a plurality of candidate classification labels related to the target scene therefrom.

[0050] For example, if the general classification label set 312 includes 6000 classification labels, the predetermined number is 35, and the electronic device 110 obtains 5 sample video clips 302, for each sample video clip 302, the label screening unit 320 can select, based on the ordering of the confidence of the 6000 classification labels, the 35 classification labels with the highest confidence from the 6000 classification labels that match the sample video clip 302. That is, for the 5 sample video clips 302, the electronic device 110 can obtain 35*5 = 175 classification labels. The electronic device 110 can perform de-duplication on the 175 classification labels, and order the de-duplicated plurality of classification labels based on the confidence corresponding to each of the de-duplicated plurality of classification labels. The electronic device 110 can determine, based on the ordering of the de-duplicated plurality of classification labels, the 35 classification labels with the highest confidence therefrom, and determine the 35 classification labels as the 35 candidate classification labels related to the target scene.

[0051] In some embodiments, the electronic device 110 may obtain a type classification instruction. For example, the electronic device 110 may obtain a type classification instruction input by the user. The type classification instruction classifies each category label in the target category label set into at least two of the following types: blacklist type 332, whitelist type 334, rewritten list type 336, and graylist type 338. Blacklist type 332 may include potentially inappropriate or unreasonable labels, such as labels involving privacy, specific location names, specific person names, etc., and labels that do not match the target scenario. Whitelist type 334 includes labels that match the target scenario, which may reflect the characteristics of the target scenario, such as the label "barbecue" in a food scenario or the label "ancient architecture" in a tourism scenario. Graylist type 338 may include labels that do not reflect the characteristics of the scenario but match the content of the sample video clip 302. For example, the video content of the sample video clip 302 of the food scenario includes people, and the corresponding candidate label may include the label "people". The label "people" does not reflect the characteristics of the food scenario.

[0052] Rewrite list type 336 can be, for example, a tag that can be rewritten to better match the target scenario while maintaining semantic integrity. For example, the tag "room interior" can be rewritten as "store environment" in a food scenario, and the tag "seafood market" can be rewritten as "seafood" in a food scenario, and so on. Here, rewrite list type 336 corresponds to a rewrite mapping for the target scenario, that is, electronic device 110 needs to rewrite the tags of rewrite list type 336 based on the rewrite mapping for the target scenario. The rewrite mapping indicates the mapping between the category tag to be rewritten (e.g., candidate category tag) and the rewritten category tag. Here, the rewrite mapping can be a rewrite mapping for the target scenario pre-constructed by electronic device 110, or a rewrite mapping for the target scenario directly obtained by electronic device 110.

[0053] Multiple candidate category labels are provided to the label generation unit 330. The label generation unit 330 can generate category labels in the target category label set by rewriting at least one of the multiple candidate category labels. The label generation unit 330 can, for example, perform the rewriting of at least one candidate category label based on a type partitioning indicator and a rewriting mapping. For example, in response to determining that at least one candidate category label is a rewriting list type 336, the electronic device 110 can rewrite this at least one candidate category based on the rewriting mapping to obtain rewritten category labels that match the target scenario.

[0054] For example, if the target scene is a food scene, and the at least one candidate label includes a candidate label “room interior”, the rewriting mapping indicates that the classification label “room interior” to be rewritten in the food scene corresponds to the label “store environment”, the electronic device 110 can, in response to determining that the candidate label “room interior” is the rewriting list type 336, rewrite the candidate label “room interior” to obtain the rewritten classification label “store environment” matching the food scene based on the rewriting mapping.

[0055] Thus, the label generation unit 330 can output the labels of the black list type 332, the labels of the white list type 334, the labels of the rewriting list type 336, and the labels of the gray list type 338. It should be noted that the labels of the white list type 334 include the labels of the white list type 334 as such and the rewritten labels obtained by rewriting the labels of the rewriting list type 336. For example, if the plurality of candidate labels include 35 labels, of which 15 are of the white list type, 5 are of the black list type, 5 are of the rewriting list type, and 10 are of the gray list type, the label generation unit 330 can output 40 labels, including 20 of the white list type (15 of the white list type as such and 5 of the rewritten labels corresponding to the rewriting list type), 5 of the black list type, 5 of the rewriting list type, and 10 of the gray list type. The electronic device 110 can determine all the labels output by the label generation unit 330 as the classification labels in the target classification label set. That is, the electronic device 110 can determine the target classification label set related to the target scene in the manner shown. Figure 3

[0056] Referring back to Figure 2 , the at least one classification label 212 is provided to the post-processing unit 220. The post-processing unit 220 can perform a post-processing operation for the target scene on each of the at least one classification label 212 based on the type in which each of the classification labels in the target classification label set is divided in the target scene, to obtain at least one target classification label 222 (e.g., which can include target classification labels 222-1, 222-2, …, 222-M, where M is a positive integer greater than or equal to 1, and one or more target classification labels can be collectively referred to as target classification labels 222 hereinafter) of the video clip in the target scene. The post-processing operation here can include, for example, any appropriate operation such as a discard operation, a rewriting operation, a marking operation, etc., which will be described in detail later.

[0057] Figure 4 A schematic diagram of an example architecture 400 of video recognition according to further embodiments of the present disclosure is shown. As Figure 4 ​As shown, the post-processing unit 220 can include a label adjustment unit 430 and a label determination unit 440. In some embodiments, after the at least one classification label 212 is provided to the post-processing unit 220, for each classification label 212, the post-processing unit 220 determines (410) whether the classification label 212 is a blacklist type in the target scene. If a classification label 212 (e.g., a first classification label) is a blacklist type, the post-processing unit 220 can perform a discard operation on the classification label 212. That is, the post-processing unit 220 can discard the first classification label from being included in the at least one target classification label 222 in response to determining that the first classification label of the at least one classification label 212 is classified as a blacklist type.

[0058] For each classification label 212, the post-processing unit 220 can further determine (420) whether the classification label 212 is a rewrite list type in the target scene in response to determining that the classification label 212 is not a blacklist type. If a classification label 212 (e.g., a second classification label) is a rewrite list type, the post-processing unit 220 can provide the classification label 212 to the label adjustment unit 430. The label adjustment unit 430 can perform a rewrite operation on the second classification label based on a rewrite mapping to obtain a rewritten third classification label. That is, the post-processing unit 220 determines the rewritten third classification label to which the second classification label of the at least one classification label is mapped from the rewrite mapping for the target scene in response to determining that the second classification label of the at least one classification label is classified as a rewrite list type. The third classification label will be used to determine the at least one target classification label 222. In some embodiments, if a classification label 212 (e.g., a fourth classification label) is a gray list type, the post-processing unit 220 can also provide the classification label 212 to the label adjustment unit 430. The label adjustment unit 430 can perform a flagging operation on the fourth classification label to flag the fourth classification label as a gray list label. The label adjustment unit 430 can provide the rewritten third classification label and / or the flagged fourth classification label to the label determination unit 440.

[0059] For each classification label 212, the post-processing unit 220 can further determine, in response to determining that the classification label 212 is not the graylist / rewrite list type, that the classification label 212 is the whitelist type, directly providing the classification label 212 to the label determining unit 440. The label determining unit 440 can determine at least one target classification label 222 from the obtained at least one classification label (including the whitelist type label, the rewritten third classification label and / or the marked fourth classification label). Illustratively, the label determining unit 440 determines the at least one target classification label 222 from the obtained at least one classification label based on the ranking of the confidence of each classification label. Illustratively, the label determining unit 440 can select a predetermined number of classification labels with the highest confidence based on the ranking of the confidence of each classification label (for example, if the ranking of the confidence of each classification label is in descending order, the label determining unit 440 can select the classification labels ranked in the top predetermined number), which are the at least one target classification label 222 related to the target scene. For example, if the label determining unit 440 obtains 5 classification labels and the predetermined number is 2, the label determining unit 440 can select the 2 classification labels with the highest confidence from the 5 classification labels based on the ranking of the confidence of the 5 classification labels, and determine the 2 classification labels as the 2 target classification labels 222 related to the target scene.

[0060] The at least one target classification label 222 and the prompt word 204 are provided to the text generation model 230 together. The prompt word 204 here can be pre-set by the user, input in real time by the user, or determined by the electronic device 110, which is not limited in the present disclosure. The prompt word 204 may, for example, be the text “Please generate a script according to the video content”. The text generation model 230 can generate a text description 235 for the video segment 202 based on the obtained at least one target classification label 222 and the prompt word 204. If the label “performance” is included in the at least one target classification label 222, based on the label, the generated text description 235 may, for example, be “The performance here is really rich in national characteristics, with deafening drums and a strong festive atmosphere, making people feel as if they are there!”.

[0061] It is noted that, as the at least one target classification label 222 can include a gray-list type label, and the gray-list type includes a label that cannot characterize a live scene but matches the content of the video clip 202, in some embodiments, to improve the accuracy of the generated text description 235, if it is determined that a fourth classification label (i.e., a classification label that is labeled as a gray-list label) is included in the at least one target classification label 222, the electronic device 110 can set a weight of the fourth classification label, which indicates the degree of influence of the fourth classification label on the text description, to be lower than a predetermined threshold when generating the text description 235 for the video clip based on the at least one target classification label 222. The predetermined threshold here can be 0 or a small value close to 0, for example. As the gray-list label has a low relevance to the target scene and can not characterize the target scene, by setting a low weight, the influence of the gray-list label on the text description 235 in the generation process of the text description 235 can be reduced, and the text description 235 is generated as much as possible based on other labels that are more relevant to the target scene. In some embodiments, if the weight of the gray-list label is set to 0, the gray-list label will not participate in the generation of the text description 235, although the corresponding video clip can still be used to generate the final video.

[0062] In some embodiments, if there are multiple video clips 202, the electronic device 110 can determine the at least one target classification model 222 for each of the multiple video clips 202. The electronic device 110 can provide the at least one target classification model 222 for each of the multiple video clips 202 and the prompt word 204 to the text generation model 230 together. The text generation model 230 can generate one overall text description 235 corresponding to the multiple video clips 202 based on the at least one target classification model 222 for each of the multiple video clips 202 and the prompt word 204.

[0063] The above describes the model identification and the generation of the text description in general. The various embodiments of the present disclosure can perform post-processing on the obtained classification labels to obtain target classification labels that match the target scene. The accuracy of the finally determined target classification labels can be improved, which helps to improve the accuracy of the subsequently generated text description for the video clip of the target scene, and can improve the quality of the service related to video recognition.

[0064] The training process of the target video recognition model will be described below in conjunction with Figure 5 . Figure 5 A schematic diagram of an example process 500 for model training according to some embodiments of the present disclosure is shown. The process 500 can be implemented at the electronic device 110. For the purpose of discussion, the process 500 is described below with reference to the system 100. Figure 1 As described above, the system 100 includes the electronic device 110 and the server 120. Figure 5As shown, the process 500 can include three training stages, a first training stage, a second training stage, and a third training stage. The first and second training stages can also be collectively referred to as pre-training stages, and the third training stage can also be referred to as an adjustment stage.

[0065] In the first training stage, the electronic device 110 can obtain video clips 502 and corresponding video texts 504. It can be understood that the electronic device 110 can obtain one or more video clips 502 and corresponding one or more video texts 504 (i.e., multiple pairs of matching video clips 502 and video texts 504). In some embodiments, if the video texts 504 corresponding to the video clips 502 cannot be directly obtained, the electronic device 110 can also process the video clips 502 to extract the video texts 504 therefrom. For example, the electronic device 110 can extract the video texts 504 from the video clips 502 by means of a text extraction model. In the case where the video texts 504 cannot be directly extracted, the text extraction model 420 may, for example, also identify the video texts 504 from the video clips 502 based on an automatic speech recognition (ASR) technique or a text recognition (e.g., optical character recognition, OCR) technique. The text recognition technique can be used to identify the subtitles of the video clips 502, and the ASR technique can be used to identify the speech (e.g., character dialogues, voiceovers, etc.) of the video clips 502. The specific selection of which technique can be determined based on the specific sample video.

[0066] The electronic device 110 can then train the video recognition model 510 and the text recognition model 512 based on the obtained video clips 502 and video texts 504. The video recognition model 510 can at least include a video encoder. The text recognition model 512 can at least include a text encoder. During the training process, the parameter values of the video recognition model 510 and the text recognition model 512 are updated and adjusted based on the multiple pairs of matching video clips 502 and video texts 504. In some embodiments, the training process of the first training stage includes contrastive learning based on the multiple pairs of matching video clips 502 and video texts 504. Contrastive learning is a discriminative self-supervised learning method that obtains a learning model by automatically constructing matching sample pairs (i.e., a pair of matching video clips 502 and video texts 504) and non-matching samples (i.e., a pair randomly constructed from the multiple pairs of matching video clips 502 and video texts 504). Through this model, the matching sample pairs are relatively close in the projection space, while the non-matching sample pairs are relatively far apart in the projection space. Using the contrastive learning method, the distance between the positive sample pairs (text features and video features that match each other) is pulled closer, and the distance between the negative sample pairs (text features and video features that do not match each other) is pushed farther apart during the model training stage. This can enhance sample diversity and improve the performance of the video recognition model 510 and the text recognition model 512.

[0067] Further, after the first stage training is completed, in a second training stage, the electronic device 110 can obtain the video recognition model 510 trained in the first stage (also referred to as a pre-trained model). The electronic device 110 can pre-train the obtained video recognition model 510 trained in the first stage 510 using the first set of sample video clips 522 to obtain a general video recognition model 520 trained in the second stage, on the set of general classification labels 524. The general video recognition model 520 can include, for example, the video encoder trained in the first stage and an output layer that can serve the purpose of general classification. The first set of sample video clips can include any suitable number of video clips. During the second training stage, for each of the first set of sample video clips 522, the electronic device 110 can determine a first cross-entropy loss, such as binary cross-entropy (BCE) loss, between a predicted general classification label output by the video recognition model 510 and an actual general classification label. The electronic device 110 can train the video recognition model 510 to obtain the general video recognition model 520 by reducing or minimizing the first cross-entropy loss. The first and second training stages can be referred to as pre-training stages.

[0068] Further, after the first stage training is completed, in a second training stage, the electronic device 110 can obtain the video recognition model 510 trained in the first stage (also referred to as a pre-trained model). The electronic device 110 can pre-train the obtained video recognition model 510 trained in the first stage 510 using the first set of sample video clips 522 to obtain a general video recognition model 520 trained in the second stage, on the set of general classification labels 524. The general video recognition model 520 can include, for example, the video encoder trained in the first stage and an output layer that can serve the purpose of general classification. The first set of sample video clips can include any suitable number of video clips. During the second training stage, for each of the first set of sample video clips 522, the electronic device 110 can determine a first cross-entropy loss, such as binary cross-entropy (BCE) loss, between a predicted general classification label output by the video recognition model 510 and an actual general classification label. The electronic device 110 can train the video recognition model 510 to obtain the general video recognition model 520 by reducing or minimizing the first cross-entropy loss. The first and second training stages can be referred to as pre-training stages.

[0069] The electronic device 110 can train different target video recognition models 530 for different target scenes based on the general video recognition model 520. Illustratively, for a target scene 1, the electronic device 110 can utilize a second set of sample video clips 532-1 related to the target scene to adjust the general video recognition model 520 on a target set of classification labels 534-1 to obtain a target video recognition model 530-1 that matches the target scene 1. For a target scene 2, the electronic device 110 can utilize a second set of sample video clips 532-2 related to the target scene to adjust the general video recognition model 520 on a target set of classification labels 534-2 to obtain a target video recognition model 530-2 that matches the target scene 2, and so on. Thus, the electronic device 110 can train multiple target video recognition models 530 that respectively match multiple target scenes based on the pre-trained general video recognition model 520.

[0070] Thus, with various embodiments of the present disclosure, without a large amount of labeled data for target scenes, a three-stage training can be conveniently and quickly performed based on a large-scale general set of classification labels and a small part of target set of classification labels for target scenes to obtain a target video recognition model that matches the target scene.

[0071] Figure 6 A flowchart of a process 600 for video recognition is shown in accordance with some embodiments of the present disclosure. The process 600 can be implemented at the electronic device 110. The process 600 is described below with reference to the electronic device 110, the general video recognition model 520, and the target video recognition model 530. Figure 1 The process 600 is described.

[0072] At block 610, the electronic device 110 classifies the video clip into at least one classification label in a target set of classification labels related to a target scene using a target video recognition model related to the target scene, the target video recognition model being trained on the target set of classification labels.

[0073] At block 620, the electronic device 110 performs a post-processing operation for the target scene for each classification label in the at least one classification label based on a type in which each classification label in the target set of classification labels is divided in the target scene to obtain at least one target classification label for the video clip in the target scene.

[0074] In some embodiments, the process 600 further includes generating a textual description for the video clip based on the at least one target classification label using a text generation model.

[0075] In some embodiments, the type of each classification label in the target classification label set in the target scene includes a blacklist type, and wherein performing the post-processing operation for the target scene on each of the at least one classification label includes: in response to determining that a first classification label in the at least one classification label is classified as the blacklist type, discarding the first classification label such that the first classification label is not included in the at least one target classification label.

[0076] In some embodiments, the type of each classification label in the target classification label set in the target scene includes a rewrite list type, and wherein performing the post-processing operation for the target scene on each of the at least one classification label includes: in response to determining that a second classification label in the at least one classification label is classified as the rewrite list type, determining, from a rewrite mapping for the target scene, a rewritten third classification label to which the second classification label is mapped, wherein the at least one target classification label is determined based at least on the third classification label.

[0077] In some embodiments, the type of each classification label in the target classification label set in the target scene includes a grey list type, and wherein performing the post-processing operation for the target scene on each of the at least one classification label includes: in response to determining that a fourth classification label in the at least one classification label is classified as the grey list type, marking the fourth classification label as a grey list label; and the process 600 further includes: in response to determining that the fourth classification label is included in the at least one target classification label, setting a weight of the fourth classification label to be lower than a predetermined threshold when generating a textual description for the video segment based on the at least one target classification label, the weight indicating a degree of influence of the fourth classification label on the textual description.

[0078] In some embodiments, the target classification label set related to the target scene is generated by: obtaining a sample video segment related to the target scene; determining, by utilizing a trained general video recognition model, a confidence that the sample video segment is classified to each classification label in a general classification label set, the general video recognition model being trained on the general classification label set; determining the target classification label set from the general classification label set based on a ranking of the confidence that the sample video segment is classified to each classification label in the general classification label set.

[0079] In some embodiments, determining the target set of classification labels from the universal set of classification labels comprises: selecting a plurality of candidate classification labels related to the target scene from the universal set of classification labels based on an ordering of confidence that the sample video segments are classified to respective classification labels in the universal set of classification labels; and generating a classification label in the target set of classification labels by a rewrite of at least one candidate classification label in the plurality of candidate classification labels. In some embodiments, the process 600 further comprises: constructing a rewrite mapping for the target scene, the rewrite mapping indicating a mapping between the at least one candidate classification label and the rewritten at least one classification label.

[0080] In some embodiments, the process 600 further comprises: obtaining a type division indication, the type division indication dividing respective classification labels in the target set of classification labels into at least two types: a whitelist type, a blacklist type, a rewrite list type, the rewrite list type corresponding to a rewrite mapping for the target scene, the rewrite mapping indicating a mapping between a classification label to be rewritten and a rewritten classification label, and a gray list type.

[0081] In some embodiments, the target video recognition model related to the target scene is trained by: obtaining a pre-trained model; pre-training the model on a universal set of classification labels using a first set of sample video segments, resulting in a trained universal video recognition model; and adjusting the universal video recognition model on a target set of classification labels using a second set of sample video segments related to the target scene, resulting in the target video recognition model.

[0082] In some embodiments, the process 600 further comprises: adjusting the universal video recognition model on a set of classification labels related to another scene using a third set of sample video segments related to the another scene, resulting in a video recognition model related to the another scene.

[0083] Embodiments of the present disclosure also provide a corresponding apparatus for implementing the above-described method or process. Figure 7 A schematic structural block diagram of an apparatus 700 for video recognition according to certain embodiments of the present disclosure is shown. The apparatus 700 can be implemented as or included in the electronic device 110. Respective modules / components in the apparatus 700 can be implemented by hardware, software, firmware, or any combination thereof.

[0084] As shown, the apparatus 700 includes a video classification module 710 configured to classify the video clip into at least one classification label in a target classification label set related to a target scene using a target video recognition model trained on the target classification label set. The apparatus 700 also includes a label post-processing module 720 configured to perform a post-processing operation for each classification label in the at least one classification label based on a type in which the target classification label set is divided in the target scene to obtain at least one target classification label of the video clip in the target scene.

[0085] In some embodiments, the apparatus 700 further includes a text description generation module configured to generate a text description for the video clip based on the at least one target classification label using a text generation model.

[0086] In some embodiments, the type of each classification label in the target classification label set in the target scene includes a blacklist type, and the label post-processing module 720 is further configured to, in response to determining that a first classification label in the at least one classification label is divided into the blacklist type, discard the first classification label so that the first classification label is not included in the at least one target classification label.

[0087] In some embodiments, the type of each classification label in the target classification label set in the target scene includes a rewrite list type, and the label post-processing module 720 is further configured to, in response to determining that a second classification label in the at least one classification label is divided into the rewrite list type, determine a rewritten third classification label to which the second classification label is mapped from a rewrite mapping for the target scene, wherein the at least one target classification label is determined based at least on the third classification label.

[0088] In some embodiments, the type of each classification label in the target classification label set in the target scene includes a gray list type, and the label post-processing module 720 is further configured to, in response to determining that a fourth classification label in the at least one classification label is divided into the gray list type, mark the fourth classification label as a gray list label; and the apparatus 700 further includes a weight setting module configured to, in response to determining that the fourth classification label is included in the at least one target classification label, set a weight of the fourth classification label to be lower than a predetermined threshold when generating the text description for the video clip based on the at least one target classification label, the weight indicating a degree of influence of the fourth classification label on the text description.

[0089] In some embodiments, the apparatus 700 further includes a label set generation module configured to generate a target classification label set related to the target scene by: obtaining a sample video clip related to the target scene; determining, by using the trained general video recognition model, a confidence that the sample video clip is classified to each classification label in a general classification label set, the general video recognition model being trained on the general classification label set; and determining the target classification label set from the general classification label set based on an ordering of the confidence that the sample video clip is classified to each classification label in the general classification label set.

[0090] In some embodiments, the label set generation module is further configured to: select a plurality of candidate classification labels related to the target scene from the general classification label set based on the ordering of the confidence that the sample video clip is classified to each classification label in the general classification label set; and generate a classification label in the target classification label set by paraphrasing at least one candidate classification label in the plurality of candidate classification labels. In some embodiments, the apparatus 700 further includes a mapping construction module configured to construct a paraphrasing mapping for the target scene, the paraphrasing mapping indicating a mapping between the at least one candidate classification label and the paraphrased at least one classification label.

[0091] In some embodiments, the apparatus 700 further includes an indication acquisition module configured to obtain a type division indication, the type division indication dividing each classification label in the target classification label set into at least two types: a white list type, a black list type, a paraphrasing list type, the paraphrasing list type corresponding to a paraphrasing mapping for the target scene, the paraphrasing mapping indicating a mapping between a classification label to be paraphrased and a paraphrased classification label, and a gray list type.

[0092] In some embodiments, the apparatus 700 further includes a training module configured to train a target video recognition model related to the target scene by: obtaining a pre-trained model; pre-training the model on the general classification label set by using a first sample video clip set to obtain a trained general video recognition model; and adjusting the general video recognition model on the target classification label set by using a second sample video clip set related to the target scene to obtain the target video recognition model.

[0093] In some embodiments, the apparatus 700 further includes an adjustment module configured to adjust the general video recognition model on a classification label set related to another scene by using a third sample video clip set related to the another scene to obtain a video recognition model related to the another scene.

[0094] The units and / or modules included in the apparatus 700 can be implemented utilizing various means, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, e.g., machine-executable instructions stored on a storage medium. In addition to or alternatively, some or all of the units and / or modules in the apparatus 700 can be implemented at least partially by one or more hardware logic components. As an example and not by way of limitation, example types of hardware logic components that can be used include Field- programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-Chip (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0095] Figure 8 A block diagram of an electronic device 800 in which one or more embodiments of the disclosure can be implemented is shown. It should be understood that Figure 8 The electronic device 800 shown is merely exemplary and should not be construed as limiting the scope of the embodiments described herein. Figure 8 The electronic device 800 shown can be used to implement the electronic device 110 of Figure 1 or the apparatus 700 of Figure 7 .

[0096] As shown in Figure 8 , the electronic device 800 is in the form of a general- purpose computing device. Components of the electronic device 800 can include, but are not limited to, one or more processors or processing units 810, a memory 820, a storage device 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. The processing unit 810 can be a real or virtual processor and capable of executing various processing in accordance with programs stored in the memory 820. In a multi-processing system, multiple processing units execute computer-executable instructions in parallel to improve the processing power of the electronic device 800.

[0097] The electronic device 800 typically includes a plurality of computer storage media. Such media can be any available media that is localized or remote to the electronic device 800, including both volatile and nonvolatile media, removable and non-removable media. The memory 820 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 830 can be a removable or non-removable media, and can include machine-readable media, such as a flash drive, a magnetic disk, or any other media that can be used to store information and / or data and that can be accessed by the electronic device 800.

[0098] The electronic device 800 can further include additional detachable / non-detachable, volatile / non-volatile storage media. Although not shown in FIG. 8, a disk drive for reading from or writing to a detachable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a detachable, non-volatile optical disk (e.g., a CD-ROM) can be provided. In these cases, each drive can be connected to the bus (not shown) by one or more data media interfaces. The memory 820 can include a computer program product 825 having one or more program modules configured to carry out the various methods or acts of the various embodiments of the present disclosure. Figure 8

[0099] The communication unit 840 enables communication with other electronic devices through communication media. Additionally, the functionality of the components of the electronic device 800 can be implemented in a single computing cluster or a plurality of computer machines capable of communicating over a communication connection. As such, the electronic device 800 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network nodes in the networking environment.

[0100] The input device 850 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 860 can be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 800 can also communicate with one or more external devices (not shown) such as a storage device, a display device, etc., one or more devices that enable a user to interact with the electronic device 800, or any devices (e.g., a network card, a modem, etc.) that enable the electronic device 800 to communicate with one or more other electronic devices, through the communication unit 840, as desired. Such communication can be carried out via an input / output (I / O) interface (not shown).

[0101] According to an example embodiment of the present disclosure, there is provided a computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an example embodiment of the present disclosure, there is also provided a computer program product tangibly stored on a non-transitory computer-readable medium and comprising computer-executable instructions, wherein the computer-executable instructions are executed by a processor to implement the method described above.

[0102] Various aspects of the disclosure can be described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.​

[0103] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data storage cycles that change state. The instructions can be executed by one or more processors of a computer, other programmable data processing apparatus, or other devices to produce a computer-implemented process such that the instructions which execute via the one or more processors of the computer or other programmable data processing devices create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0104] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data storage cycles that change state. The instructions can be executed by one or more processors of a computer, other programmable data processing apparatus, or other devices to produce a computer-implemented process such that the instructions which execute via the one or more processors of the computer or other programmable data processing devices create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0105] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0106] The implementations of the disclosure have been described with the intent to be illustrative rather than restrictive. Alternative means of implementing the implementations described herein will become apparent to those of ordinary skill in the art upon reviewing the description. The features described herein can be employed in various combinations; in some aspects all features are combined in a single implementation, in other aspects, not all features are combined in a single implementation. The scope of the disclosure should be determined, not with reference to the above description, but should instead be determined with reference to the appended claims, along with the full range of equivalents to which such claims are entitled. In this regard, the description and representations disclosed herein are to be interpreted in an illustrative manner. Alternative uses will become apparent to the skilled artisan.

Claims

1. A method for video recognition, comprising: Using a target video recognition model related to a target scene, video clips are classified into at least one classification label in a target classification label set related to the target scene, wherein the target video recognition model is trained on the target classification label set. as well as Based on the type of each category tag in the target category tag set in the target scene, post-processing operations for the target scene are performed on each category tag in the at least one category tag to obtain at least one target category tag of the video segment in the target scene, wherein the type includes whitelist type, blacklist type, rewritten list type or graylist type, and the post-processing operations corresponding to the category tags of different types are different.

2. The method according to claim 1, further comprising: A text generation model is used to generate a text description for the video segment based on the at least one target classification label.

3. The method of claim 1, wherein the type of each category tag in the target category tag set in the target scenario includes a blacklist type, and wherein performing post-processing operations for the target scenario on each category tag in the at least one category tag includes: In response to determining that a first category label among the at least one category label is classified as the blacklist type, the first category label is discarded so that it is not included in the at least one target category label.

4. The method of claim 1, wherein the type of each category label in the target category label set in the target scenario includes a rewritten list type, and wherein performing post-processing operations for the target scenario on each category label in the at least one category label includes: In response to determining that a second category label among the at least one category label is classified as the rewritten list type, a rewritten third category label to which the second category label is mapped is determined from the rewritten mapping for the target scenario. The at least one target classification label is determined based on the third classification label.

5. The method of claim 1, wherein the type of each category label in the target category label set in the target scenario includes a graylist type, and wherein performing post-processing operations for the target scenario on each category label in the at least one category label includes: In response to determining that the fourth category label among the at least one category label is classified into the gray list type, the fourth category label is marked as a gray list label; and The method further includes: In response to determining that the fourth category label is included in the at least one target category label, when generating a text description for the video segment based on the at least one target category label, the weight of the fourth category label is set to be below a predetermined threshold, the weight indicating the degree of influence of the fourth category label on the text description.

6. The method of claim 1, wherein the target classification label set associated with the target scene is generated by: Obtain sample video clips related to the target scene; Using a trained general video recognition model, the confidence level of classifying the sample video segment into each classification label in a general classification label set is determined, wherein the general video recognition model is trained on the general classification label set; The target classification label set is determined from the general classification label set based on the confidence ranking of each classification label in the general classification label set to which the sample video clips are classified.

7. The method of claim 6, wherein determining the target classification tag set from the general classification tag set comprises: Based on the confidence ranking of each classification label in the general classification label set to which the sample video segment is classified, multiple candidate classification labels related to the target scene are selected from the general classification label set. as well as By rewriting at least one of the multiple candidate classification labels, classification labels in the target classification label set are generated. and The method further includes: Construct a rewrite mapping for the target scenario, wherein the rewrite mapping indicates the mapping between the at least one candidate category label and the at least one rewritten category label.

8. The method according to claim 1, further comprising: Obtain a type partitioning indicator that classifies each category label in the target category label set into at least two of the following types: The whitelist type, The blacklist type, The rewrite list type corresponds to a rewrite mapping for the target scenario. The rewrite mapping indicates the mapping between the category tags to be rewritten and the rewritten category tags. The gray list type.

9. The method of claim 1, wherein the target video recognition model associated with the target scene is trained by: Obtain a pre-trained model; Using the first set of sample video clips, the model is pre-trained on a general classification label set to obtain a trained general video recognition model; and Using a second set of sample video clips related to the target scene, the general video recognition model is adjusted on the target classification label set to obtain the target video recognition model.

10. The method of claim 9, further comprising: By using a third set of sample video clips related to another scene, the general video recognition model is adjusted on a set of classification labels related to the other scene to obtain a video recognition model related to the other scene.

11. An apparatus for video recognition, comprising: The video classification module is configured to classify video segments into at least one classification label in a target classification label set related to the target scene using a target video recognition model associated with the target scene, wherein the target video recognition model is trained on the target classification label set. as well as The tag post-processing module is configured to perform post-processing operations on each of the at least one category tag in the target scene based on the type in which each category tag in the target category tag set is divided in the target scene, so as to obtain at least one target category tag of the video segment in the target scene, wherein the type includes whitelist type, blacklist type, rewritten list type or graylist type, and the post-processing operations corresponding to the category tags divided into different types are different.

12. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 10 when executed by the at least one processing unit.

13. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 10.

Images