A video data processing method, device, apparatus and readable storage medium

Abstract

The application discloses a video data processing method, device and equipment and a readable storage medium. The method comprises the following steps: acquiring business requirement script information input by a first object and video resources uploaded by a second object; performing text label extraction processing on the business requirement script information to obtain business attribute statistical labels and business semantic feature labels; performing video resource preprocessing on the video resources to obtain video resource attribute statistical labels; performing key frame image extraction processing on the video resources to obtain at least two key frame images, and performing image label extraction processing on the at least two key frame images respectively to obtain image content feature labels; and if the business attribute statistical labels and the video resource attribute statistical labels match, determining a matching result of the video resources for the business requirement script information according to the business semantic feature labels and the image content feature labels. The application can reduce the cost and time consumption of video resource auditing.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a video data processing method, apparatus, device, and readable storage medium. Background Technology

[0002] Creative custom videos are a new trend on the internet in recent years. The characteristic of this trend is that when the video requester lacks creative ideas and materials, the custom service provider is responsible for facilitating the production of corresponding creative custom videos by creative suppliers. In other words, the video requester provides a business requirement script in text form, and the creative supplier shoots the creative custom video based on the business requirement script.

[0003] The platforms provided by custom service providers are public, so a single business requirement script posted by a video requester may receive responses from hundreds, thousands, or even more creative suppliers. However, some of these suppliers' uploaded custom videos may not match the business requirement script. To save the video requester's time and provide better service, the custom service provider needs to review the custom videos first, eliminating those that don't match the business requirement script. Most industry practice uses manual review, requiring reviewers to understand both the requester's needs and the content of the custom videos—a labor-intensive and time-consuming process, prone to omissions and misjudgments. Summary of the Invention

[0004] This application provides a video data processing method, apparatus, device, and readable storage medium, which can reduce the cost and time consumption of video resource review.

[0005] One embodiment of this application provides a video data processing method, including:

[0006] Obtain the business requirement script information entered by the first object, and obtain the video resources uploaded by the second object; the video resources are the resources uploaded in response to the business requirement script information.

[0007] The business requirement script information is processed by extracting text tags to obtain business script tags; business script tags include business attribute statistical tags and business semantic feature tags.

[0008] Video resources are preprocessed to obtain statistical labels for video resource attributes;

[0009] Keyframe image extraction processing is performed on video resources to obtain at least two keyframe images. Image label extraction processing is then performed on the at least two keyframe images to obtain image content feature labels.

[0010] If the business attribute statistical tags and the video resource attribute statistical tags match, then the matching result of the video resource for the business requirement script information is determined based on the business semantic feature tags and the image content feature tags.

[0011] One embodiment of this application provides a video data processing apparatus, including:

[0012] The acquisition module is used to acquire the business requirement script information entered by the first object and the video resources uploaded by the second object; the video resources are the resources uploaded in response to the business requirement script information.

[0013] The first extraction module is used to extract text tags from business requirement script information to obtain business script tags; business script tags include business attribute statistical tags and business semantic feature tags.

[0014] The video preprocessing module is used to preprocess video resources and obtain statistical tags for video resource attributes.

[0015] The second extraction module is used to perform keyframe image extraction processing on video resources to obtain at least two keyframe images, and to perform image label extraction processing on the at least two keyframe images to obtain image content feature labels.

[0016] The first matching module is used to determine the matching result of the video resource for the business requirement script information based on the business semantic feature label and the image content feature label if the business attribute statistical label and the video resource attribute statistical label match.

[0017] The business requirement script information includes a description of the business requirement scenario, keywords for the business requirement script, and basic information about the business requirement script.

[0018] The first extraction module includes:

[0019] The first processing unit is used to perform structured attribute statistical processing on the business requirement scenario description text, business requirement script keywords, and basic information of the business requirement script to obtain business attribute statistical tags.

[0020] The second processing unit is used to perform unstructured semantic feature extraction processing on the business requirement scenario description text to obtain the first semantic feature label;

[0021] The third processing unit is used to match the keywords of the business requirement script with the tags in the tag library, and to use the tags that match the keywords of the business requirement script as the second semantic feature tags.

[0022] The fourth processing unit is used to use the first semantic feature label and the second semantic feature label as business semantic feature labels.

[0023] The second processing unit includes:

[0024] The word segmentation subunit is used to segment the text describing the business requirement scenario into words to obtain a set of words to be processed; the set of words to be processed includes one or more words to be processed.

[0025] The model calling subunit is used to call the text semantic feature extraction model to perform unstructured semantic feature extraction on the set of words to be processed, and obtain the first semantic feature label.

[0026] The text semantic feature extraction model includes a first embedding layer, a memory network layer, and a normalized classification layer. Specifically, the model's calling sub-unit calls the first embedding layer to perform embedding feature processing on each word to be processed, obtaining a word embedding vector corresponding to each word. It then identifies the ranking position of each word in the word set, calls the memory network layer to perform hidden layer feature representation processing on the word embedding vectors corresponding to each word according to its ranking position, obtaining a hidden layer representation vector corresponding to the word set. Finally, it calls the normalized classification layer to normalize the hidden layer representation vectors, obtaining the text label prediction distribution corresponding to the words to be processed. Based on the text label prediction distribution, it determines the first semantic feature label corresponding to the business requirement scenario description text. The text label prediction distribution contains the probability value of matching each label in the label library with the business requirement scenario description text.

[0027] The third processing unit includes:

[0028] The word representation subunit is used to perform word representation processing on the keywords of the business requirement script to obtain the word representation vector corresponding to the keywords of the business requirement script.

[0029] The tag determination subunit is used to obtain the tag representation vector corresponding to each tag in the tag library and determine the vector distance between the word representation vector and each tag representation vector.

[0030] The label determines the sub-unit, and specifically, it is used to select the label with the smallest vector distance as the second semantic feature label.

[0031] Among them, at least two keyframe images include keyframe image M. i , where i is a positive integer less than or equal to the total number of at least two keyframe images;

[0032] The second extraction module includes:

[0033] Image representation unit, used for keyframe image M i Matrix representation processing is performed to obtain the keyframe image M. i The image representation matrix;

[0034] The label extraction unit is used to call the image content feature extraction model to extract keyframe images M. i Image label extraction is performed on the image representation matrix to obtain the keyframe image M. i Corresponding single-image content feature labels;

[0035] The label determination unit is used to take the single image content feature labels corresponding to at least two keyframe images as image content feature labels.

[0036] The image content feature extraction model includes a second embedding layer, an encoding layer, a decoding layer, and a feedforward neural network classification layer.

[0037] The tag extraction unit includes:

[0038] Embedded subunit, used to call the second embedding layer on the keyframe image M i The image representation matrix is ​​processed by convolution vector representation to obtain the image representation vector;

[0039] The encoding subunit is used to call the encoding layer to encode the image representation vector to obtain the image encoded representation vector;

[0040] The decoding subunit is used to call the decoding layer to decode the image encoding vector and obtain the image decoded representation vector.

[0041] The classification subunit is used to call the classification layer of the feedforward neural network to classify the image decoded representation vector and obtain the keyframe image M. i The corresponding image label prediction distribution is used to determine the keyframe image M. i Corresponding single-image content feature labels; image label prediction distribution includes keyframe image M i The probability value of matching each tag in the tag library.

[0042] The business attribute statistics label includes S business attribute statistics sub-labels; S is a positive integer;

[0043] The aforementioned video data processing device further includes:

[0044] The second matching module is used to traverse the S business attribute statistical sub-labels and sequentially obtain the k-th business attribute statistical sub-label; k is a positive integer less than or equal to S;

[0045] The second matching module is also used to determine that the business attribute statistics tag and the video resource attribute statistics tag do not match if the target attribute statistics tag does not exist in the video resource attribute statistics tag; the target attribute statistics tag refers to the tag with the same attribute type as the kth business attribute statistics sub-tag.

[0046] The second matching module is also used to determine that the business attribute statistics tag and the video resource attribute statistics tag do not match if the target attribute statistics tag exists in the video resource attribute statistics tag and the kth business attribute statistics sub-tag is different from the target attribute statistics tag.

[0047] The second matching module is also used to determine that the kth business attribute statistical sub-label matches the video resource attribute statistical label if the target attribute statistical label exists in the video resource attribute statistical label and the kth business attribute statistical sub-label is the same as the target attribute statistical label.

[0048] The second matching module is also used to determine that the business attribute statistical label and the video resource attribute statistical label match if each business attribute statistical sub-label matches the video resource attribute statistical label.

[0049] The first matching module includes:

[0050] The first coefficient determination unit is used to determine the coefficients based on the business semantic feature labels and the keyframe image M. i The corresponding single-image content feature labels are used to determine the keyframe image M. i The corresponding image label repetition coefficient;

[0051] The second coefficient determination unit is used to average the image label repetition coefficients corresponding to at least two keyframe images to obtain the resource label repetition coefficients corresponding to the video resources.

[0052] The matching unit is used to determine that the matching result of the video resource for the business requirement script information is a matching failure result if the resource tag repetition coefficient is less than the resource tag repetition threshold.

[0053] The matching unit is also used to determine the matching result of the video resource for the business requirement script information as a successful match if the resource tag repetition coefficient is greater than or equal to the resource tag repetition threshold.

[0054] The first coefficient determination unit includes:

[0055] The label classification subunit is used to classify business semantic feature labels and keyframe images M. i The same label in the corresponding single image content feature label is used as the keyframe image M. i Corresponding intersection tags;

[0056] The label classification subunit is also used to classify business semantic feature labels and keyframe images M i The corresponding single-image content feature labels are deduplicated to obtain the keyframe image M. i The corresponding union tag;

[0057] The coefficient calculation subunit is used to calculate the keyframe image M. i The total number of corresponding intersection labels and keyframe images M i Divide the total number of corresponding union labels to obtain the keyframe image M. i The corresponding image label repetition coefficient.

[0058] The aforementioned video data processing device further includes:

[0059] The display module is used to display at least two keyframe images and the single image content feature tags corresponding to each of the at least two keyframe images in the immersive review workbench if the matching result of the video resource for the business requirement script information is a successful match; the immersive review workbench is used by the review object to determine the review result of the video resource.

[0060] The display module includes:

[0061] Preprocessing unit, used for processing keyframe image M i In the corresponding single-image content feature label, it will be related to the keyframe image M i Labels with different intersection labels are used as keyframe images M. i Corresponding mismatched tags;

[0062] The preprocessing unit is also used to process the keyframe image M i Determine the keyframe image M at the playback time point in the video resource. i The order of presentation in at least two keyframe images;

[0063] Image display unit for displaying keyframe images M in the immersive review workbench according to the presentation order. i ;

[0064] Label display unit, used for displaying labels on keyframe images M i The corresponding text display area displays the keyframe image M in the first style. i The corresponding intersection labels are used to display the keyframe image M in the second style. i The corresponding mismatched tag.

[0065] The display module includes:

[0066] The display area determination unit is used to determine the area size corresponding to the video resource based on the image size corresponding to at least two keyframe images and the total number of at least two keyframe images.

[0067] The display area determination unit is also used to determine the available display area in the immersive review workbench, and to allocate a dynamic display area corresponding to the size of the area for the video resources in the available display area;

[0068] The display area determination unit is also used to display at least two keyframe images and single image content feature labels corresponding to the at least two keyframe images in the dynamic display area.

[0069] One embodiment of this application provides a computer device, including: a processor, a memory, and a network interface;

[0070] The processor is connected to the memory and the network interface. The network interface is used to provide a data communication network element, the memory is used to store a computer program, and the processor is used to call the computer program to execute the method in the embodiments of this application.

[0071] One aspect of this application provides a computer-readable storage medium storing a computer program adapted for loading by a processor and executing the methods described in this application.

[0072] One aspect of this application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described in this application.

[0073] In this embodiment, after obtaining the business requirement script information entered by the first object and the video resources uploaded by the second object in response to the business requirement script information, text tag extraction processing can be performed on the business requirement script information to obtain business script tags, wherein the business script tags include business attribute statistical tags and business semantic feature tags; simultaneously, video resource preprocessing can be performed on the video resources to obtain video resource attribute statistical tags, and then keyframe image extraction processing can be performed on the video resources to obtain at least two keyframe images, and image tag extraction processing can be performed on the at least two keyframe images to obtain image content feature tags; finally, if the business attribute statistical tags and the video resource attribute statistical tags match, the matching result of the video resources for the business requirement script information is determined based on the business semantic feature tags and the image content feature tags. Using the method provided in this application embodiment, the first step is to determine whether the attributes of the business requirement script information and the video resource match based on the business attribute statistical tags and the video resource attribute statistical tags. If the attributes match successfully, the semantics of the business requirement script information and the content of the video resource can be determined based on the business semantic feature tags and the image content feature tags. The above process does not require manual intervention, which can reduce the cost and time consumption of determining the matching result of the video resource for the business requirement script information. In addition, the extraction process of business semantic feature tags and image content feature tags is unified and standardized, which can ensure the consistency and accuracy of the extraction, thereby reducing the probability of missed judgments and misjudgments. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application;

[0076] Figure 2 This is a schematic diagram illustrating an application scenario of a video data processing method provided in an embodiment of this application;

[0077] Figure 3 This is a flowchart illustrating a video data processing method provided in an embodiment of this application;

[0078] Figure 4 This is a flowchart illustrating a text semantic feature extraction method provided in an embodiment of this application;

[0079] Figure 5This is a schematic diagram of the structure of a text semantic feature extraction model provided in an embodiment of this application;

[0080] Figure 6 This is a flowchart illustrating an image content feature extraction method provided in an embodiment of this application;

[0081] Figure 7 This is a schematic diagram of the structure of an image content feature extraction model provided in an embodiment of this application;

[0082] Figure 8 This is a flowchart illustrating a video data processing method provided in an embodiment of this application;

[0083] Figure 9 This is a schematic diagram of an immersive review workbench provided in an embodiment of this application;

[0084] Figure 10 This is a schematic diagram of the structure of a video data processing device provided in an embodiment of this application;

[0085] Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0086] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0087] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0088] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0089] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, and intelligent transportation technologies.

[0090] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close relationship with linguistic research. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0091] The solutions provided in this application involve technologies such as computer vision and natural language processing in artificial intelligence, and are specifically illustrated through the following embodiments:

[0092] Please refer to [the website / information] first. Figure 1 , Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application. For example... Figure 1As shown, the network architecture may include a backend server 100 and a cluster of terminal devices. The cluster of terminal devices may include terminal devices 10a, 10b, 10c, ..., 10n. Each terminal device in the cluster may have a communication connection with the backend server 100. For example, terminal device 10a may have a communication connection with the backend server 100, terminal device 10b may have a communication connection with the backend server 100, and terminal device 10c may have a communication connection with the backend server 100. The communication connection is not limited to a specific method. It may be a direct or indirect connection via wired communication, a direct or indirect connection via wireless communication, or other methods. This application does not impose any restrictions on these methods.

[0093] It should be understood that, such as Figure 1 Each terminal device in the terminal device cluster shown can have an application client installed. When the application client runs on each terminal device, it can interact with the aforementioned... Figure 1 Data interaction occurs between the backend servers 100 shown. The application client can be a game application, video editing application, social application, instant messaging application, live streaming application, short video application, video application, music application, shopping application, novel application, payment application, browser, or any other application client capable of displaying text, images, audio, and video data. This application client can be a standalone client or an embedded sub-client integrated into another client (such as an instant messaging client, social client, video client, etc.); this is not limited here.

[0094] like Figure 1As shown, any terminal device in the terminal device cluster, associated with a first object, can send a video resource customization request carrying business requirement script information to the backend server 100 by running the application client. The backend server 100 can respond to the video resource customization request, obtain the business requirement script information, and push the business requirement script information to the application client of each terminal device in the aforementioned terminal device cluster for display. The business requirement script information describes the attributes, scenes, and plot of the video resource that the first object wishes to customize. The video resource can refer to a resource that disseminates product or service information to an object in the form of video. Any terminal device in the aforementioned terminal device cluster, associated with a second object, can respond to the business requirement script information by running the application client, upload the corresponding video resource to the backend server 100, and then the backend server 100 sends it to the terminal device associated with the first object. It can be understood that a business requirement script information published by the first object may be responded to by multiple second objects, and the first object may receive multiple video resources, but the first object may ultimately only select one or a portion of the received video resources for promotion. Among the video resources received by the first target, there is a high possibility that some video resources may not match the business requirement script information. If the video resources are not matched and filtered first, the first target will waste a lot of time when selecting video resources for promotion. Therefore, after receiving the video resources corresponding to the business requirement script information entered by the first target, the backend server 100 can first determine the matching result of the video resources with respect to the business requirement script information. If the matching result of the video resources with respect to the business requirement script information is a failure, the backend server 100 can directly filter out the video resources without pushing them to the terminal devices associated with the first target, thus saving the first target the browsing time of video resources that do not match the business requirement script information.

[0095] After obtaining the business requirement script information entered by the first object and the video resources uploaded by the second object, the backend server 100 can perform text tag extraction processing on the business requirement script information to obtain business script tags, which include business attribute statistical tags and business semantic feature tags. Then, it can perform video resource preprocessing to obtain video resource attribute statistical tags, and perform keyframe image extraction processing on the video resources to obtain at least two keyframe images. Then, it can perform image tag extraction processing on the at least two keyframe images to obtain image content feature tags. If the business attribute statistical tags and the video resource attribute statistical tags match, the backend server 100 can determine the matching result of the video resources for the business requirement script information based on the business semantic feature tags and the image content feature tags.

[0096] To facilitate understanding of the above process, please refer to [the relevant documentation / reference]. Figure 2 , Figure 2 This is a schematic diagram illustrating an application scenario of a video data processing method provided in an embodiment of this application. Wherein, as... Figure 2 The terminal device 20a or terminal device 20b shown can be the one described above. Figure 1 Any terminal device in the terminal device cluster shown, for example, terminal device 20a can be terminal device 10a, and terminal device 20b can be terminal device 10b; for example Figure 2 The backend server 200 shown can be the one described above. Figure 2 The backend server shown is 100.

[0097] like Figure 2As shown, both terminal devices 20a and 20b integrate a resource customization application 21. This application provides video resource customization functionality for objects. A first object without video resource creation ideas can initiate a video resource customization request through the application 21, while a second object with video resource creation ideas can respond to the request and upload video resources. Suppose object A, associated with terminal device 20a, wants to promote financial product C, but doesn't know how to shoot suitable video resources. Object A can input business requirement script information 22 through the resource customization application 21 running on terminal device 20a. This script information 22 describes the attributes, scenes, and plot of the video resources that object A wants to customize. The script information 22 may include a business requirement scene description text, business requirement script keywords, and basic business requirement script information. The business requirement scene description text briefly describes the scene or plot of the video resources that object A wants to customize. For example, the business requirement scene... The description text can be: "Two women and one man discuss financial management methods. All three are urban white-collar workers. Both are dressed formally, have good looks and temperaments. The man shares his financial management experience with the two women, how to obtain free financial management solutions, and describes his own financial returns." The business requirement script keywords summarize the type and characteristics of the video resources that Object A wants to customize. For example, the business requirement script keywords could be: "white-collar, free, financial management, returns, modern, temperament, image, metropolis." The business requirement script basic information describes the basic information of the video resources that Object A wants to customize. For example, the business requirement script basic information can include the number of characters, video duration, etc. After Object A enters the business requirement script information 22, the terminal device 20a will send a video resource customization request carrying this business requirement script information 22 to the backend server 200. The backend server 200 can distribute the business requirement script information 22 to other terminal devices that have integrated and installed the resource customization application 21. Assuming that object B, which is associated with terminal device 20b, sees the business requirement script information 22 through the resource customization application 21 running on terminal device 20b, object B can shoot video resource 23 according to the business requirement script information 22 and upload the video resource 23 through the resource customization application 21 running on terminal device 20b. Terminal device 20b will then send the video resource 23 to the backend server 200.

[0098] It should be understood that the video resources that object A wants to customize should naturally match the business requirement script information 22. However, it is difficult to avoid the occurrence of video resources that do not match the business requirement script information 22 among the video resources uploaded in response to it. If the backend server 22 directly pushes all received video resources associated with the business requirement script information 22 to the terminal device 20a, object A will need to spend a lot of time and effort to select suitable video resources. Therefore, the backend server 200 can first determine the matching result of the received video resources with respect to the associated business requirement script information.

[0099] like Figure 2 As shown, the backend server 200 obtains the business requirement script information 22, and then performs text tag extraction processing on the business requirement script information 22 to obtain business attribute statistical tags 241 and business semantic feature tags 244. Among them, such as... Figure 2 As shown, a specific process by which the backend server 200 extracts text tags from the business requirement script information 22 to obtain business attribute statistical tags 241 and business semantic feature tags 244 can be described as follows: The backend server 200 performs structured attribute statistical processing on the business requirement script information 22 to obtain business attribute statistical tags 241. These tags describe the attribute information of the video resources to be customized, such as duration tags, number of characters tags, etc. The backend server 200 can call the text semantic feature model 201 to perform unstructured semantic feature extraction on the business requirement scenario description text contained in the business requirement script information 22, obtaining a first semantic feature tag 242 that describes the semantic features of the business requirement scenario description text. Furthermore, the backend server 200 can match the business requirement script keywords contained in the business requirement script information 22 with tags in the tag library, using the tags matching the business requirement script keywords as second semantic feature tags 243. The backend server 200 then performs deduplication and aggregation processing on the first semantic feature tag 242 and the second semantic feature tag 243 to obtain the business semantic feature tag 244.

[0100] like Figure 2As shown, after obtaining video resource 23, the backend server 200 needs to perform video preprocessing on video resource 23, that is, to perform data reading and parsing processing on video resource 23 to obtain video resource attribute statistical labels 261 corresponding to video resource 23, which are labels used to describe the attribute information of video resource 23. After the video preprocessing is completed, the backend server 200 can perform keyframe image extraction processing on video resource 23 in the labeled format to obtain the images corresponding to keyframes (e.g., transition frames, time frames, etc.) of video resource 23, assuming keyframe images 25a, ..., keyframe images 25n are obtained. The backend server 200 can call the image content feature extraction model 202 to perform image label extraction processing on keyframe images 25a, ..., keyframe images 25n respectively to obtain single image content feature labels 262a, ..., and single image content feature labels 262n corresponding to keyframe image 25a and keyframe image 25n respectively.

[0101] like Figure 2 As shown, the backend server first performs attribute matching based on business attribute statistical label 241 and video resource attribute statistical label 261, that is, to determine whether the attributes of video resource 23 are the same as the attributes of the video resource desired by business requirement script information 22. If the backend server 200 determines that the attribute matching is successful, it can further perform feature matching based on business semantic feature label 244 and image content feature label (i.e., the above-mentioned single image content feature label 262a, ..., single image content feature label 262n), that is, to determine the label repetition rate between the two. When the label repetition rate is higher than the specified threshold, the backend server 200 can determine that the matching result of video resource 23 for business requirement script information 22 is a successful match result. At this time, the backend server 200 can send video resource 23 to terminal device 20a. Otherwise, it is a matching failure result. The backend server 200 can filter out video resource 23 and send a matching failure notification message to terminal device 20b.

[0102] Optionally, if the terminal device 20a has a text semantic feature model 201 and an image content feature extraction model 202 stored locally, the process of determining the matching result of the video resource 23 for the business requirement script information 22 can be executed in the terminal device 20a. Since the training of the text semantic feature model 201 and the image content feature extraction model 202 involves a large amount of offline computation, the text semantic feature model 201 and the image content feature extraction model 202 can be trained by the backend server 200 and then sent to the terminal device 20a.

[0103] It is understood that the video data processing method provided in this application embodiment can be executed by a computer device, which includes, but is not limited to, the aforementioned backend server 100 or terminal devices in a terminal device cluster. The aforementioned server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The aforementioned terminal can be a smartphone, tablet computer, laptop computer, desktop computer, intelligent voice interaction device, smart home appliance, vehicle terminal, etc., but is not limited to these.

[0104] It is understood that the embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving.

[0105] It is understood that, in the specific embodiments of this application, the business requirement script information, video resources and other related data involved need to obtain user permission or consent when the above embodiments of this application are applied to specific products or technologies, and the collection, use and processing of related data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0106] Further, please see Figure 3 , Figure 3 This is a flowchart illustrating a video data processing method provided in an embodiment of this application. The method can be executed by a computer device, which can be the aforementioned... Figure 1 The backend server 100 shown can also be the one mentioned above. Figure 1 Any terminal device in the terminal device cluster shown, such as terminal device 10c. The following description assumes this method is executed by a computer device, wherein the video data processing method may include at least the following steps S101-S105:

[0107] Step S101: Obtain the business requirement script information entered by the first object and obtain the video resources uploaded by the second object; the video resources are resources uploaded in response to the business requirement script information.

[0108] Specifically, the first object can be considered as the material demander, that is, the object that needs to customize video resources; the second object can be considered as the material supplier, that is, the object that provides video resources to the first object.

[0109] Specifically, the first entity can input business requirement script information through a connected terminal device. This terminal device may include a script input module, which comprises a basic information inputter, a scene description inputter, and a keyword inputter. The basic information inputter generates basic business requirement script information in response to the first entity's input; the scene description inputter generates a business requirement scene description text in response to the first entity's input; and the keyword inputter generates business requirement script keywords in response to the first entity's input. The terminal device associated with the first entity then sends the basic business requirement script information, the business requirement scene description text, and the business requirement script keywords as business requirement script information to a computer device. The computer device can first store the business requirement script information in a script storage module and then wait for a second entity to upload video resources in response to the business requirement script information. The script storage module can store multiple business requirement scripts uploaded by the first entity.

[0110] Specifically, the second entity can input video resources through a connected terminal device. This terminal device may include a video input module, which may contain a video uploader and a video compressor / transcoder. The video uploader provides video resource upload functionality; the video compressor / transcoder compresses and transcodes the initial video resources uploaded by the second entity to obtain video resources in a standard playback format. Then, the terminal device associated with the second entity uploads the video resources to a computer device, which stores the video resources in a video storage module. This video storage module can store multiple video resources uploaded in response to different business requirement scripts.

[0111] Specifically, the computer device can perform preliminary association matching between the business requirement script information in the script storage module and the video resources in the video resource storage module at a fixed frequency. This involves associating a business requirement script with the video resources uploaded in response to that script in the material storage module, and then waiting for matching. Therefore, the computer device can retrieve the business requirement script information entered by the first object from the material storage module, and simultaneously retrieve the video resources uploaded by the second object.

[0112] Step S102: Extract text tags from the business requirement script information to obtain business script tags; the business script tags include business attribute statistics tags and business semantic feature tags.

[0113] Specifically, as shown in step S101 above, the business requirement script information may include business requirement scenario description text, business requirement script keywords, and basic business requirement script information. A feasible implementation process for extracting text tags from the business requirement script information to obtain business script tags can be as follows: perform structured attribute statistical processing on the business requirement scenario description text, business requirement script keywords, and basic business requirement script information to obtain business attribute statistical tags; perform unstructured semantic feature extraction processing on the business requirement scenario description text to obtain first semantic feature tags; match the business requirement script keywords with tags in the tag library, and use the tags matching the business requirement script keywords as second semantic feature tags; and use the first and second semantic feature tags as business semantic feature tags.

[0114] Specifically, business attribute statistical tags are tags used to describe the attribute information of video resources that match the business requirement script information. The attribute types of business attribute statistical tags can be video duration, number of characters, etc. When computer equipment performs structured attribute statistical processing, the attribute types of the business attribute statistical tags that need to be statistically analyzed can be set in advance. Then, based on the set attribute types, structured data searches and statistics are performed on the business requirement scenario description text, business requirement script keywords, and basic information of the business requirement script to determine the business attribute statistical tags corresponding to the set attribute types.

[0115] Specifically, business requirement scenario description text is typically sentence text, and often unstructured. Therefore, it requires unstructured semantic feature extraction to obtain a first semantic feature label that can summarize its semantic features. Business requirement script keywords are essentially words used to describe the semantic features of the business requirement scenario description text. However, business requirement script keywords are not necessarily tags stored in a tag library. Therefore, tags matching the business requirement script keywords in the tag library can be used as second semantic feature labels. The tags in the tag library are standard words trained based on a large amount of sample business requirement script information; the first semantic feature label is also actually a tag from the tag library.

[0116] Specifically, a feasible implementation process for extracting unstructured semantic features from the business requirement scenario description text to obtain the first semantic feature label can be as follows: The business requirement scenario description text is segmented to obtain a set of words to be processed; the set of words to be processed includes one or more words to be processed; then, a text semantic feature extraction model is called to extract unstructured semantic features from the set of words to be processed to obtain the first semantic feature label. Here, the business requirement scenario description text is a sentence text, which can be understood as a sequence composed of one or more words. Each word is the basic unit of the sentence text, and therefore, the semantic information of each word is very important. The segmentation process is the process of recombinizing a continuous sequence of characters into a sequence of words according to certain rules. Rule-based segmentation methods can be used to segment the business requirement scenario description text, mainly by pre-establishing a lexicon, also called a dictionary or word dictionary, and segmenting it through dictionary matching; alternatively, segmentation tools or other methods can be used, without limitation here. The text semantic feature model can be trained based on the business requirement scenario description samples and an initial text model capable of extracting text semantic features. The initial text model can be an LSTM (Long-Short Term Memory) model, which is a special type of recurrent neural network that can effectively model the contextual dependency information of text.

[0117] Step S103: Perform video resource preprocessing on the video resources to obtain video resource attribute statistical tags.

[0118] Specifically, video resource attribute statistics tags are tags used to describe the attribute information of video resources. Computer equipment can perform structured data reading and parsing processing on video resources, and then determine the corresponding video resource attribute statistics tags based on pre-set attribute types. The attribute type of the video resource attribute statistics tags should be the same as the aforementioned business attribute statistics tags; that is, it can also be video duration, number of characters, etc. When the video resource attribute statistics tags and business attribute statistics tags corresponding to the same attribute type are the same, it indicates that the video resource attribute statistics tags and business attribute statistics tags match. This means that the attribute information of the video resource meets the requirements, and feature matching can be performed on the video resource to determine the matching result of the video resource for the business requirement script information.

[0119] Optionally, a feasible implementation process for the computer device to determine whether the video resource attribute statistical label matches the aforementioned business attribute statistical label can be as follows: Assume the matching business attribute statistical labels include S business attribute statistical sub-labels, where S is a positive integer; the computer device can traverse the S business attribute statistical sub-labels and sequentially obtain the k-th business attribute statistical sub-label, where k is a positive integer less than or equal to S; if the target attribute statistical label does not exist in the video resource attribute statistical labels, then it is determined that the business attribute statistical label and the video resource attribute statistical label do not match, where the target attribute statistical label refers to the attribute corresponding to the k-th business attribute statistical sub-label. Tags of the same type are considered. If a target attribute statistical tag exists in the video resource attribute statistical tags, and the k-th business attribute statistical sub-tag is different from the target attribute statistical tag, then the business attribute statistical tags and video resource attribute statistical tags are determined to be mismatched. If a target attribute statistical tag exists in the video resource attribute statistical tags, and the k-th business attribute statistical sub-tag is the same as the target attribute statistical tag, then the k-th business attribute statistical sub-tag is determined to be a match between the video resource attribute statistical tags. If every business attribute statistical sub-tag matches the video resource attribute statistical tags, then the business attribute statistical tags and video resource attribute statistical tags are determined to be a match. In short, assuming that the attribute type corresponding to the business attribute statistical tag includes video duration, number of roles, and scene type, then the attribute type corresponding to the video resource attribute statistical tag should also include video duration, number of roles, and scene type. Furthermore, business attribute statistical tags and video resource attribute statistical tags with the same attribute type are identical. For example, if the business attribute statistical tag with the attribute type "video duration" is identical to the video resource attribute statistical tag with the attribute type "video duration," it indicates that the duration of the video resource is consistent with the duration required in the business requirement script information, and this type of attribute meets the requirements.

[0120] Step S104: Perform keyframe image extraction processing on the video resource to obtain at least two keyframe images, and perform image tag extraction processing on the at least two keyframe images to obtain image content feature tags.

[0121] Specifically, when computer equipment performs keyframe image extraction processing on video resources, it can crop the keyframes of the video resources to obtain keyframes such as transition frames and time frames, and then obtain the images corresponding to the keyframes, thus obtaining at least two keyframe images.

[0122] Specifically, assume that at least two keyframe images include keyframe image M. i , where i is a positive integer less than or equal to the total number of at least two keyframe images, for keyframe images M i Image label extraction processing can yield keyframe image M. iThe corresponding single-image content feature labels. After obtaining the single-image content feature labels for each keyframe image, each single-image content feature label can be used as the image content feature label. The single-image content feature label describes the content features of the image; for example, the single-image content feature label for a keyframe image could be "finance," "live streaming," or "commerce."

[0123] Specifically, for keyframe image M i A feasible implementation process for image label extraction can be as follows: For keyframe image M... i Matrix representation processing is performed to obtain the keyframe image M. i The image representation matrix is ​​then used, and the image content feature extraction model is called to extract features from the keyframe image M. i Image label extraction is performed on the image representation matrix to obtain the keyframe image M. i The corresponding single-image content feature labels. Matrix representation processing refers to processing the keyframe image M... i Converting to a matrix representation is necessary because computers cannot recognize images. Therefore, the image needs to be converted into a form that computers can understand. Computers are well-suited for matrix operations, so the keyframe image M can be represented as a matrix. i This is converted into a matrix representation because all images are composed of red, green, and blue (RGB) superpositions. That is, each pixel in an image can be represented by RGB; for example, white can be represented as RGB(255, 255, 255). An image can be described using a three-dimensional RGB matrix. Therefore, the keyframe image M... i The corresponding image representation matrix can be the keyframe image M. i The corresponding RGB three-matrix. The image content feature extraction model can be trained based on keyframe image samples and an initial image model capable of image content feature recognition and extraction. The initial image model can be a DETR (DEtection Transformer, a target detection method based on transformers (a type of neural network)) model, which can achieve the recognition and extraction of single image content feature labels.

[0124] Step S105: If the business attribute statistical label and the video resource attribute statistical label match, then determine the matching result of the video resource for the business requirement script information based on the business semantic feature label and the image content feature label.

[0125] Specifically, based on the business semantic feature labels and the keyframe image M i The corresponding single-image content feature labels are used to determine the keyframe image M. iThe corresponding image label repetition coefficient is used; the image label repetition coefficients corresponding to the at least two keyframe images are averaged to obtain the resource label repetition coefficient corresponding to the video resource; if the resource label repetition coefficient is less than the resource label repetition threshold, the matching result of the video resource for the business requirement script information is determined to be a matching failure; if the resource label repetition coefficient is greater than or equal to the resource label repetition threshold, the matching result of the video resource for the business requirement script information is determined to be a matching success. The image label repetition coefficient can be selected from Jacard similarity, also known as the Jaccard similarity coefficient, which is used to compare the similarity and differences between a finite set of samples. The larger the Jaccard coefficient value, the higher the sample similarity. The resource label repetition coefficient is essentially the average image label repetition coefficient.

[0126] Specifically, if the image label repetition coefficient is the Jaccard similarity coefficient, then based on the business semantic feature labels and keyframe images M... i The corresponding single-image content feature labels determine the keyframe image M. i A feasible implementation process for the corresponding image label repetition coefficient can be as follows: The business semantic feature labels and keyframe images M... i The same label in the corresponding single image content feature label is used as the keyframe image M. i The corresponding intersection labels; the business semantic feature labels and keyframe images M i The corresponding single-image content feature labels are deduplicated to obtain the keyframe image M. i Corresponding union labels; keyframe image M i The total number of corresponding intersection labels and keyframe images M i Divide the total number of corresponding union labels to obtain the keyframe image M. i The corresponding image label repetition coefficient. The process of determining the above image label repetition coefficient can be expressed based on the following formula (1):

[0127]

[0128] Where J(A,B) is the keyframe image M i The corresponding image label repetition coefficient, where A is the business semantic feature label and B is the keyframe image M. i The corresponding single-image content feature label.

[0129] Using the method provided in this application embodiment, the first step is to determine whether the attributes of the business requirement script information and the video resource match based on the business attribute statistical tags and the video resource attribute statistical tags. If the attributes match successfully, the semantics of the business requirement script information and the content of the video resource can be determined based on the business semantic feature tags and the image content feature tags. The above process does not require manual intervention, which can reduce the cost and time consumption of determining the matching result of the video resource for the business requirement script information. In addition, the extraction process of business semantic feature tags and image content feature tags is unified and standardized, which can ensure the consistency and accuracy of the extraction, thereby reducing the probability of missed judgments and misjudgments.

[0130] Furthermore, the text semantic feature extraction model described in step S102 above may include a first embedding layer, a memory network layer, and a normalized classification layer. In this case, the computer device can use the first embedding layer, memory network layer, and normalized classification layer of the text semantic feature extraction model to perform unstructured semantic feature extraction processing on the aforementioned business requirement scenario description text, thereby obtaining the first semantic feature. For easier understanding, please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a flowchart illustrating a text semantic feature extraction method provided in an embodiment of this application. The method is a specific implementation of the unstructured semantic feature extraction processing of the business requirement scenario description text described in step S102 above to obtain a first semantic feature label. This method can be executed by a computer device, wherein the computer device can be the aforementioned... Figure 1 The backend server 100 shown can also be the one mentioned above. Figure 1 Any terminal device in the terminal device cluster shown, such as terminal device 10c. The following description assumes this method is executed by a computer device, wherein the text semantic feature extraction method may include at least the following steps S201-S204:

[0131] Step S201: Perform word segmentation on the business requirement scenario description text to obtain a set of words to be processed; the set of words to be processed includes one or more words to be processed.

[0132] Specifically, the implementation process of step S201 can be found in the description of step S102 above, and will not be repeated here.

[0133] Step S202: Call the first embedding layer of the text semantic feature extraction model to perform embedding feature processing on each word to be processed, and obtain the word embedding vector corresponding to each word to be processed.

[0134] Specifically, the first embedding layer, or embedding layer, uses a corpus stored in the computer device to generate word embedding vectors corresponding to the words to be processed, based on the index of the words in the corpus. The dimension of this word embedding vector can be set according to the actual situation, but it is usually 1024 dimensions.

[0135] Step S203: Identify the sorting position of each word to be processed in the set of words to be processed, and call the memory network layer of the text semantic feature extraction model to perform hidden layer feature representation processing on the word embedding vector corresponding to each word to be processed according to the sorting position, so as to obtain the hidden layer representation vector corresponding to the set of words to be processed.

[0136] Specifically, the memory network layer contains Z memory units, where Z is a positive integer greater than or equal to the number of words to be processed. The computer algorithm sequentially inputs the word embedding vectors corresponding to the words to be processed into different memory units based on their order of appearance in the word set. That is, the word embedding vector corresponding to the first word in the word set is input into the first memory unit, the word embedding vector corresponding to the second word is input into the second memory unit, and so on, until all words in the word set have been input into memory units. Each memory unit concatenates the input word embedding vector with the hidden layer vector output from the previous memory unit to obtain the processing vector. Then, it performs a gate transformation on the processing vector, that is, it calculates the processing vector using a pre-constructed long short-term periodic gate function to obtain the output hidden layer vector. It should be noted that the first memory unit does not have a previous memory unit, so it concatenates the input word embedding vector with a zero vector of the same dimension to obtain the processing vector. The memory unit input to the last word in the word set to be processed is the tail memory unit. The hidden layer vector corresponding to the word set to be processed can be determined by its output hidden layer vector.

[0137] Step S204: The normalized classification layer of the text semantic feature extraction model is invoked to normalize the hidden layer representation vector to obtain the text label prediction distribution corresponding to the word to be processed. The first semantic feature label corresponding to the business requirement scenario description text is determined according to the text label prediction distribution. The text label prediction distribution contains the probability value of the business requirement scenario description text matching each label in the label library.

[0138] Specifically, since the text label prediction distribution contains the probability values ​​of matching the business requirement scenario description text with each label in the label library, the labels with probability values ​​greater than the matching threshold can be used as the first semantic feature labels corresponding to the business requirement scenario description text.

[0139] For ease of understanding, please refer to the following: Figure 5 , Figure 5 This is a schematic diagram of the structure of a text semantic feature extraction model provided in an embodiment of this application. For example... Figure 5 As shown, the text semantic feature extraction model 5 can be constructed based on the LSTM model. The text semantic feature extraction model 5 includes a first embedding layer 51, a memory network layer 52, and a normalization layer 53. The computer device first inputs the words to be processed from the word set into the first embedding layer 51, i.e., the embedding layer, in order. Figure 5 As shown, the computer device inputs word 1, word 2, ..., word t into the first embedding layer 51. The first embedding layer 51 will sequentially input word embedding vector x1, word embedding vector x2, ..., word embedding vector xt. t In this context, each word embedding vector has the same dimension, which can be assumed to be 1024. Then, the computer device sequentially inputs word embedding vector x1 into memory unit 52a of memory network layer 52, word embedding vector x2 into memory unit 52b of memory network layer 52, ..., word embedding vector x... t Input memory units 52t in memory network layer 52. For example... Figure 5 As shown, the hidden vector output by each memory unit is used as the input to the next memory unit, and is concatenated with the word embedding vector input in the next memory unit to obtain the vector to be processed in the next memory unit. However, for memory unit 52a, since it is the first memory unit in the memory network layer 52, it selects a zero vector with the same dimension as the word embedding vector x1, i.e., a 1024-dimensional zero vector h0, and concatenates it with the word embedding vector x1 to obtain a 2048-dimensional vector to be processed. Subsequently, memory unit 52a can perform a gate transformation on the vector to be processed to obtain a 1024-dimensional hidden vector h1. This hidden vector h1 is input into memory unit 52b and concatenated with the word embedding vector x2, and so on, finally obtaining the tail memory unit, which is the hidden vector h input to memory unit 52t. t According to the hidden layer vector h t This allows us to determine the hidden layer representation vector y corresponding to the word set to be processed. t .

[0140] The aforementioned gate transformation is actually calculated using a pre-constructed long-short-term periodic gate function on the vector to be processed. Taking memory unit 52t as an example, the input of memory unit 52t is the word embedding vector x. t And the hidden layer vector h output by the previous memory unit t-1 .

[0141] Therefore, the vector to be processed can be denoted as By passing the vector to be processed through four hidden layer matrices and four gate functions, four variables can be obtained. For specific implementation details, please refer to the following calculation formula:

[0142]

[0143]

[0144]

[0145]

[0146] Among them, z, z i z f and z o That is, the variables W, W' in the four gate transformation process. i W f and W o Given four hidden layer matrices, after obtaining the four variables, performing addition and multiplication operations on them yields the hidden layer vector h. t Furthermore, the hidden layer representation vector y is obtained. t For a detailed implementation, please refer to the following formula:

[0147] c t =z f ⊙c t-1 +z i ⊙z formula (6)

[0148] h t =z o tanh( t ) Formula (7)

[0149] y t =σ(W′h t ) Formula (8)

[0150] Where W′ is also a hidden layer matrix, c t It is an intermediate variable, c t-1 It can be determined in the memory unit preceding memory unit 52t.

[0151] like Figure 5 As shown, y t By feeding the data into the normalization (Softmax) layer, it can be mapped to the tag library dimension and probability normalized to obtain the text tag prediction distribution. The mathematical representation is as follows:

[0152]

[0153] according to Then the first semantic feature label can be determined as label 1, label 2, ..., label n.

[0154] Using the method provided in the embodiments of this application, semantic features of text describing business requirements scenarios can be extracted quickly through a text semantic feature extraction model, which can speed up the extraction time and improve accuracy.

[0155] Furthermore, the image content feature extraction model described in step S104 above may include a second embedding layer, an encoding layer, a decoding layer, and a feedforward neural network classification layer. In this case, the computer device can use the second embedding layer, encoding layer, decoding layer, and feedforward neural network classification layer of the image content feature extraction model to perform image label extraction processing on each keyframe image, obtaining a single image content feature label corresponding to each keyframe image. For easier understanding, please refer to [link to relevant documentation]. Figure 6 , Figure 6 This is a flowchart illustrating an image content feature extraction method provided in an embodiment of this application. The method involves the keyframe image M described in step S104 above. i Image label extraction is performed to obtain keyframe image M. i A specific implementation method for corresponding single-image content feature labels, which can be executed by a computer device, wherein the computer device can be the aforementioned Figure 1 The backend server 100 shown can also be the one mentioned above. Figure 1 Any terminal device in the terminal device cluster shown, such as terminal device 10c. The following description assumes this method is executed by a computer device, wherein the text semantic feature extraction method may include at least the following steps S301-S303:

[0156] Step S301: Call the second embedding layer of the image content feature extraction model to process the keyframe image M. i The image representation matrix is ​​processed by convolution vector representation to obtain the image representation vector.

[0157] Specifically, the second embedding layer can adopt the backbone structure of CNN (Convolutional Neural Networks). The backbone structure can be understood as a preprocessor or a feature extractor. In the second embedding layer using the backbone structure, the computer device processes the keyframe image M... i The image representation matrix is ​​convolved with a CNN to obtain an initial image representation vector. Subsequently, the computer device can add position encoding to the initial image representation vector, that is, add the initial image representation vector to the position encoding of the same dimension, finally obtaining the final image representation vector. For easier understanding, please refer to [link to previous documentation]. Figure 7 , Figure 7 This is a schematic diagram of the structure of an image content feature extraction model provided in an embodiment of this application. For example... Figure 7 As shown, the image content feature extraction model includes a second embedding layer. In the second embedding layer, the computer device performs CNN convolution processing on the input image representation matrix 71 to obtain an initial image representation vector 72. Subsequently, the computer device adds the position encoding to the initial image representation vector 72 to obtain an image representation vector 73.

[0158] Step S302: The encoding layer of the image content feature extraction model is called to encode the image representation vector to obtain the image encoded representation vector; the decoding layer of the image content feature extraction model is called to decode the image encoded vector to obtain the image decoded representation vector.

[0159] Specifically, the encoding and decoding layers are designed based on the transformer's encoder-decoder framework. Simply put, the encoder-decoder framework converts an image representation vector into another form of image decoding representation vector, while maintaining the same semantic or content features. Encoding can be understood as the brain seeing an image and reading its information, while decoding can be understood as the brain reconstructing that information into another image.

[0160] Specifically, in the coding layer, the computer device can perform an alpha transformation, or dimension adaptation, on the image representation vector, converting it into three vectors: Q, K, and V. Then, iterative addition and multiplication are performed on these three vectors to obtain the image coded representation vector. This image coded representation vector is then input into the decoding layer. For easier understanding, please refer to [link to previous section]. Figure 7 ,like Figure 7 As shown, after image representation vector 73 is input into the encoding layer for encoding, the encoding layer outputs image encoded representation vector 74, which then enters the decoding layer. Simultaneously, the computer device inputs n (n is a positive integer) random vectors (object queries) into the decoding layer, namely random vector 75a, random vector 75b, ..., random vector 75n. These n vectors are randomly initialized and then trained. Each random vector carries some information; for example, random vector 75a may carry information about "what object is in the lower left corner of the image." The encoding layer then decodes image encoded representation vector 74 based on the information carried by random vector 75a, thereby obtaining the corresponding image decoded representation vector. This image decoded representation vector can be used to represent the object information in the lower left corner of the image. The number of random vectors determines the number of image decoded representation vectors output by the encoding layer.

[0161] Step S303: The feedforward neural network classification layer of the image content feature extraction model is invoked to classify the image decoded representation vector to obtain the keyframe image M. i The corresponding image label prediction distribution is used to determine the keyframe image M. i The corresponding single-image content feature labels; the image label prediction distribution includes the keyframe image M i The probability value of matching each tag in the tag library.

[0162] Specifically, an image decoding representation vector is input into a feedforward neural network classification layer, which outputs a corresponding image label prediction distribution. Based on this image label prediction distribution, the label corresponding to the image decoding representation vector can be determined. The labels corresponding to all image decoding representation vectors are represented by the keyframe image M. i The corresponding single-image content feature label.

[0163] The method provided in this application embodiment can accurately extract image content features from keyframe images, thereby accelerating extraction time and improving accuracy.

[0164] Further, please see Figure 8 , Figure 8 This is a flowchart illustrating a video data processing method provided in an embodiment of this application. The method can be executed by a computer device, which can be the aforementioned... Figure 1 The backend server 100 shown can also be the one mentioned above. Figure 1 Any terminal device in the terminal device cluster shown, such as terminal device 10c. The following description assumes this method is executed by a computer device, wherein the video data processing method may include at least the following steps S401-S406:

[0165] Step S401: Obtain the business requirement script information entered by the first object, and obtain the video resources uploaded by the second object; the video resources are resources uploaded in response to the business requirement script information.

[0166] Step S402: Extract text tags from the business requirement script information to obtain business script tags; the business script tags include business attribute statistics tags and business semantic feature tags.

[0167] Step S403: Perform video resource preprocessing on the video resources to obtain video resource attribute statistical tags;

[0168] Step S404: Perform keyframe image extraction processing on the video resource to obtain at least two keyframe images; perform image tag extraction processing on the at least two keyframe images to obtain image content feature tags; the image content feature tags include single image content feature tags corresponding to the at least two keyframe images respectively.

[0169] Step S405: If the business attribute statistical label and the video resource attribute statistical label match, then determine the matching result of the video resource for the business requirement script information based on the business semantic feature label and the image content feature label.

[0170] Specifically, the implementation process of steps S401-S405 can be found above. Figure 3 The descriptions of steps S101-S105 in the corresponding embodiments will not be repeated here.

[0171] Step S406: If the matching result of the video resource for the business requirement script information is a successful match, then the at least two keyframe images and the single image content feature tags corresponding to the at least two keyframe images are displayed in the immersive review workbench; the immersive review workbench is used by the review object to determine the review result of the video resource.

[0172] Specifically, assume that at least two keyframe images include keyframe image M. i If i is a positive integer less than or equal to the total number of the at least two keyframe images, then a feasible implementation process for displaying at least two keyframe images and their corresponding single-image content feature tags in the immersive review workbench can be: In keyframe image M... i In the corresponding single-image content feature label, it will be related to the keyframe image M i Labels with different intersection labels are used as keyframe images M. i Corresponding mismatched labels; based on keyframe image M i Determine the keyframe image M at the playback time point in the video resource. i The order in which keyframe images are displayed; the keyframe images M are displayed in the immersive review workbench according to the display order. i In keyframe image M i The corresponding text display area displays the keyframe image M in the first style. i The corresponding intersection labels are used to display the keyframe image M in the second style. i The corresponding mismatched tag.

[0173] Specifically, a feasible implementation process for displaying at least two keyframe images and their corresponding single-image content feature tags in an immersive review workbench can be as follows: determining the area size corresponding to the video resource based on the image sizes corresponding to the at least two keyframe images and the total number of the at least two keyframe images; determining the available display area in the immersive review workbench, and allocating a dynamic display area corresponding to the area size for the video resource within the available display area; and displaying the at least two keyframe images and their corresponding single-image content feature tags in the dynamic display area.

[0174] For a better understanding of the above display method, please refer to [link / reference]. Figure 9 , Figure 9 This is a schematic diagram of an immersive review workbench provided in an embodiment of this application. Figure 9 As shown, the immersive review workbench can display one or more dynamic display areas. Each dynamic display area is dynamically generated based on the size and number of keyframe images corresponding to a video resource, as well as the area of ​​the currently available display space. In other words, the computer device can dynamically adjust the layout through scaling optimization algorithms to ensure full utilization of the page display space, allowing as many video resources as possible to be displayed within a single page of the immersive review workbench. A dynamic display area is used to display the tag matching status of a video resource. For example, the three keyframe images displayed in dynamic display area 91 are extracted from the same video resource and are sorted according to their playback time points within the video resource. The dynamic display area 91 can display the business semantic feature tags corresponding to the business requirement script information of the video resource, i.e., the A0 script tags. This helps the reviewer quickly understand the semantic features corresponding to the A0 script. Simultaneously, the dynamic display area 91 also displays the single image content feature tags corresponding to the keyframe images, and bolds the tags in the single image content feature tags that are identical to the A0 script tags (different colors can also be used; there are no restrictions here). In this way, the reviewer can quickly determine the review result of the video resource without viewing the entire video resource or reading the business script requirement information. Furthermore, if a keyframe image contains illegal or other issues, it can be detected quickly.

[0175] Using the method provided in this application embodiment, only the keyframe images of the video resources and the single image content feature tags corresponding to the keyframe images are displayed in the immersive review console. The matching tags and the unmatched tags are displayed in different styles, which can effectively prompt the reviewer on the matching degree between the current video resources and the recording script, as well as the content it presents, whether it can well express the main idea of ​​the recording script, and whether it can meet the creative production requirements of the video client.

[0176] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a video data processing apparatus provided in an embodiment of this application. The video data processing apparatus can be a computer program (including program code) running on a computer device; for example, the video data processing apparatus is an application software. The apparatus can be used to execute corresponding steps in the video data processing method provided in the embodiments of this application. Figure 10 As shown, the video data processing device 1 may include: an acquisition module 11, a first extraction module 12, a video preprocessing module 13, a second extraction module 14, and a first matching module 15.

[0177] The acquisition module 11 is used to acquire the business requirement script information entered by the first object and the video resources uploaded by the second object; the video resources are the resources uploaded in response to the business requirement script information.

[0178] The first extraction module 12 is used to perform text tag extraction processing on the business requirement script information to obtain business script tags; the business script tags include business attribute statistical tags and business semantic feature tags.

[0179] Video preprocessing module 13 is used to preprocess video resources to obtain video resource attribute statistical tags;

[0180] The second extraction module 14 is used to perform key frame image extraction processing on video resources to obtain at least two key frame images, and to perform image label extraction processing on the at least two key frame images to obtain image content feature labels.

[0181] The first matching module 15 is used to determine the matching result of the video resource for the business requirement script information based on the business semantic feature label and the image content feature label if the business attribute statistical label and the video resource attribute statistical label match.

[0182] The specific implementation methods of the acquisition module 11, the first extraction module 12, the video preprocessing module 13, the second extraction module 14, and the first matching module 15 can be found above. Figure 3 The specific descriptions of steps S101-S105 in the corresponding embodiments will not be repeated here.

[0183] The business requirement script information includes a description of the business requirement scenario, keywords for the business requirement script, and basic information about the business requirement script.

[0184] The first extraction module 12 includes: a first processing unit 121, a second processing unit 122, a third processing unit 123, and a fourth processing unit 124.

[0185] The first processing unit 121 is used to perform structured attribute statistical processing on the business requirement scenario description text, business requirement script keywords and basic information of business requirement script to obtain business attribute statistical tags.

[0186] The second processing unit 122 is used to perform unstructured semantic feature extraction processing on the business requirement scenario description text to obtain the first semantic feature label;

[0187] The third processing unit 123 is used to match the keywords of the business requirement script with the tags in the tag library, and to use the tags that match the keywords of the business requirement script as the second semantic feature tags.

[0188] The fourth processing unit 124 is used to use the first semantic feature label and the second semantic feature label as business semantic feature labels.

[0189] The specific implementation methods of the first processing unit 121, the second processing unit 122, the third processing unit 123, and the fourth processing unit 124 can be found above. Figure 3 The specific description of step S102 in the corresponding embodiment will not be repeated here.

[0190] The second processing unit 122 includes a word segmentation subunit 1221 and a model calling subunit 1222.

[0191] The word segmentation subunit 1221 is used to segment the text describing the business requirement scenario to obtain a set of words to be processed; the set of words to be processed includes one or more words to be processed.

[0192] Model calling subunit 1222 is used to call the text semantic feature extraction model to perform unstructured semantic feature extraction on the set of words to be processed, and obtain the first semantic feature label.

[0193] The text semantic feature extraction model includes a first embedding layer, a memory network layer, and a normalized classification layer. Specifically, the model calling subunit 1222 calls the first embedding layer to perform embedding feature processing on each word to be processed, obtaining the word embedding vector corresponding to each word. It then identifies the ranking position of each word in the word set, calls the memory network layer to perform hidden layer feature representation processing on the word embedding vector corresponding to each word according to its ranking position, obtaining the hidden layer representation vector corresponding to the word set. Finally, it calls the normalized classification layer to normalize the hidden layer representation vector, obtaining the text label prediction distribution corresponding to the word. Based on the text label prediction distribution, it determines the first semantic feature label corresponding to the business requirement scenario description text. The text label prediction distribution contains the probability value of matching each label in the label library with the business requirement scenario description text.

[0194] The specific implementation methods of the word segmentation subunit 1221 and the model invocation subunit 1222 can be found in the above description. Figure 3 The specific description of step S102 in the corresponding embodiment will not be repeated here.

[0195] The third processing unit 123 includes a word representation subunit 1231 and a label determination subunit 1232.

[0196] Word representation subunit 1231 is used to perform word representation processing on business requirement script keywords to obtain the word representation vector corresponding to the business requirement script keywords;

[0197] The tag determination subunit 1232 is used to obtain the tag representation vector corresponding to each tag in the tag library and determine the vector distance between the word representation vector and each tag representation vector.

[0198] The label determination subunit 1232 is also used to select the label with the smallest vector distance as the second semantic feature label.

[0199] The specific implementation methods of the word representation subunit 1231 and the label determination subunit 1232 can be found above. Figure 3 The specific description of step S102 in the corresponding embodiment will not be repeated here.

[0200] Among them, at least two keyframe images include keyframe image M. i , where i is a positive integer less than or equal to the total number of at least two keyframe images;

[0201] The second extraction module 14 includes: an image representation unit 141, a label extraction unit 142, and a label determination unit 143.

[0202] Image representation unit 141 is used for keyframe image M i Matrix representation processing is performed to obtain the keyframe image M. i The image representation matrix;

[0203] Tag extraction unit 142 is used to call the image content feature extraction model to extract keyframe images M. i Image label extraction is performed on the image representation matrix to obtain the keyframe image M. i Corresponding single-image content feature labels;

[0204] The label determination unit 143 is used to take the single image content feature labels corresponding to at least two keyframe images as image content feature labels.

[0205] The specific implementation methods of the image representation unit 141, the label extraction unit 142, and the label determination unit 143 can be found in the above description. Figure 3 The specific description of step S104 in the corresponding embodiment will not be repeated here.

[0206] The image content feature extraction model includes a second embedding layer, an encoding layer, a decoding layer, and a feedforward neural network classification layer.

[0207] The tag extraction unit 142 includes: an embedding subunit 1421, an encoding subunit 1422, a decoding subunit 1423, and a classification subunit 1424.

[0208] Embedding subunit 1421 is used to call the second embedding layer on the keyframe image M i The image representation matrix is ​​processed by convolution vector representation to obtain the image representation vector;

[0209] The encoding subunit 1422 is used to call the encoding layer to encode the image representation vector to obtain the image encoded representation vector;

[0210] Decoding subunit 1423 is used to call the decoding layer to decode the image encoding vector to obtain the image decoded representation vector;

[0211] Classification subunit 1424 is used to call the classification layer of the feedforward neural network to classify the image decoded representation vector and obtain the keyframe image M. i The corresponding image label prediction distribution is used to determine the keyframe image M. i Corresponding single-image content feature labels; image label prediction distribution includes keyframe image M i The probability value of matching each tag in the tag library.

[0212] The specific implementations of the embedding subunit 1421, encoding subunit 1422, decoding subunit 1423, and classification subunit 1424 can be found above. Figure 6 The specific descriptions of steps S301-S303 in the corresponding embodiments will not be repeated here.

[0213] The business attribute statistics label includes S business attribute statistics sub-labels; S is a positive integer;

[0214] The aforementioned video data processing device 1 further includes: a second matching module 16.

[0215] The second matching module 16 is used to traverse the S business attribute statistical sub-labels and sequentially obtain the k-th business attribute statistical sub-label; k is a positive integer less than or equal to S;

[0216] The second matching module 16 is also used to determine that the business attribute statistical label and the video resource attribute statistical label do not match if the target attribute statistical label does not exist in the video resource attribute statistical label; the target attribute statistical label refers to the label with the same attribute type as the kth business attribute statistical sub-label.

[0217] The second matching module 16 is also used to determine that the business attribute statistical label and the video resource attribute statistical label do not match if the target attribute statistical label exists in the video resource attribute statistical label and the kth business attribute statistical sub-label is different from the target attribute statistical label.

[0218] The second matching module 16 is also used to determine that the kth business attribute statistical sub-label matches the video resource attribute statistical label if the target attribute statistical label exists in the video resource attribute statistical label and the kth business attribute statistical sub-label is the same as the target attribute statistical label.

[0219] The second matching module 16 is also used to determine that the business attribute statistical label and the video resource attribute statistical label match if each business attribute statistical sub-label matches the video resource attribute statistical label.

[0220] The specific implementation of the second matching module 16 can be found in the above description. Figure 3 The optional descriptions in the corresponding embodiments will not be repeated here.

[0221] The first matching module 15 includes: a first coefficient determination unit 151, a second coefficient determination unit 152, and a matching unit 153.

[0222] The first coefficient determination unit 151 is used to determine the coefficients based on the business semantic feature labels and the keyframe image M. i The corresponding single-image content feature labels are used to determine the keyframe image M. i The corresponding image label repetition coefficient;

[0223] The second coefficient determination unit 152 is used to average the image label repetition coefficients corresponding to at least two key frame images to obtain the resource label repetition coefficients corresponding to the video resources.

[0224] Matching unit 153 is used to determine that the matching result of the video resource for the business requirement script information is a matching failure result if the resource tag repetition coefficient is less than the resource tag repetition threshold.

[0225] The matching unit 153 is also used to determine the matching result of the video resource for the business requirement script information as a successful match if the resource tag repetition coefficient is greater than or equal to the resource tag repetition threshold.

[0226] The specific implementation methods of the first coefficient determination unit 151, the second coefficient determination unit 152, and the matching unit 153 can be found above. Figure 3 The specific description of step S105 in the corresponding embodiment will not be repeated here.

[0227] The first coefficient determination unit 151 includes a label classification subunit 1511 and a coefficient calculation subunit 1512.

[0228] The label classification subunit 1511 is used to classify business semantic feature labels and keyframe images M. i The same label in the corresponding single image content feature label is used as the keyframe image M. i Corresponding intersection tags;

[0229] The label classification subunit 1511 is also used to classify business semantic feature labels and keyframe images M i The corresponding single-image content feature labels are deduplicated to obtain the keyframe image M. i The corresponding union tag;

[0230] Coefficient calculation subunit 1512 is used to calculate the keyframe image M i The total number of corresponding intersection labels and keyframe images M i Divide the total number of corresponding union labels to obtain the keyframe image M. i The corresponding image label repetition coefficient.

[0231] The specific implementation methods of the label classification subunit 1511 and the coefficient calculation subunit 1512 can be found in the above description. Figure 3 The specific descriptions of steps S101-S105 in the corresponding embodiments will not be repeated here.

[0232] The video data processing device 1 mentioned above also includes a display module 17.

[0233] Display module 17 is used to display at least two keyframe images and the single image content feature tags corresponding to the at least two keyframe images in the immersive review workbench if the matching result of the video resource for the business requirement script information is a successful match; the immersive review workbench is used by the review object to determine the review result of the video resource.

[0234] The specific implementation of the display module 17 can be found in the above description. Figure 8 The specific description of step S406 in the corresponding embodiment will not be repeated here.

[0235] The display module 17 includes a preprocessing unit 171, an image display unit 172, and a label display unit 173.

[0236] Preprocessing unit 171, used for processing keyframe image M i In the corresponding single-image content feature label, it will be related to the keyframe image M i Labels with different intersection labels are used as keyframe images M. i Corresponding mismatched tags;

[0237] Preprocessing unit 171 is also used to process keyframe images M i Determine the keyframe image M at the playback time point in the video resource. i The order of presentation in at least two keyframe images;

[0238] Image display unit 172 is used to display keyframe images M in the immersive review workbench according to the presentation order. i ;

[0239] Tag display unit 173, used for displaying tags on keyframe image M i The corresponding text display area displays the keyframe image M in the first style. i The corresponding intersection labels are used to display the keyframe image M in the second style. i The corresponding mismatched tag.

[0240] The specific implementation methods of the preprocessing unit 171, the image display unit 172, and the label display unit 173 can be found in the above description. Figure 8 The specific description of step S406 in the corresponding embodiment will not be repeated here.

[0241] The display module 17 includes a display area determination unit 174.

[0242] Display area determination unit 174 is used to determine the area size corresponding to the video resource based on the image size corresponding to at least two keyframe images and the total number of at least two keyframe images.

[0243] The display area determination unit 174 is also used to determine the available display area in the immersive review workbench, and allocate a dynamic display area corresponding to the size of the video resource in the available display area.

[0244] The display area determination unit 174 is also used to display at least two keyframe images and single image content feature labels corresponding to the at least two keyframe images in the dynamic display area.

[0245] The specific implementation of the display area determination unit 174 can be found in the above description. Figure 8 The specific description of step 406 in the corresponding embodiment will not be repeated here.

[0246] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 11 As shown, the above Figure 10 The data processing device 1 in the corresponding embodiment (e.g., including) Figure 10 The acquisition module 11, first extraction module 12, video preprocessing module 13, second extraction module 14, and first matching module 15 in the above-mentioned module can be applied to a computer device 1000. The computer device 1000 may include a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the computer device 1000 may also include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to realize communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage device. The memory 1005 may also optionally be at least one storage device located remotely from the aforementioned processor 1001. Figure 10 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application.

[0247] In such Figure 10 In the computer device 1000 shown, the network interface 1004 provides network communication elements; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0248] Obtain the business requirement script information entered by the first object, and obtain the video resources uploaded by the second object; the video resources are the resources uploaded in response to the business requirement script information.

[0249] The business requirement script information is processed by extracting text tags to obtain business script tags; business script tags include business attribute statistical tags and business semantic feature tags.

[0250] Video resources are preprocessed to obtain statistical labels for video resource attributes;

[0251] Keyframe image extraction processing is performed on video resources to obtain at least two keyframe images. Image label extraction processing is then performed on the at least two keyframe images to obtain image content feature labels.

[0252] If the business attribute statistical tags and the video resource attribute statistical tags match, then the matching result of the video resource for the business requirement script information is determined based on the business semantic feature tags and the image content feature tags.

[0253] It should be understood that the computer device 1000 described in the embodiments of this application can perform the video data processing method described in any of the preceding embodiments, and will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0254] Furthermore, it should be noted that this application also provides a computer-readable storage medium storing a computer program executed by the aforementioned video data processing apparatus 1. This computer program includes program instructions, which, when executed by the processor, enable the execution of the video data processing method described in any of the preceding embodiments. Therefore, further details will not be repeated here. Additionally, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the embodiments of the computer-readable storage medium involved in this application, please refer to the description of the method embodiments of this application.

[0255] The aforementioned computer-readable storage medium can be an internal storage unit of the video data processing apparatus or the computer device provided in any of the foregoing embodiments, such as a hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0256] Furthermore, it should be noted that this application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned... Figure 3, Figure 4 The method provided in any of the corresponding embodiments.

[0257] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0258] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the foregoing description as a network element. Whether these network elements are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can implement the described network elements using different methods for each specific application, but such implementation should not be considered beyond the scope of this application.

[0259] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A video data processing method, characterized in that, include: Obtain the business requirement script information entered by the first object, and obtain the video resources uploaded by the second object; The video resources are uploaded in response to the business requirement script information, which includes a business requirement scenario description text, business requirement script keywords, and basic business requirement script information. The business requirement scenario description text, the business requirement script keywords, and the basic information of the business requirement script are subjected to structured attribute statistical processing to obtain business attribute statistical tags. Unstructured semantic feature extraction is performed on the business requirement scenario description text to obtain the first semantic feature label; the business requirement script keywords are matched with the tags in the tag library, and the tags that match the business requirement script keywords are used as the second semantic feature label. The first semantic feature label and the second semantic feature label are used as business semantic feature labels; The video resources are preprocessed to obtain statistical tags for video resource attributes; The video resource is subjected to keyframe image extraction processing to obtain at least two keyframe images. Image label extraction processing is then performed on the at least two keyframe images to obtain image content feature labels. If the business attribute statistical label and the video resource attribute statistical label match, then the matching result of the video resource for the business requirement script information is determined based on the business semantic feature label and the image content feature label.

2. The method according to claim 1, characterized in that, The unstructured semantic feature extraction process performed on the business requirement scenario description text to obtain the first semantic feature label includes: The text describing the business requirement scenario is segmented to obtain a set of words to be processed; the set of words to be processed includes one or more words to be processed. The text semantic feature extraction model is invoked to perform unstructured semantic feature extraction on the word set to be processed, and the first semantic feature label is obtained.

3. The method according to claim 2, characterized in that, The text semantic feature extraction model includes a first embedding layer, a memory network layer, and a normalized classification layer; the text semantic feature extraction model is invoked to perform unstructured semantic feature extraction processing on the word set to be processed, resulting in a first semantic feature label, including: The first embedding layer is invoked to perform embedding feature processing on each word to be processed, thereby obtaining the word embedding vector corresponding to each word to be processed. Identify the sorting position of each word to be processed in the set of words to be processed, and call the memory network layer to perform hidden layer feature representation processing on the word embedding vector corresponding to each word to be processed according to the sorting position to obtain the hidden layer representation vector corresponding to the set of words to be processed; The normalized classification layer is invoked to normalize the hidden layer representation vector to obtain the text label prediction distribution corresponding to the word to be processed. The first semantic feature label corresponding to the business requirement scenario description text is determined based on the text label prediction distribution. The text label prediction distribution contains the probability value of the business requirement scenario description text matching each label in the label library.

4. The method according to claim 1, characterized in that, The step of matching the business requirement script keywords with tags in the tag library, and using the tags that match the business requirement script keywords as second semantic feature tags, includes: The business requirement script keywords are processed to obtain word representation vectors corresponding to the business requirement script keywords; Obtain the tag representation vector corresponding to each tag in the tag library, and determine the vector distance between the word representation vector and each tag representation vector; The label with the smallest vector distance is used as the second semantic feature label.

5. The method according to claim 1, characterized in that, The at least two keyframe images include keyframe image M. i i is a positive integer less than or equal to the total number of the at least two keyframe images; The step of extracting image labels from the at least two keyframe images to obtain image content feature labels includes: For the keyframe image M i Matrix representation processing is performed to obtain the keyframe image M. i The image representation matrix; The image content feature extraction model is called to extract features from the keyframe image M. i The image representation matrix is ​​used to extract image labels, resulting in the keyframe image M. i Corresponding single-image content feature labels; The single image content feature labels corresponding to the at least two keyframe images are used as image content feature labels.

6. The method according to claim 5, characterized in that, The image content feature extraction model includes a second embedding layer, an encoding layer, a decoding layer, and a feedforward neural network classification layer; The image content feature extraction model is invoked for the keyframe image M. i The image representation matrix is ​​used to extract image labels, resulting in the keyframe image M. i The corresponding single-image content feature labels include: The second embedding layer is invoked on the keyframe image M i The image representation matrix is ​​processed by convolution vector representation to obtain the image representation vector; The encoding layer is invoked to encode the image representation vector to obtain the image encoded representation vector; The decoding layer is invoked to decode the image encoding vector to obtain the image decoded representation vector; The feedforward neural network classification layer is invoked to classify the image decoded representation vector to obtain the keyframe image M. i The corresponding image label prediction distribution is used to determine the keyframe image M. i The corresponding single-image content feature labels; the image label prediction distribution includes the keyframe image M i The probability value of matching each tag in the tag library.

7. The method according to claim 1, characterized in that, The business attribute statistics label includes S business attribute statistics sub-labels; S is a positive integer; The method further includes: Iterate through the S business attribute statistical sub-labels and sequentially obtain the k-th business attribute statistical sub-label; k is a positive integer less than or equal to S; If the target attribute statistical label is not present in the video resource attribute statistical label, then it is determined that the business attribute statistical label and the video resource attribute statistical label do not match; the target attribute statistical label refers to a label with the same attribute type as the k-th business attribute statistical sub-label. If the target attribute statistical label exists in the video resource attribute statistical label, and the kth business attribute statistical sub-label is different from the target attribute statistical label, then it is determined that the business attribute statistical label and the video resource attribute statistical label do not match. If the target attribute statistical label exists in the video resource attribute statistical label, and the kth business attribute statistical sub-label is the same as the target attribute statistical label, then the kth business attribute statistical sub-label is determined to match the video resource attribute statistical label. If each business attribute statistical sub-tag matches the video resource attribute statistical tag, then the business attribute statistical tag and the video resource attribute statistical tag are determined to be a match.

8. The method according to claim 5, characterized in that, The step of determining the matching result of the video resource for the business requirement script information based on the business semantic feature tags and the image content feature tags includes: Based on the business semantic feature tags and the keyframe image M i The corresponding single-image content feature labels are used to determine the keyframe image M. i The corresponding image label repetition coefficient; The image label repetition coefficients corresponding to the at least two keyframe images are averaged to obtain the resource label repetition coefficients corresponding to the video resource. If the resource tag repetition coefficient is less than the resource tag repetition threshold, then the matching result of the video resource for the business requirement script information is determined to be a matching failure result. If the resource tag repetition coefficient is greater than or equal to the resource tag repetition threshold, then the matching result of the video resource for the business requirement script information is determined to be a successful match.

9. The method according to claim 8, characterized in that, The method based on the business semantic feature tags and the keyframe image M i The keyframe image M is determined by the corresponding single-image content feature label. i The corresponding image label repetition coefficients include: The business semantic feature labels and the keyframe image M i The same label in the corresponding single image content feature label is used as the keyframe image M. i Corresponding intersection tags; The business semantic feature labels and the keyframe image M i The corresponding single-image content feature labels are deduplicated to obtain the keyframe image M. i The corresponding union tag; The keyframe image M i The total number of corresponding intersection tags and the keyframe image M i Divide the total number of corresponding union tags to obtain the keyframe image M. i The corresponding image label repetition coefficient.

10. The method according to claim 5, characterized in that, Also includes: If the matching result of the video resource for the business requirement script information is a successful match, then the at least two keyframe images and the single image content feature tags corresponding to the at least two keyframe images are displayed in the immersive review workbench. The immersive review workbench is used by the reviewer to determine the review result of the video resource.

11. The method according to claim 10, characterized in that, The step of displaying the at least two keyframe images and the single-image content feature tags corresponding to the at least two keyframe images in the immersive review workbench includes: In the keyframe image M i In the corresponding single-image content feature label, the keyframe image M will be... i The labels whose corresponding intersection labels are different are used as the keyframe image M. i Corresponding mismatched tags; According to the keyframe image M i The keyframe image M is determined at the playback time node in the video resource. i The order of display in the at least two keyframe images; The keyframe image M is displayed in the immersive review workbench according to the aforementioned display order. i ; In the keyframe image M i The corresponding text display area displays the keyframe image M in the first style. i The corresponding intersection labels are used to display the keyframe image M in a second style. i The corresponding mismatched tag.

12. The method according to claim 10, characterized in that, The step of displaying the at least two keyframe images and the single-image content feature tags corresponding to the at least two keyframe images in the immersive review workbench includes: The size of the region corresponding to the video resource is determined based on the image size corresponding to each of the at least two keyframe images and the total number of the at least two keyframe images. Determine the available display area in the immersive review workbench, and allocate a dynamic display area corresponding to the size of the area to the video resource within the available display area; The at least two keyframe images and their corresponding single-image content feature labels are displayed in the dynamic display area.

13. A video data processing apparatus, characterized in that, include: The acquisition module is used to acquire the business requirement script information entered by the first object and the video resources uploaded by the second object. The video resources are uploaded in response to the business requirement script information, which includes a business requirement scenario description text, business requirement script keywords, and basic business requirement script information. The first extraction module is used to perform structured attribute statistical processing on the business requirement scenario description text, the business requirement script keywords, and the basic information of the business requirement script to obtain business attribute statistical tags; to perform unstructured semantic feature extraction processing on the business requirement scenario description text to obtain first semantic feature tags; and to match the business requirement script keywords with tags in the tag library, and use the tags that match the business requirement script keywords as second semantic feature tags. The first semantic feature label and the second semantic feature label are used as business semantic feature labels; The video preprocessing module is used to preprocess the video resources to obtain video resource attribute statistical tags; The second extraction module is used to perform keyframe image extraction processing on the video resource to obtain at least two keyframe images, and to perform image label extraction processing on the at least two keyframe images to obtain image content feature labels. The first matching module is used to determine the matching result of the video resource for the business requirement script information based on the business semantic feature label and the image content feature label if the business attribute statistical label and the video resource attribute statistical label match.

14. A computer device, characterized in that, include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide data communication functions, the memory is used to store program code, and the processor is used to call the program code to execute the method according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and to execute the method of any one of claims 1-12.

16. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they can perform the method described in any one of claims 1-12.

Images