Video processing method, device, equipment and medium

Abstract

The embodiment of the present application provides a video processing method, device, equipment and medium, which relates to the field of artificial intelligence, and the method includes: obtaining a target video and a target network model; inputting the target text information of the target video into a first network model, and the first network model outputs a text feature vector corresponding to the target text information; evenly dividing the target video into K video segments, and performing random frame sampling processing on the K video segments based on a time sampling strategy to obtain video segments of each video segment; obtaining K first subnetworks in a second network model, and inputting each video segment into different first subnetworks respectively, and the K first subnetworks output video feature vectors of each video segment respectively; based on the text feature vector, the video feature vector of each video segment and the classifier of the target network model, obtaining the target style label of the target video. The present application can improve the accuracy of short video recommendation.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a video processing method, device, equipment and medium. Background Art

[0002] With the development of multimedia technology, video has become the main carrier for people to obtain information and enjoy entertainment in their daily lives. Due to the popularity of various video media platforms, a variety of short videos have emerged. Here, short videos refer to video content played on various video media platforms and suitable for watching on the move and in a short leisure state.

[0003] At present, short video recommendations are mainly based on user portraits. However, the user portrait here is often determined based on the user registration information of the user and the basic video information of the video requested by the user. For example, the basic video information here mainly refers to the label information and subject information of some short videos manually marked, and the user registration information here can include the user's age information and gender information. It can be seen that the existing video recommendation mainly relies on some coarse-grained information with a low degree of abstraction (i.e., a high degree of generalization). Based on this, when using this coarse-grained information to recommend videos to users, the accuracy of short video recommendations will inevitably be reduced. Summary of the invention

[0004] The embodiments of the present application provide a video processing method, apparatus, device and medium, which can improve the accuracy of short video recommendations.

[0005] An embodiment of the present application provides a video processing method, including:

[0006] Obtain a target video and a target network model corresponding to the target video; the target network model includes a first network model and a second network model;

[0007] Obtain target text information of a target video, input the target text information into a first network model, and have the first network model output a text feature vector corresponding to the target text information;

[0008] The target video is evenly divided into K video segments, and the K video segments are randomly sampled based on the time sampling strategy to obtain video segments of each video segment; K is a positive integer, and the number of video frames in each video segment is N; N is a positive integer; one video segment corresponds to one video segment, and the number of video frames in one video segment is less than or equal to N;

[0009] Obtain K first sub-networks in the second network model, input each video slice into a different first sub-network, and have the K first sub-networks output a video feature vector for each video slice;

[0010] Based on the text feature vector, the video feature vector of each video segment and the classifier of the target network model, the style of the target video is identified to obtain a target style label of the target video; the target style label is used to push associated videos related to the target video to the first user.

[0011] An embodiment of the present application provides a video processing device, including:

[0012] A target acquisition module is used to acquire a target video and a target network model corresponding to the target video; the target network model includes a first network model and a second network model;

[0013] A first output module, used to obtain target text information of a target video, input the target text information into a first network model, and output a text feature vector corresponding to the target text information by the first network model;

[0014] The first determination module is used to evenly divide the target video into K video segments, and randomly extract frames from the K video segments based on a time sampling strategy to obtain video segments of each video segment; K is a positive integer, and the number of video frames in each video segment is N; N is a positive integer; one video segment corresponds to one video segment, and the number of video frames in one video segment is less than or equal to N;

[0015] A second output module is used to obtain K first sub-networks in the second network model, input each video slice into a different first sub-network, and output a video feature vector of each video slice from the K first sub-networks;

[0016] The style recognition module is used to identify the style of the target video based on the text feature vector, the video feature vector of each video segment and the classifier of the target network model to obtain a target style label of the target video; the target style label is used to push associated videos related to the target video to the first user.

[0017] Among them, the target acquisition module includes:

[0018] A video acquisition unit, configured to call a video style recognition service through a dispatch center service component, and acquire a target video uploaded by a second user through a video upload interface through the video style recognition service;

[0019] The target acquisition unit is used to acquire a target network model for performing style recognition on a target video through a video style recognition service.

[0020] Wherein, the first output module includes:

[0021] A text acquisition unit, used to acquire video subject information, video classification information and video tag information of a target video, and use the video subject information, video classification information and video tag information as target text information of the target video;

[0022] A text segmentation unit is used to perform segmentation processing on the target text information to obtain text segmentation of the target text information, and to convert the text segmentation into a vector to obtain a text segmentation vector corresponding to the text segmentation;

[0023] A feature extraction unit, used for inputting the text word segmentation vector into the first network model, and extracting the vector feature of the text word segmentation vector by the first network model;

[0024] The text determination unit is used to determine the text position of the text segmentation in the target text information, and determine the text feature vector corresponding to the target text information based on the text position and the extracted vector features.

[0025] Among them, the first subnetwork includes a neural network and a multilayer perceptron; wherein the multilayer perceptron includes an input layer, a hidden layer and an output layer; the neural network is used to extract the initial video features of the input video segment; the input layer is used to use the initial video features as the input features of the hidden layer, and the hidden layer is used to extract the hidden features of the input features; the output layer is used to output the video feature vector of the input video segment according to the hidden features.

[0026] Among them, the painting style recognition module includes:

[0027] A vector projection unit, used for projecting the text feature vector and the video feature vector of each video slice into a target vector space;

[0028] A vector fusion unit, used for fusing the text feature vector and the video feature vector of each video slice in the target vector space to obtain a fused vector feature;

[0029] A matching degree determination unit is used to input the fused vector feature into the classifier of the target network model, and the classifier of the target network model performs style recognition on the target video to obtain the matching degree between the fused vector feature and the sample attribute feature in the classifier; the matching degree is used to describe the probability that the target video belongs to the sample label corresponding to the sample attribute feature;

[0030] The label determination unit is used to use the sample label corresponding to the sample attribute feature with the maximum matching probability from the matching degree as the target style label of the target video.

[0031] The device further comprises:

[0032] An initial acquisition module, used to acquire sample videos and sample style labels of the sample videos for training an initial network model; the initial network model includes a first sample model and a second sample model;

[0033] A third output module, used to obtain sample text information of the sample video, input the sample text information into the first sample model, and output a sample text vector corresponding to the sample text information by the first sample model;

[0034] The second determination module is used to evenly divide the sample video into K sample segments, and randomly extract frames from the K sample segments based on a time sampling strategy to obtain sample slices of each sample segment; K is a positive integer, and the number of video frames in each sample segment is M; M is a positive integer; one sample segment corresponds to one sample slice, and the number of video frames in one sample slice is less than or equal to M;

[0035] A fourth output module is used to obtain K second sub-networks in the second sample model, input each sample slice into a different second sub-network, and output a sample video vector of each sample slice by the K second sub-networks;

[0036] The initial training module is used to iteratively train the initial network model based on the sample style label, the sample text vector, the sample video vector of each sample segment and the classifier of the initial network model, and use the initial network model after iterative training as the target network model.

[0037] The initial training modules include:

[0038] An iterative training unit, used for iteratively training the initial network model based on the sample painting style label, the sample text vector, the sample video vector of each sample slice and the classifier of the initial network model to obtain a model training result;

[0039] A transition determination unit, configured to use the initial network model after iterative training as a transition network model when the model training result indicates that the model loss function of the initial network model after iterative training does not meet the model convergence condition;

[0040] The target determination unit is used to adjust the model parameters of the transition network model based on the model loss function that does not meet the model convergence condition, until the adjusted transition network model meets the model convergence condition, and the transition network model that meets the model convergence condition is used as the target network model.

[0041] The device further comprises:

[0042] A duplicate removal processing module, used for obtaining a target video uploaded by a second user through a dispatch center service component, performing duplicate removal processing on the target video, obtaining a duplicate removal processing result, and storing the target video in a content database based on the duplicate removal processing result;

[0043] The video review module is used to review the target video according to the target video and the meta information of the target video read from the content database to obtain the video review result;

[0044] A distribution activation module is used to start the content distribution service when the video review result indicates that the target video is a distributable video, call the content distribution service to obtain the aggregation cluster mapped by the target style label from the video recommendation database, and add the distributable video to the aggregation cluster;

[0045] The video push module is used to obtain a user portrait of the first user, obtain at least one related video to be pushed to the first user from the aggregation cluster based on the target style tag carried in the user portrait, and push the at least one related video to the first user.

[0046] The duplicate removal processing module includes:

[0047] A feature configuration unit, used to obtain a target video uploaded by a second user through a dispatch center service component, and configure a video fingerprint feature for the target video;

[0048] A distance determination unit, used to obtain a historical fingerprint feature of each historical video in the content database, and determine a vector distance between each historical fingerprint feature and a video fingerprint feature;

[0049] The video storage unit is used to determine that the video fingerprint feature meets the duplicate elimination processing condition if there is no vector distance greater than the vector threshold associated with the historical video in the vector distance, and store the target video uploaded by the second user in the content database.

[0050] The duplicate removal processing module also includes:

[0051] The video deletion unit is used to determine that the video fingerprint feature does not meet the deduplication processing conditions if there is a vector distance less than the vector threshold in the vector distance, and determine that there is a historical video matching the target video in the content database, and delete the target video uploaded by the second user.

[0052] The video review module includes:

[0053] The video reading unit is used to read the target video from the content database, and read the first auxiliary information, the second auxiliary information and the third auxiliary information of the target video from the content database; the first auxiliary information is the information entered by the second user when uploading the target video; the second auxiliary information is obtained after the target video is classified by the video classification model; the third auxiliary information is obtained after transcoding the target video;

[0054] A first determining unit is used to use the first auxiliary information, the second auxiliary information, and the third auxiliary information as meta information of a target video, and perform a first video review on the target video based on the meta information to obtain an initial video review result;

[0055] The second determination unit is used to perform a second video review on the target video if the initial video review result indicates that the target video is legal, obtain the target video review result, and use the target video review result as the video review result of the target video.

[0056] The target video is a video played by the first user in the video client;

[0057] The video push module includes:

[0058] An information acquisition unit, configured to acquire a behavior log table of the first user, and extract historical video information associated with the first user from the behavior log table; the historical video information is determined based on the historical user behavior of the first user in the video client;

[0059] The portrait determination unit is used to obtain a historical painting style tag of the historical video information, and determine a user portrait for characterizing the first user based on the historical painting style tag and the target painting style tag.

[0060] An embodiment of the present application provides a computer device, including: a processor and a memory;

[0061] The processor is connected to the memory, wherein the memory is used to store a computer program. When the computer program is executed by the processor, the computer device executes the method provided in the embodiment of the present application.

[0062] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the method provided by the embodiment of the present application.

[0063] In one aspect, the present application provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in the present application.

[0064] In an embodiment of the present application, when a computer device obtains a certain video (for example, a target video uploaded by a second user), it can obtain a target network model corresponding to the video, where the target network model can include a first network model and a second network model. Further, when the computer device obtains the target text information of the target video, it can output a text feature vector corresponding to the target text information through the first network model. Further, when the computer device divides the target video into K video segments evenly, it can randomly extract frames from the K video segments based on the time sampling strategy to obtain video segments of each video segment, and then obtain K first subnetworks in the second network model to output the video feature vector of each video segment through different first subnetworks. Wherein, K here can be a positive integer, and the number of video frames in each video segment is N, and a video segment can correspond to a video segment, therefore, the number of video frames in a video segment is less than or equal to N, where N can be a positive integer. Further, the computer device can identify the style of the target video based on the text feature vector, the video feature vector of each video segment, and the classifier of the target network model to obtain the target style label of the target video. It can be seen that the embodiment of the present application can accurately identify the target style label of the target video through the target network model with the style recognition function. For example, the embodiment of the present application can extract the text feature vector of the aforementioned target text information through the first network model. It should be understood that the text feature vector here is the auxiliary video vector of the target video. In addition, the embodiment of the present application can also extract the video feature vector of each video segment through the K first sub-networks in the second network model. It should be understood that by inputting different video segments into different first sub-networks respectively, a richer video feature vector can be extracted in the time dimension. In this way, when the text feature vector (i.e., the aforementioned auxiliary video vector) and the richer video feature vector are vector fused by the classifier in the target network model, the video features obtained by fusion for accurately characterizing the target video can be identified by style, so as to accurately obtain the target style label of the target video. It should be understood that the target style label here can be used to characterize the video style and video tonality of the target video, which belongs to a higher-dimensional video classification attribute at a deeper level. Based on this, after video clustering is performed based on the identified target style label, associated videos related to the target video can be quickly and accurately obtained for pushing to the first user. This means that after obtaining the target style label, the accuracy of short video recommendations can be improved when making short video recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0066] Figure 1 It is a structural diagram of a network architecture provided by an embodiment of the present application;

[0067] Figure 2 This is a schematic diagram of a scenario for data interaction provided by an embodiment of the present application;

[0068] Figure 3 It is a flowchart of a video processing method provided by an embodiment of the present application;

[0069] Figure 4 It is a schematic diagram of a scenario for extracting video segments provided in an embodiment of the present application;

[0070] Figure 5 is a structural schematic diagram of a painting style recognition model provided in an embodiment of the present application;

[0071] Figure 6 It is a structural schematic diagram of a multi-layer perceptron provided in an embodiment of the present application;

[0072] Figure 7 It is a flowchart of a video processing method provided by an embodiment of the present application;

[0073] Figure 8 This is a system flow chart of painting style label recognition provided by an embodiment of the present application;

[0074] Fig. 9 This is a schematic diagram of a scene for performing painting style recognition provided by an embodiment of the present application;

[0075] Fig.10a This is a schematic diagram of a scenario for video distribution provided by an embodiment of the present application;

[0076] Fig.10b This is a schematic diagram of a scenario for performing video recommendation provided by an embodiment of the present application;

[0077] Fig.10c is a schematic diagram of a scenario for performing video search provided in an embodiment of the present application;

[0078] Fig.11 is a structural schematic diagram of a video processing device provided in an embodiment of the present application;

[0079] Fig.12It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0081] It should be understood that artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0082] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0083] Among them, the solution provided in the embodiment of the present application mainly involves natural language processing (NLP) technology and machine learning (ML) technology of artificial intelligence.

[0084] Among them, Natural Language Processing is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0085] Among them, machine learning is a multi-disciplinary interdisciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0086] Data Compliance Statement: The collection and processing of data related to "user behavior, user portrait" in this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, and have a legal basis or obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and / or personal information subjects.

[0087] For details, see Figure 1 , Figure 1 Schematic diagram of a network architecture provided by an embodiment of the present application. Figure 1 As shown, the network architecture may include a service server 2000 and a user terminal cluster. The user terminal cluster may specifically include one or more user terminals, and the number of user terminals in the user terminal cluster is not limited. Figure 1 As shown, the multiple user terminals may specifically include user terminal 3000a, user terminal 3000b, user terminal 3000c, ..., user terminal 3000n. The user terminal 3000a, user terminal 3000b, user terminal 3000c, ..., user terminal 3000n may be respectively connected to the service server 2000 directly or indirectly through a wired or wireless communication mode, so that each user terminal may exchange data with the service server 2000 through the network connection.

[0088] Among them, Figure 1 The business server 2000 shown can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0089] It should be understood that Figure 1Each user terminal in the user terminal cluster shown in the figure may be integrated with an application client. When the application client runs in each user terminal, it may be respectively connected to the above-mentioned Figure 1 The application client can be understood as an application that can load and display videos. For example, the application client here can specifically include: a car client, a smart home client, an entertainment client (for example, a game client), a multimedia client (for example, a video client), a social client, and an information client (for example, a news client). For ease of understanding, the embodiments of the present application can be Figure 1 A user terminal is selected from the multiple user terminals shown as the target user terminal, and the target user terminal may include: a smart phone, a tablet computer, a laptop computer, a smart TV, a vehicle-mounted device, and other smart terminals with a video loading function. Figure 1 The user terminal 3000a is shown as the target user terminal.

[0090] For ease of understanding, in the embodiments of the present application, the videos (eg, video highlights) uploaded by a certain user (eg, user Y) through an application client may be collectively referred to as target videos.

[0091] It should be understood that the application client in the embodiment of the present application can be integrated into a client (e.g., a social client). For example, when the application client is integrated into the social client, the application client can be QQ Kandian. Optionally, the application client in the embodiment of the present application can also be a client independent of the above-mentioned social client (e.g., a news client). For example, when the application client is an independent client, the application client can be Kandian Express. The embodiment of the present application does not limit the type of the application client.

[0092] It is understandable that in the embodiment of the present application, the user who logs in to the application client through the first account information (for example, account information 1) may be referred to as the first user, and the user terminal corresponding to the first user may be referred to as the first terminal. In the embodiment of the present application, any user terminal in the above-mentioned user terminal cluster may be selected as the first terminal. For example, in the embodiment of the present application, the user terminal 3000a in the above-mentioned user terminal cluster may be used as the first terminal. It should be understood that the first user in the embodiment of the present application may be a user who receives a video through the application client, that is, a video receiver.

[0093] It is understandable that, in the embodiment of the present application, the user who logs in to the application client through the second account information (e.g., account information 2) may be referred to as the second user (e.g., the above-mentioned user Y), and the user terminal corresponding to the second user may be referred to as the second terminal. In the embodiment of the present application, any user terminal may be selected from the above-mentioned user terminal cluster as the second terminal. For example, in the embodiment of the present application, the user terminal 3000b in the above-mentioned user terminal cluster may be used as the second terminal. It should be understood that the second user in the embodiment of the present application may be a user who uploads a video through the application client, i.e., a video uploader.

[0094] It should be understood that the first user in the embodiment of the present application can be used as the above-mentioned video receiver or the above-mentioned video uploader. For example, the first user can become a video receiver through the application client in the first terminal, and the first user can also become a video uploader through the application client in the first terminal. Similarly, the second user in the embodiment of the present application can be used as the above-mentioned video uploader or the above-mentioned video receiver. For example, the second user can become a video uploader through the application client in the second terminal, and the second user can also become a video receiver through the application client in the second terminal.

[0095] It can be understood that the business scenarios applicable to the above network framework may specifically include: video distribution scenarios, video recommendation scenarios, video search scenarios, etc., and the specific business scenarios will not be listed one by one here.

[0096] For example, in a video distribution scenario, a computer device (e.g., the business server 2000 described above) can recommend at least one associated video to user Y based on the user portrait of user Y, wherein at least one associated video here can have the same style tag as the video that user Y likes (or has watched). For example, by analyzing the user portrait of user Y, it can be determined that the style tag that user Y likes (or has watched) is cute pets, then the computing device can obtain one or more associated videos (e.g., video S1 and video S2) with the style tag of cute pets, and then the computer device can push video S1 and video S2 to the application client corresponding to user Y. It should be understood that the number of style tags that user Y likes can also be multiple (e.g., 2), and the embodiment of the present application does not limit the number of style tags here.

[0097] For another example, in a video recommendation scenario, a computer device (e.g., the business server 2000) may obtain the style tag of the video being watched by user Y, and recommend at least one associated video to user Y, wherein the at least one associated video may have the same style tag as the video being watched by user Y. For example, if the style tag of the video being watched by user Y (e.g., video S3) is campus life, the computer device may obtain one or more associated videos (e.g., video S4) whose style tag is campus life, and then the computer device may push video S4 to the application client corresponding to video S3 being watched by user Y.

[0098] For another example, in a video search scenario, the computer device (e.g., the business server 2000) can identify the style tag corresponding to the content searched by user Y when user Y searches, and recommend at least one related video to user Y, wherein the at least one related video here can have the same style tag as the content searched by user Y. For example, if the content searched by user Y is rice transplanting, and the style tag corresponding to rice transplanting is rural life, the computer device can obtain one or more related videos (e.g., video S5 and video S6) with the style tag of rural life, and then the computer device can push video S5 and video S6 to the search result interface of the application client corresponding to user Y.

[0099] For further understanding, please refer to Figure 2 , Figure 2 is a schematic diagram of a scenario for data interaction provided by an embodiment of the present application. Figure 2 The business server shown can be the above Figure 1 The service server 2000 in the corresponding embodiment is as follows: Figure 2 The terminal Z shown (i.e. the first terminal) can be the Figure 1 For ease of understanding, any user terminal in the user terminal cluster of the corresponding embodiment is described in the embodiment of the present application. Figure 1 The user terminal 3000a shown is used as the terminal Z as an example to illustrate Figure 2 The specific process of data interaction between terminal Z and the service server is shown.

[0100] like Figure 2 As shown, when the target video is acquired, the service server may evenly divide the target video into K video segments, where the K video segments may specifically include: video segment 20a, video segment 20b, ..., video segment 20K, where K may be a positive integer. Each of the K video segments has the same number of video frames, for example, each video segment may include N video frames, where N may be a positive integer.

[0101] Further, such as Figure 2 As shown, the service server can perform random frame sampling processing on the K video segments based on the time sampling strategy to obtain video segments of each video segment, where the number of video segments is K, and the K video segments here can specifically include: video segment 200a, video segment 200b, ..., video segment 200K. Among them, video segment 200a can be obtained by performing random frame sampling processing on video segment 20a, video segment 200b can be obtained by performing random frame sampling processing on video segment 20b, ..., video segment 200K can be obtained by performing random frame sampling processing on video segment 20K. It can be understood that the number of video frames in video segment 200a is less than or equal to the number of video frames in video segment 20a, the number of video frames in video segment 200b is less than or equal to the number of video frames in video segment 20b, ..., the number of video frames in video segment 200K is less than or equal to the number of video frames in video segment 20K.

[0102] Further, such as Figure 2 As shown, the business server can obtain K first subnetworks in the second network model, and input each of the K video slices into different first subnetworks to output the video feature vector of each video slice. The K first subnetworks here can specifically include: first subnetwork 1, first subnetwork 2, ..., first subnetwork K. For example, the business server can input video slice 200a into first subnetwork 1, and first subnetwork 1 outputs feature vector 1 of video slice 200a; input video slice 200b into first subnetwork 2, and first subnetwork 2 outputs feature vector 2 of video slice 200b; ...; input video slice 200K into first subnetwork K, and first subnetwork K outputs feature vector K of video slice 200K. It can be understood that feature vector 1, feature vector 2, ..., feature vector K here can be called video feature vectors.

[0103] like Figure 2 As shown, the service server can also obtain the target text information of the target video, input the target text information into the first network model, and the first network model outputs the text feature vector of the target text information. Figure 2 The service server can perform style recognition on the target video based on the text feature vector of the target text information, the video feature vector of each video segment (i.e., feature vector 1, feature vector 2, ..., feature vector K), and the classifier of the target network model to obtain the target style label of the target video. It can be understood that the above-mentioned first network model and the second network model can be collectively referred to as the target network model.

[0104] It can be understood that by analyzing the historical user behavior of user Y (i.e., the first user mentioned above), a user portrait of user Y can be obtained. When user Y has played or is playing a target video in an application client (e.g., a video client) (i.e., when the user portrait carries the target style tag of the target video), Figure 2 The service server shown can obtain at least one associated video related to the target style tag as at least one associated video related to the target video, and then push the at least one associated video here to the terminal Z corresponding to the user Y. In this way, the user Y can select one or more videos from the at least one associated video displayed on the terminal Z to watch the one or more videos in the video client of the terminal Z. For example, the user Y can select to watch the video G in the video client.

[0105] The specific process of the business server identifying the style of the target video can be seen in the following Figure 3-10c The corresponding embodiment.

[0106] For further information, see Figure 3 , Figure 3 is a flow chart of a video processing method provided by an embodiment of the present application. The method can be performed by a user terminal (for example, the above Figure 1 The user terminal 3000a shown in the figure may also be executed by a service server (for example, the above Figure 1 The video processing method may be executed by the service server 2000 shown in the figure, or may be executed by the user terminal and the service server together. For ease of understanding, this embodiment takes the method executed by the service server as an example for explanation. The video processing method may at least include the following steps S101 to S105:

[0107] Step S101, obtaining a target video and a target network model corresponding to the target video;

[0108] Specifically, the business server can call the video style recognition service through the dispatch center service component, and obtain the target video uploaded by the second user through the video upload interface through the video style recognition service. Further, the business server can obtain the target network model used for style recognition of the target video through the video style recognition service. Among them, the target network model can include a first network model and a second network model.

[0109] It should be understood that the second user can be a self-media or video production organization (for example, a multi-channel network MCN, Multi-Channel Network), and the target video uploaded by the second user through the video upload interface can be the PGC (Professional Generated Content, professionally produced content, can also be Professionally-produced Content, referred to as PPC) content and UGC (User Generated Content, user-generated content) content of the self-media or video production organization. Among them, MCN is a product form of a multi-channel network, which can unite PGC content and, with the strong support of capital, ensure the continuous output of content, thereby ultimately achieving stable commercial realization. Among them, PGC content can be used to refer to content personalization, diversified perspectives, and virtualization of social relationships.

[0110] Among them, it can be understood that the target video can be a video that needs to be style recognized in any scenario. For example, the target video can be a video recorded by the second user through the second terminal in an audio and video call scenario; the target video can also be a video recorded by the second user through the camera of the second terminal; the target video can also be a video downloaded by the second user on the Internet; the target video can also be a video edited by the second user through a video editing tool; the target video can also be a video automatically generated by the second terminal or the service server through a video generation method. The embodiment of the present application does not limit the generation method of the target video.

[0111] Step S102, obtaining target text information of a target video, inputting the target text information into a first network model, and having the first network model output a text feature vector corresponding to the target text information;

[0112] Specifically, the business server can obtain the video theme information, video classification information and video tag information of the target video, and use the video theme information, video classification information and video tag information as the target text information of the target video. Further, the business server can perform word segmentation processing on the target text information to obtain text segmentation of the target text information, perform vector conversion on the text segmentation, and obtain a text segmentation vector corresponding to the text segmentation. Further, the business server can input the text segmentation vector into the first network model, and extract the vector features of the text segmentation vector from the first network model. Further, the business server can determine the text position of the text segmentation in the target text information, and determine the text feature vector corresponding to the target text information based on the text position and the extracted vector features.

[0113] It should be understood that word segmentation is the process of recombining continuous word sequences into word sequences according to certain specifications. For example, if the video theme information is: "Me and my two friends", after word segmentation of the video theme information, the text segmentation of the video theme information can be obtained as: "Me", "and", "my", "two", "friends".

[0114] Optionally, it is understandable that when the business server obtains the text segmentation vector corresponding to the text segmentation, it can perform vector conversion on the text position to obtain the text position vector corresponding to the text position, and then perform feature fusion of the text segmentation vector and the text position vector (for example, perform vector addition of the text segmentation vector and the text position vector) to obtain the target text vector corresponding to the target text information, so that the target text vector can be input into the first network model, and the text feature vector of the target text vector can be extracted by the first network model.

[0115] Among them, it can be understood that the business server can convert the text segmentation into a vector through a word embedding algorithm (for example, Word2Vec, Glove and other algorithms) to obtain a text segmentation vector corresponding to the text segmentation. Optionally, the business server can convert the text position into a vector through a position formula to obtain a text position vector corresponding to the text position. It should be understood that the embodiment of the present application does not limit the specific implementation method of converting the text segmentation and the text position into a vector.

[0116] It should be understood that the first network model can be a Transformer model, where the Transformer model is a classic NLP (i.e. natural language processing) model. The Transformer model uses a self-attention mechanism (i.e., Self-Attention mechanism) and does not adopt the sequential structure of RNN (Recurrent Neural Network, referred to as recurrent neural network), so that the model can be trained in parallel and has global information. Among them, RNN (Recurrent Neural Network) is a type of recursive neural network (recursive neural network) that takes sequence data as input, performs recursion in the direction of sequence evolution, and all nodes (recurrent units) are connected in a chain. It should be understood that the embodiment of the present application does not limit the specific type of the first network model.

[0117] Among them, it can be understood that the self-attention mechanism is an improvement of the attention mechanism (i.e., the Attention mechanism), which imitates the internal process of biological observation behavior, that is, a mechanism that aligns internal experience and external sensation to increase the observation refinement of some areas, while the self-attention mechanism is an improvement of the attention mechanism, which reduces dependence on external information and is better at capturing the internal correlation of data or features.

[0118] Step S103, evenly dividing the target video into K video segments, and performing random frame sampling processing on the K video segments based on a time sampling strategy to obtain video segments of each video segment;

[0119] Here, K may be a positive integer, and the number of video frames in each video segment is N; here, N may be a positive integer. Here, a video segment may correspond to a video segment, and the number of video frames in a video segment is less than or equal to N.

[0120] For easier understanding, see Figure 4 , Figure 4 FIG. 1 is a schematic diagram of a scenario for extracting video segments provided in an embodiment of the present application. Figure 4 The target video shown may be the target video acquired by the service server, and the target video may include n video frames, where the n video frames may specifically include: video frame 1, video frame 2, video frame 3, ..., video frame (n-2), video frame (n-1) and video frame n.

[0121] Furthermore, through Figure 4 The target video shown is evenly divided to obtain K video segments, where K can be a positive integer greater than 1. The K video segments here can specifically include: segment 4a, segment 4b, ..., segment 4k. The number of video frames in each video segment is 3 (i.e., N is equal to 3), that is, each video segment can include 3 video frames. Segment 4a can include video frame 1, video frame 2, and video frame 3, segment 4b can include video frame 4, video frame 5, and video frame 6, ..., and segment 4k can include video frame (n-2), video frame (n-1), and video frame n.

[0122] Furthermore, through Figure 4Each of the K video segments shown is randomly sampled to obtain K video segments corresponding to the K video segments, where the K video segments may specifically include: segment 40a, segment 40b, ..., segment 40k. Among them, one video segment corresponds to one video segment, and the number of video frames in one video segment is less than or equal to N (i.e., 3). For example, 3 video frames (e.g., video frame 1, video frame 2, and video frame 3) may be randomly sampled from segment 4a as the video segment corresponding to segment 4a, i.e., segment 40a; 1 video frame (e.g., video frame 6) may be randomly sampled from segment 4b as the video segment corresponding to segment 4b, i.e., segment 40b; ...; 2 video frames (e.g., video frame (n-1) and video frame n) may be randomly sampled from segment 4k as the video segment corresponding to segment 4k, i.e., segment 40k.

[0123] It should be understood that when a video segment includes N video frames, the number of video frames randomly extracted from the video segment can be 1, ..., N. It is understandable that when a video segment includes 3 video frames, the number of video frames randomly extracted from the video segment can be 1, 2 or 3. For ease of understanding, the video segment is taken as segment 4a as an example. It is understandable that when the number of video frames obtained by the random frame extraction process is 1, the randomly extracted video frames can include video frame 1, that is, video frame 1 can be used as the video slice corresponding to segment 4a; the randomly extracted video frames can include video frame 2, that is, video frame 2 can be used as the video slice corresponding to segment 4a; the randomly extracted video frames can include video frame 3, that is, video frame 3 can be used as the video slice corresponding to segment 4a.

[0124] Similarly, it can be understood that when the number of video frames obtained by the random frame sampling process is 2, the randomly extracted video frames may include video frame 1 and video frame 2, that is, video frame 1 and video frame 2 can be used as the video fragments corresponding to segment 4a; the randomly extracted video frames may include video frame 2 and video frame 3, that is, video frame 2 and video frame 3 can be used as the video fragments corresponding to segment 4a.

[0125] Similarly, it can be understood that when the number of video frames obtained by the random frame sampling process is 3, the randomly sampled video frames can include video frame 1, video frame 2 and video frame 3, that is, video frame 1, video frame 2 and video frame 3 can be used as the video fragments corresponding to segment 4a.

[0126] It should be understood that the specific process of performing random frame sampling processing on segments 4b, ..., and segment 4k can be referred to the above description of performing random frame sampling processing on segment 4a, and will not be repeated here.

[0127] It should be understood that the video frames obtained by randomly extracting frames from a video segment (e.g., segment 4a) may be continuous video frames. Optionally, the video frames obtained by randomly extracting frames from a video segment may also be discontinuous video frames. For example, the video frames randomly extracted from segment 4a may include video frame 1 and video frame 3, that is, video frame 1 and video frame 3 may be used as the video fragments corresponding to segment 4a. This application does not limit this.

[0128] Step S104, obtaining K first sub-networks in the second network model, inputting each video slice into a different first sub-network respectively, and having the K first sub-networks respectively output a video feature vector of each video slice;

[0129] The first sub-network includes a neural network and a multi-layer perceptron (i.e., a multi-layer perceptron). The multi-layer perceptron includes an input layer, a hidden layer, and an output layer; the neural network is used to extract the initial video features of the input video slices, the input layer is used to use the initial video features as the input features of the hidden layer, the hidden layer is used to extract the hidden features of the input features, and the output layer is used to output the video feature vectors of the input video slices according to the hidden features.

[0130] It should be understood that the second network model may be a TSN (Temporal Segment Networks) network model, and the specific execution process of the TSN network model may be expressed as: TSN (T1, T2, ..., T K )=H(G(F(T1;W),F(T2;W),…,F(T K ;W))). Wherein, T1 may represent the first video segment, T2 may represent the second video segment, ..., T K It can represent the Kth video segment, F can represent the video feature vector of each video segment output by the first sub-network (i.e., Convolutional Neural Networks (CNN)), G can represent the features obtained by fusing the K video segments through a feature fusion function (i.e., aggregation function), and H can represent the features obtained after classification through a normalized exponential function (i.e., Softmax function). It should be understood that the embodiment of the present application does not limit the specific type of the second network model.

[0131] Among them, it can be understood that the above-mentioned neural network can be a convolutional neural network, which is a type of feedforward neural network (Feedforward Neural Networks, FNN for short) that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning (DL for short). It has the ability of representation learning and can perform shift-invariant classification on input information according to its hierarchical structure. Among them, the concept of deep learning originates from the research of artificial neural networks (ANN for short). Deep learning can form more abstract high-level representation attribute categories or features by combining low-level features to discover distributed feature representations of data. It can be understood that the fully connected layer in the convolutional neural network can be a multi-layer perceptron (MLP for short), which is a deep learning structure that can perform nonlinear combination of extracted features to obtain output.

[0132] Step S105 , based on the text feature vector, the video feature vector of each video segment and the classifier of the target network model, the style of the target video is identified to obtain a target style label of the target video.

[0133] Specifically, the business server can project the text feature vector and the video feature vector of each video segment to the target vector space. Furthermore, the business server can perform feature fusion on the text feature vector and the video feature vector of each video segment in the target vector space to obtain a fused vector feature. Furthermore, the business server can input the fused vector feature into the classifier of the target network model, and the classifier of the target network model can perform style recognition on the target video to obtain the matching degree between the fused vector feature and the sample attribute feature in the classifier. The matching degree is used to describe the probability that the target video belongs to the sample label corresponding to the sample attribute feature. Furthermore, the business server can use the sample label corresponding to the sample attribute feature with the maximum matching probability from the matching degree as the target style label of the target video. The target style label can be used to push associated videos related to the target video to the first user.

[0134] For easier understanding, see Figure 5 , Figure 5 Schematic diagram of a painting style recognition model provided in an embodiment of the present application. Figure 5 The small video shown is the target video. The specific execution process of the style recognition model (i.e., the target network model) can be expressed as: TSN(V, T1, T2, …, T K)=H(G(F(V),F(T1;W),F(T2;W),…,F(T K ; W))). Wherein, V can represent the target text information, and F(V) can represent the text feature vector output by the first network model.

[0135] It is understandable that if Figure 5 The time slices shown may be the above video slices. For example, time slice 1 (i.e., Fragment 1) may be the above video slices. Figure 4 The fragment 40a shown in the figure, the time fragment 2 (i.e., Fragment 2) can be the above Figure 4 The fragments 40b, ..., time fragment K (i.e., Fragment K) shown in the figure can be the above Figure 4 Shard 40k shown.

[0136] like Figure 5 As shown, when the first sub-network is a convolutional neural network, the service server can input each time slice (i.e., video slice) into different convolutional neural networks, and the convolutional neural network outputs the initial video features of each time slice. Different time slices correspond to different convolutional neural networks, and different convolutional neural networks have their own weight parameters. Furthermore, the service server can input the initial video features of each time slice into a multi-layer perceptron, and the multi-layer perceptron in the convolutional neural network outputs the video feature vector of each time slice.

[0137] like Figure 5 As shown, the business server can also input the target text information (i.e., video subject information (i.e., subject), video tag information (i.e., tag), and video classification information (i.e., classification)) into the first network model, and the Encoder module (i.e., encoder) in the first network model outputs the initial text features of the target text information (i.e., encoding process). The target text information here can include the video subject information, video tag information, and video classification information of the target video. Furthermore, the business server can input the initial text features of the target text information into a multi-layer perceptron, and the multi-layer perceptron outputs the text feature vector of the target text information.

[0138] Furthermore, the service server can combine the text feature vector (i.e., the feature output by F(V)) and the feature vector of each time slice video (i.e., F(T1; W), F(T2; W), ..., F(T K ; W) output features) for multimodal feature fusion, that is, the text feature vector and each time slice video feature vector are input into the feature fusion function to obtain the fused vector feature (i.e. G(F(V), F(T1; W), F(T2; W), …, F(T K; W)) output features). The feature fusion function can combine the target text information and the probability of each time slice relative to all sample labels to obtain a consensus between them on the sample label. For example, the probability that the target text information and each time slice belong to the same sample label can be calculated, for example, by performing mean calculation through a mean function.

[0139] Furthermore, the service server can use the output of the feature fusion function (i.e., the fused vector feature) as the input of the normalized exponential function to obtain the label vector feature corresponding to the fused vector feature (i.e., H(G(F(V), F(T1; W), F(T2; W), …, F(T K ; W))) output features), and then the probability that the short video belongs to different sample labels can be determined according to the label vector features, so that when the value of the maximum matching probability is greater than the probability threshold, the sample label corresponding to the maximum matching probability can be used as the style label of the short video (that is, the target style label of the short video).

[0140] Among them, for the convenience of storage and processing, the normalized exponential function can normalize a real vector so that the real vector meets specific conditions, for example, compressing the value range of real numbers in the real vector so that the range of each real number is between (0,1) and the sum of the real numbers is 1. It can be understood that the normalized exponential function in the embodiment of the present application can normalize the fusion vector feature to obtain the label vector feature corresponding to the fusion vector feature. Therefore, the normalized exponential function can be understood as a classifier of the target network model, and the value in the label vector feature is the probability that the target video belongs to the sample label.

[0141] The specific process of outputting the text feature vector of the target text information through the multi-layer perceptron and the video feature vector of each video segment can be found in Figure 6 , Figure 6 Schematic diagram of the structure of a multilayer perceptron provided in the embodiment of the present application. The layers of the multilayer perceptron are fully connected (that is, the outputs of the nodes in the previous layer are all connected together as the inputs of the interface in the next layer), such as Figure 6 The figure shows the basic structure of a three-layer neural network, where the bottom layer is the input layer (i.e., Layer L1), the middle layer is the hidden layer (i.e., Layer L2), and the top layer is the output layer (i.e., Layer L3). The input layer, hidden layer, and output layer are composed of neurons, and each neuron has its own weight.

[0142] It can be understood that the input layer can be an n-dimensional vector, so that the input layer has n neurons. For example, the input layer can be a 3-dimensional vector X={X1, X2, X3}, so that the input layer has 3 neurons. It should be understood that the input and output of the input layer are the same. When the output of the input layer is vector X, the output of the input layer can also be vector X. In the embodiment of the present application, the sub-network vector (i.e., the initial video features of each video segment (i.e., time segment) and the target text information (i.e., video theme information, video tag information, and video classification information) extracted a priori from the target video can be used as the input of the input layer.

[0143] It can be understood that the hidden layer and the input layer are fully connected. When the output of the input layer is vector X, the input of the hidden layer can be vector X. Therefore, the output of the hidden layer (i.e., a1, a2, and a3) can be: f(W1X+b1), where W1 can be the weight corresponding to each neuron in the input layer (also referred to as the connection coefficient 1), and b1 can be the bias term of the input layer. The function f here can be called an activation function, which can be a commonly used sigmoid function or tanh function. The sigmoid function is selected as the activation function here. The (+1) on the input layer neuron can indicate that the weight of the bias term b1 is (+1). Nonlinearity can be introduced into the output of the neuron through the activation function and the bias term, so as to achieve the purpose of the neuron learning nonlinear representation.

[0144] It can be understood that the hidden layer to the output layer can be regarded as a category of logistic regression (i.e., softmax regression), and the output layer and the hidden layer are fully connected. When the output of the hidden layer is f(W1X+b1), the input of the output layer can be f(W1X+b1). Therefore, the output of the output layer can be: softmax(W2X1+b2), where X1 can represent the output of the hidden layer, W2 can be the weight corresponding to each neuron of the hidden layer (also known as the connection coefficient 2), and b2 can be the bias term of the hidden layer. The softmax function here can be a sigmoid function. The (+1) on the hidden layer neuron can represent that the weight of the bias term b2 is (+1).

[0145] Among them, it is understandable that through Figure 6The multilayer perceptron shown in FIG. 1 can output an n-dimensional (e.g., 1024-dimensional) low-dimensional dense vector, and then obtain the posterior probability: the result F (i.e., the video feature vector of each video segment and the text feature vector of the target text information). Among them, F can represent the probability that each video segment and the target text information belong to different sample labels. For example, F(V) can represent the probability that the target text information belongs to different sample labels, and F(T1; W) can represent the above Figure 5 The probability that the time slice 1 belongs to different sample labels, ..., F(T K ; W) can represent the above Figure 5 The probability that the time slice K shown belongs to different sample labels.

[0146] It is understandable that the feature fusion function can be used to perform feature fusion on the text feature vector and the video feature vector of each video segment in the target vector space to obtain a fused vector feature. The feature fusion method can be a vector splicing method. For example, after performing vector splicing on three 128-dimensional feature vectors, the fused vector features corresponding to the three feature vectors can be obtained. The dimension of the fused vector features here can be 384 dimensions. Optionally, the feature fusion method can also be a weighted average method, and the feature fusion method can also be a maximum or minimum value method. The embodiment of the present application does not limit the specific implementation method of the feature fusion function.

[0147] It should be understood that in the painting (including comics) world, style refers to the style of painting, which mainly includes the author's painting techniques and line treatment for characters, scenery, etc., such as the line treatment and painting techniques for the character's nose, eyes, ears, muscles, and hair. In illustrations and other color paintings, style also includes the author's treatment of other aspects such as color and tone.

[0148] Therefore, the painting style label can represent the style and tonality of a short video (i.e., the target video), that is, the visual style characteristics. The style and tonality of a short video is the overall visual experience of a short video. The content of the same style and tonality has certain commonalities and can resonate with a certain type of consumers. For the same artistic conception, different style tonality (i.e., painting style) can be used to express it to give people different feelings. Among them, the style tonality can include the color style, tone style, light style, and picture style (for example, sense of space, sense of hierarchy) of the short video. In this way, through the definition and distinction of high-level dimensions such as style tonality, short videos can be divided into high-level dimensions.

[0149] It should be understood that the embodiment of the present application can divide the painting style label into three types, and these three types can specifically include: high-key, medium-key and low-key. Among them, the high-key here can specifically include: beautiful scenes, fashion, eye-catching (for example, handsome men and beautiful women), super cool painting style (for example, extreme sports, creative special effects, creative handicrafts, creative experiments, showmanship, luxury cars), etc.; the medium-key here can specifically include: campus painting style (for example, campus jokes, campus life), entrepreneurial inspirational style (for example, entrepreneurship franchise, chef), healing painting style (for example, cute pets, cute babies, two-dimensional, emotions), middle-aged and elderly painting style (for example, square dance, market, calligraphy), etc.; the low-key here can specifically include: agriculture, rural areas and farmers (for example, agriculture-related, rural food, rural life, manufacturing and repair), heavy taste (for example, horror, disgusting), etc. It should be understood that the painting style label in the embodiment of the present application is not available in the existing video tag information and video classification information.

[0150] The embodiment of the present application can accurately identify the target style label of the target video through the target network model with the style recognition function. For example, the embodiment of the present application can extract the text feature vector of the aforementioned target text information through the first network model. It should be understood that the text feature vector here is the auxiliary video vector of the target video. In addition, the embodiment of the present application can also extract the video feature vector of each video segment through the K first sub-networks in the second network model. It should be understood that by inputting different video segments into different first sub-networks respectively, a richer video feature vector can be extracted in the time dimension. In this way, when the text feature vector (i.e., the aforementioned auxiliary video vector) and the richer video feature vector are subjected to vector fusion processing by the classifier in the target network model, the video features obtained by the fusion for accurately characterizing the target video can be subjected to style recognition to accurately obtain the target style label of the target video. It should be understood that the target style label here can be used to characterize the video style and video tonality of the target video, which belongs to a higher-dimensional video classification attribute at a deeper level. Based on this, after video clustering is performed based on the identified target style label, associated videos related to the target video can be quickly and accurately obtained for pushing to the first user. This means that after obtaining the target style label, the accuracy of short video recommendations can be improved when making short video recommendations.

[0151] For further information, see Figure 7 , Figure 7 is a flow chart of a video processing method provided by an embodiment of the present application. The method can be performed by a user terminal (for example, the above Figure 1 The user terminal 3000a shown in the figure may also be executed by a service server (for example, the above Figure 1The video processing method may be executed by the service server 2000 shown in the figure, or may be executed by the user terminal and the service server together. For ease of understanding, this embodiment takes the method executed by the service server as an example for explanation. The video processing method may include the following steps:

[0152] Step S201, obtaining sample videos and sample style labels of the sample videos for training an initial network model;

[0153] The initial network model includes a first sample model and a second sample model.

[0154] It should be understood that the embodiments of the present application can label sample videos through supervised learning to obtain sample style labels of the sample videos.

[0155] Step S202, obtaining sample text information of the sample video, inputting the sample text information into a first sample model, and having the first sample model output a sample text vector corresponding to the sample text information;

[0156] The sample text information may include sample subject information, sample label information and sample classification information of the sample video.

[0157] For the specific process of outputting the sample text vector corresponding to the sample text information through the first sample model, please refer to the description of outputting the text feature vector corresponding to the target text information through the first network model, that is, refer to the above Figure 3 The description of step S102 in the corresponding embodiment will not be repeated here.

[0158] Step S203, evenly dividing the sample video into K sample segments, and performing random frame sampling processing on the K sample segments based on a time sampling strategy to obtain sample segments of each sample segment;

[0159] Here, K can be a positive integer, and the number of video frames in each sample segment is M; here, M can be a positive integer. Here, one sample segment corresponds to one sample slice, and the number of video frames in one sample slice is less than or equal to M.

[0160] It can be understood that when the number of video frames of the target video is equal to the number of video frames of the sample video, the number of video frames of each sample segment in the K sample segments can be equal to the number of video frames of each video segment in the K video segments, that is, M can be equal to N.

[0161] The specific process of the service server obtaining K sample segments based on the sample video can refer to the description of obtaining K video segments based on the target video, that is, refer to the above Figure 3 The description of step S103 in the corresponding embodiment will not be repeated here.

[0162] Step S204, obtaining K second sub-networks in the second sample model, inputting each sample slice into a different second sub-network, and having the K second sub-networks output sample video vectors of each sample slice respectively;

[0163] The specific process of outputting the sample video vector of each sample slice through the K second sub-networks can refer to the description of outputting the video feature vector of each video slice through the K first sub-networks, that is, refer to the above Figure 3 The description of step S104 in the corresponding embodiment will not be repeated here.

[0164] Step S205, iteratively training the initial network model based on the sample style label, the sample text vector, the sample video vector of each sample slice, and the classifier of the initial network model, and using the iteratively trained initial network model as the target network model;

[0165] Specifically, the business server can iteratively train the initial network model based on the sample style label, the sample text vector, the sample video vector of each sample segment, and the classifier of the initial network model to obtain the model training result. Furthermore, when the model training result indicates that the model loss function of the initial network model after iterative training does not meet the model convergence condition, the business server can use the initial network model after iterative training as the transition network model. Furthermore, the business server can adjust the model parameters of the transition network model based on the model loss function that does not meet the model convergence condition, until the adjusted transition network model meets the model convergence condition, and use the transition network model that meets the model convergence condition as the target network model. It should be understood that through the trained target network model, richer image features can be extracted to ensure the accuracy of short video recommendations.

[0166] It should be understood that the loss function in the model training process can be used to represent the degree of difference between the predicted value and the actual value. The smaller the loss function, the better the model. Therefore, the goal of training a machine learning model is to find the point where the loss function reaches a minimum value. When the sample style labels of the sample video can be divided into two categories (for example, the sample style labels of the sample video can be handsome men or beautiful women, that is, the predicted values ​​can be divided into handsome men (positive) or beautiful women (negative)), the binary cross entropy can be used as the loss function of the model, that is, the loss function can be expressed as: C = -(1 / n)∑ n [ylna+(1-y)ln(1-a)], where n is the number of sample videos, y is the actual value of the sample style label (when the actual value is a handsome guy, y can be 1; when the actual value is a beautiful girl, y can be 0), and a is the probability that the predicted value is a handsome guy (i.e., positive).

[0167] It can be understood that the embodiment of the present application can divide the sample videos into multiple categories (for example, the sample style labels of the sample videos can be extreme sports, cute kids, agriculture-related, etc.), and the multi-classification cross entropy can be used as the loss function of the model, that is, the loss function can be expressed as: Here n is the number of sample videos, m is the number of sample style labels, and y is c It can represent an indicator variable (when the actual value is the same as the sample style label c, the indicator variable can be 1; when the actual value is different from the sample style label c, the indicator variable can be 0). Here a c It can represent the probability that the predicted value is the sample style label c. For example, the loss function value of sample video 1 is: loss1 = -(0×ln0.3 + 0×ln0.3 + 1×ln0.4) = 0.91, the loss function value of sample video 2 is: loss2 = -(0×ln0.3 + 1×ln0.4 + 0×ln0.3) = 0.91, and the loss function value of sample video 3 is: loss3 = -(1×ln0.1 + 0×ln0.2 + 0×ln0.7) = 2.30, then the loss function value of all sample videos is: C = (1 / 3) × (0.91 + 0.91 + 2.30) = 1.37.

[0168] For easier understanding, see Figure 8 , Figure 8 This is a system flow chart of a painting style tag recognition provided by an embodiment of the present application. Figure 8 As shown, number 11-number 20 can be an execution path, number 31-number 32 can be an execution path, number 41-number 42 can be an execution path, number 51-number 52 can be an execution path, number 61-number 64 can be an execution path, and these five execution paths can be synchronously cross-executed to form a process for performing painting style recognition in an embodiment of the present application. It can be understood that the execution paths of number 41-number 42 and number 61-number 64 can correspond to the description of step S201-step S205.

[0169] like Figure 8As shown, in the execution path from number 41 to number 42, the business server can call the video frame extraction service to obtain the sample video (i.e., downloaded video file) used to train the initial network model from the video content storage server (i.e., the small video distribution content library), and then evenly divide the sample video and randomly extract frames to obtain sample segments of the sample video (i.e., frame extraction information). In addition, the business server can obtain the sample text information of the sample video (i.e., content metadata, where the content metadata mainly includes the sample subject information, sample classification information, and sample label information of the sample video), and then store the frame extraction information and content metadata in the sample database.

[0170] It should be understood that the video frame extraction service can be completed by a video frame extraction server, which is usually a group of parallel servers, composed of related task scheduling and distribution clusters, and can control the speed and progress of video file downloads.

[0171] Among them, the video content storage server can provide video content storage services. Among them, the video content storage server can be a group of storage servers with a wide distribution range and close to the user. Usually, the periphery can be accelerated by distributed caching through a CDN (Content Distribute Network) acceleration server. The CDN acceleration server can redirect the user's request to the service node closest to the user (i.e. the above-mentioned storage server) in real time based on network traffic and the links and load conditions between each node, as well as the user's distance, response time and other comprehensive information, so that the user can get the required content nearby, effectively solve the congestion of the storage server network, and thus improve the speed at which the user accesses the storage server.

[0172] like Figure 8 As shown, in the execution path numbered 61-64, the business server can call the style content tagging service to read the sample video from the content database (the sample video in the content database can be understood as the distributable video stored in the video content storage server, and the distributable video here indicates that the sample video has passed the deduplication processing and video review), and manually mark the sample style label of the sample video through the manual review system, and then the sample style label can be stored in the sample database.

[0173] Furthermore, the business server can obtain sample style labels, sample text information (i.e., content metadata) and sample fragments (i.e., frame extraction information) from the sample database, and iteratively train the initial network model based on the sample style labels, content metadata, frame extraction information and the classifier in the initial network model, and use the trained initial network model as Figure 8The video style recognition model shown (i.e., the target network model) can then be turned into a service.

[0174] Step S206, obtaining the target video uploaded by the second user through the dispatch center service component, performing deduplication processing on the target video to obtain a deduplication processing result, and storing the target video in the content database based on the deduplication processing result;

[0175] Specifically, the business server can obtain the target video uploaded by the second user through the dispatch center service component, and configure the video fingerprint feature for the target video. Further, the business server can obtain the historical fingerprint feature of each historical video in the content database, and determine the vector distance between each historical fingerprint feature and the video fingerprint feature. Further, if there is no vector distance greater than the vector threshold associated with the historical video in the vector distance, the business server can determine that the video fingerprint feature meets the deduplication processing condition, and store the target video uploaded by the second user in the content database. Among them, if the video fingerprint feature meets the deduplication processing condition, it is determined that there is no historical video matching the target video in the content database.

[0176] Optionally, if there is a vector distance less than the vector threshold in the vector distance, the business server can determine that the video fingerprint feature does not meet the deduplication processing conditions, and determine that there is a historical video matching the target video in the content database, and delete the target video uploaded by the second user.

[0177] It should be understood that by comparing the historical fingerprint features with the video fingerprint features, the similarity between the target video and the historical video can be obtained, so that when the similarity meets the similarity threshold, the target video can be deleted. In this way, only one duplicate or similar video can be retained in the business server, and then the video can be subsequently processed to reduce the duplicate or similar videos in the subsequent processing process, so as to effectively construct the videos that need to be processed in the video review process and improve the efficiency of video review. It should be understood that through deduplication processing, it can be effectively identified whether the second user obtains the target video based on the historical video through format conversion, editing (for example, changing the bit rate, clarity, size, filter, etc.), cropping and splicing, compression and rotation, etc.

[0178] Step S207, performing video review on the target video according to the target video and meta information of the target video read from the content database to obtain a video review result;

[0179] Specifically, the business server can read the target video from the content database, and read the first auxiliary information, the second auxiliary information, and the third auxiliary information of the target video from the content database. Among them, the first auxiliary information is the information entered by the second user when uploading the target video; the second auxiliary information is obtained after the target video is classified by the video classification model; the third auxiliary information is obtained after transcoding the target video. Further, the business server can use the first auxiliary information, the second auxiliary information, and the third auxiliary information as the meta-information of the target video, perform a first video review on the target video based on the meta-information, and obtain an initial video review result. Furthermore, if the initial video review result indicates that the target video is legal, the business server can perform a second video review on the target video to obtain a target video review result, and use the target video review result as the video review result of the target video.

[0180] It is understandable that the first auxiliary information may include the video theme information (i.e., title), video time information (i.e., release time), video summary information, and video author information (i.e., publisher) of the target video; the second auxiliary information may include the video tag information and video classification information of the target video; the third auxiliary information may include the video size information (i.e., file size, for example, 3.2MB), cover link information, video bit rate information (e.g., 4818kbps), video size information (i.e., specifications, for example, 1280*720 pixels), and video format information (e.g., avi) of the target video. It should be understood that the embodiments of the present application do not limit the types of information contained in the above-mentioned first auxiliary information, second auxiliary information, and third auxiliary information.

[0181] It can be understood that the video classification model can analyze the target video to obtain the second auxiliary information of the target video. The present application does not limit the model type of the video classification model.

[0182] It is understandable that video transcoding can convert a compressed and encoded video stream into another video stream to adapt to different network bandwidths, different terminal processing capabilities and different user needs. Therefore, video transcoding can standardize the target video and improve the compatibility of the video on various platforms by saving the target video's meta information.

[0183] It is understood that the video classification information may include a primary classification, a secondary classification, and a tertiary classification, and the number of video tag information may be one or more. For example, a video about a mobile phone (e.g., mobile phone J, where mobile phone J may be a mobile phone brand or a mobile phone model) may have a primary classification of technology, a secondary classification of mobile phones, a tertiary classification of domestic mobile phones, and video tag information such as mobile phone J, Snapdragon 855, etc.

[0184] It is understandable that the first video review can manually conduct a preliminary filtering of whether the target video involves pornography, gambling, drugs and sensitive characteristics, that is, review the legality of the target video.

[0185] It should be understood that since machine learning (e.g., deep learning) is not yet fully mature, it is not completely accurate to determine the video tag information and video classification information of the target video entirely through machine learning. It is necessary to manually review the second auxiliary information based on the second auxiliary information obtained in the video classification model, and improve the accuracy and efficiency of target video annotation through human-machine collaboration. It is understandable that the second video review can manually confirm the video tag information and video classification information of the target video, that is, review the correctness of the meta-information (e.g., second auxiliary information) of the target video. In addition, the second review can also manually review the quality of the target video content (e.g., the video is not nutritious, the video is incomplete) and security, and other quality issues that are difficult for machines to identify, that is, review the integrity of the target video.

[0186] Step S208, obtaining a target video and a target network model corresponding to the target video;

[0187] Step S209, obtaining target text information of the target video, inputting the target text information into the first network model, and having the first network model output a text feature vector corresponding to the target text information;

[0188] Step S210, evenly dividing the target video into K video segments, and performing random frame sampling processing on the K video segments based on a time sampling strategy to obtain video segments of each video segment;

[0189] Step S211, obtaining K first sub-networks in the second network model, inputting each video slice into a different first sub-network respectively, and having the K first sub-networks respectively output a video feature vector of each video slice;

[0190] Step S212, based on the text feature vector, the video feature vector of each video segment and the classifier of the target network model, the style of the target video is identified to obtain a target style label of the target video;

[0191] Step S213, when the video review result indicates that the target video is a distributable video, start the content distribution service, call the content distribution service to obtain the aggregation cluster mapped by the target style label from the video recommendation database, and add the distributable video to the aggregation cluster;

[0192] For easier understanding, see Figure 8In the execution path from number 11 to number 20, the second user (i.e., the production end) can upload the target video to the uplink and downlink content interface server through the C-end publishing system or the web publishing system (i.e., the video upload interface). When the uplink and downlink interface server obtains the target video uploaded by the second user, it can store the target video (i.e., the source file) to the video content storage server, so that the video content storage server transcodes the target video. It can be understood that the web publishing system here can be a browser in a browser / server (Browser / Server, B / S) architecture, and the C-end publishing system here can be a client in a client / server (C / S) architecture.

[0193] Furthermore, the uplink and downlink content interface servers can obtain the metadata of the target video (the metadata here may include first auxiliary information, second auxiliary information and third auxiliary information), write the metadata into the content database, and directly upload the target video to the scheduling center server (i.e., the scheduling center service component) for subsequent content processing and circulation.

[0194] Furthermore, the dispatch center server can obtain the target video, call the video deduplication service to perform deduplication processing on the target video, and write the deduplication processing result into the content database. When the deduplication processing passes, the target video can be reviewed by the manual review system; when the deduplication processing fails, the target video can be deleted. Therefore, in the system for identifying the style label of short videos (i.e., the system for identifying the content tone style), the target video that fails the deduplication processing will not be reviewed by the manual review system.

[0195] Furthermore, when the manual review system conducts a video review on the target video, the target video (i.e., the original content) and the meta information of the target video can be read from the content database to conduct a first video review (i.e., preliminary review) and a second video review (i.e., review) on the target video. It can be understood that when the first video review of the target video passes, the second video review can be conducted on the target video, and when the second video review of the target video passes, the target video can be used as a distributable video; when the first video review of the target video passes but the second video review fails, the meta information of the target video can be updated, and the target video that meets the integrity can be used as a distributable video. The manual review system is usually a system with complex business developed based on a web database.

[0196] like Figure 8As shown, in the execution path numbered 51-52, the scheduling center server can call the video style recognition service, and through the video style recognition model, perform style recognition on the target video to obtain the target style label of the target video, and then add the target video to the content pool corresponding to the target style label (i.e., the aggregation cluster corresponding to the target style label in the video recommendation database).

[0197] Therefore, the dispatch center server can call the content distribution export service, enable the content (ie, the target video) to start distribution, obtain at least one recommended video from the content pool, and distribute at least one recommended video to the video content consumption end (eg, the first terminal corresponding to the first user).

[0198] It should be understood that after the target video is uploaded to the upstream and downstream content interface servers at the production end, the target video can enter the server end through the upstream and downstream content interface servers. The server end here can be a server program deployed on multiple groups of servers and specifically provides remote network services for application clients (i.e., terminal programs). The embodiments of the present application can collectively refer to the multiple groups of servers here as business servers.

[0199] Among them, it can be understood that the embodiment of the present application can refer to the server that implements the video content storage service as a video content storage server, the server that implements the video deduplication service as a video deduplication server, the server that implements the video frame extraction service as a video frame extraction server, the server that implements the video style recognition service as a video style recognition server, the server that implements the content distribution export service as a content distribution export server, and the server that implements the style content marking service as a style content marking server, and then the upstream and downstream content interface servers, the scheduling center server, the video content storage server, the video deduplication server, the video frame extraction server, the video style recognition server, the content distribution export server and the style content marking server can be collectively referred to as business servers.

[0200] Step S214, obtaining a user portrait of the first user, obtaining at least one associated video to be pushed to the first user from the aggregation cluster based on the target style tag carried in the user portrait, and pushing the at least one associated video to the first user.

[0201] It is understandable that the business server can obtain the behavior log table of the first user, and extract the historical video information associated with the first user from the behavior log table. The historical video information is determined based on the historical user behavior of the first user in the video client. Furthermore, the business server can obtain the historical style tag of the historical video information, and determine the user portrait used to characterize the first user based on the historical style tag and the target style tag. The target style tag can be the style tag of the target video, and the target video is the video played by the first user in the application client (for example, the video client). It is understandable that the target video here can be the video that the first user is watching in the video client at the current moment, and can also be a video that the first user has watched in the video client at a historical moment.

[0202] Among them, it can be understood that the historical video information is used to record the historical user behavior (access, browsing, searching, clicking, etc.) generated when the first user accesses the application client (for example, a video client). The historical user behavior here may include the type of video accessed by the first user, the time of browsing the video, the number of times the video is browsed, the record of searching for the video, the number of times the video is clicked, as well as the first user's favorite videos, recommended videos, liked videos, purchased videos, coined videos, forwarded videos, commented videos, etc. Therefore, the user portrait of the first user can be deposited on the style tags of these videos corresponding to the historical user behavior.

[0203] It can be understood that the user portrait may include: a short-term user portrait determined by short-term interests and a long-term user portrait determined by long-term interests. Therefore, the short-term user portrait and long-term user portrait of the first user can be determined through the recording period of historical user behavior.

[0204] It is understandable that the embodiment of the present application can recommend videos to the first user through user portraits and video recommendation algorithms. It should be understood that the video recommendation algorithm for recommending related videos to the first user can be a collaborative recommendation algorithm, a matrix decomposition algorithm, a supervised learning algorithm (e.g., a logistic regression model), based on a deep learning model (e.g., a factor decomposition machine and a gradient boosting decision tree (Gradient Boosting Decision Tree, GBDT for short)), etc., and the present application does not limit this.

[0205] For example, collaborative recommendation algorithms can be divided into user-based recommendations and item-based recommendations. In the case of user-based recommendations, other users (e.g., user Y2) with the same preferences as the user (e.g., user Y1) can be found based on the preferences of the user (e.g., user Y1) for items, and recommendations can be made to user Y1 based on the preferences of user Y2. For example, if user Y1 and user Y2 have similar interests, the video (e.g., video S) that user Y2 likes can be recommended to user Y1. In the case of item-based recommendations, the similarity between items can be found based on the preferences of the user (e.g., user Y) for items (e.g., item W1), and similar items (e.g., item W2) can be recommended to user Y based on the preferences of user Y. For example, if user Y likes the video corresponding to item W1 (e.g., video S1), and item W2 has a high similarity with item W1, the video corresponding to item W2 (e.g., video S2) can be recommended to user Y. The item here can be understood as the style label of the video.

[0206] For easier understanding, see Fig. 9 , Fig. 9 is a schematic diagram of a scene for performing style recognition provided by an embodiment of the present application. Fig. 9 There are 4 target videos, which may specifically include: video 9a, video 9b, video 9c and video 9d. After performing style recognition on video 9a, it can be determined that the target style label of video 9a is extreme sports. Therefore, video 9a can be added to cluster 1 in the video recommendation database, where cluster 1 may be a cluster mapped to extreme sports.

[0207] Similarly, after performing style recognition on video 9b, it can be determined that the target style label of video 9b is cute baby. Therefore, video 9b can be added to cluster 2 in the video recommendation database, where cluster 2 can be the cluster mapped to cute baby. Similarly, after performing style recognition on video 9c, it can be determined that the target style label of video 9c is cute pet. Therefore, video 9c can be added to cluster 3 in the video recommendation database, where cluster 3 can be the cluster mapped to cute pet. Similarly, after performing style recognition on video 9d, it can be determined that the target style label of video 9d is agriculture-related. Therefore, video 9d can be added to cluster m in the video recommendation database, where cluster m can be the cluster mapped to agriculture-related.

[0208] It should be understood that the video processing method provided in the embodiment of the present application can be used to build a short video high-quality content pool (i.e., a video recommendation database). In this way, the business server can directly obtain at least one related video from the aggregate in the video recommendation database and push at least one related video to the first user.

[0209] Among them, it can be understood that the attribute of extreme sports can be high-key, the attribute of cute babies and cute pets can be medium-key, and the attribute of agriculture-related data can be low-key. Therefore, the attribute of cluster 1 can be high-key, the attributes of clusters 2 and 3 can be medium-key, and the attribute of cluster m can be low-key. Optionally, the embodiment of the present application can also add the target video to the clusters with the same attributes, so that the number of clusters in the video recommendation database can be 3 (for example, cluster m1, cluster m2 and cluster m3, where cluster m1 can be the cluster corresponding to high-key, cluster m2 can be the cluster corresponding to medium-key, and cluster m3 can be the cluster corresponding to low-key). For example, the business server can add video 9a to cluster m1, can add video 9b and video 9c to cluster m2, and can add video 9d to cluster m3.

[0210] It should be understood that the cold start of videos with the same style tag can be achieved through the style tag to recommend videos to users. For example, when a video (for example, video S1) uploaded by a certain user (for example, user Y1) is a new video, the style tag of video S1 can be identified to recommend video S1 to a certain user (for example, user Y2) who likes videos with the style tag. Optionally, the cold start of users with the same style tag can be achieved through the style tag to recommend videos to the user. For example, when a certain user (for example, user Y) is a newly registered user, videos related to the style tag (for example, video S2) can be recommended to user Y based on the style tag selected by user Y.

[0211] It is understandable that the user terminal can present at least one related video to the first user (i.e., content consumer) in the form of an information stream, i.e., a Feeds stream. Here, the Feeds stream is a data format, and the server can recommend the content that the first user is interested in to the user terminal so that the user terminal can display these contents in the form of a Feeds stream and quickly refresh these contents. Among them, the Feeds stream is usually sorted in a timeline (i.e., Timeline), that is, the Timeline is the most intuitive display form of the Feeds stream. Among them, the videos (i.e., related videos) recommended to the first user by the information stream can include vertical or horizontal videos.

[0212] It should be understood that the aggregator can aggregate the above-mentioned Feeds streams together. The aggregator here refers to software used to aggregate Feeds streams. For example, the aggregator can be software specifically used to subscribe to URLs (different URLs correspond to different servers). The aggregator can also be called an RSS (Really Simple Syndication) reader, feed reader, news reader, etc.

[0213] For easier understanding, see Fig.10a , Fig.10a FIG. 1 is a schematic diagram of a video distribution scenario provided by an embodiment of the present application. Fig.10a As shown, when a first user (e.g., user Y1) performs a trigger operation (e.g., a click operation) on the video client 1 in the client display interface, a client startup request can be sent to the service server, and the service server can respond to the client startup request and select one or more associated videos (e.g., video S1, video S2, video S3, and video S4) that user Y1 may be interested in based on the user portrait of user Y1. When the first terminal used by user Y1 opens the video client 1, the service server can output the one or more associated videos that user Y1 may be interested in to the video client 1.

[0214] Among them, the style label of video S1 can be "handsome men and beautiful women", the style label of video S2 can be "cute pets", the style label of video S3 can be "beautiful scenery", and the style label of video S4 can be "showy skills". Therefore, when analyzing the user portrait of user Y1, it can be determined that the style labels liked by user Y1 include: "handsome men and beautiful women", "cute pets", "beautiful scenery" and "showy skills". It should be understood that the number of style labels liked by user Y1 can be one or more, and this application does not limit this.

[0215] For easier understanding, see Fig.10b , Fig.10b Schematic diagram of a video recommendation scenario provided by an embodiment of the present application. Fig.10b As shown, the video client 2 can be a client used by the first user (for example, user Y2). When user Y2 performs a trigger operation (for example, a click operation) on the business control in the video client 2, a video recommendation request can be sent to the business server. The business server can respond to the video recommendation request and obtain the target style tag of the target video being watched by user Y2. The target video being watched here can be a video in the video playback interface, and then one or more associated videos (for example, video S11, video S22, video S33 and video S44) with the same style tag as the target video being watched by user Y2 can be selected, and these associated videos can be output to the video client 2.

[0216] Among them, the style label of video S11, video S22, video S33 and video S44 can be "cute pets", so the target style label of the target video being watched by user Y2 can be: "cute pets".

[0217] Optionally, the trigger operation of user Y2 on application client 2 may also be a sliding operation, and the service server may output an associated video having the same style label as the target video being watched by user Y2 to video client 2 based on the video recommendation algorithm. This application does not limit this.

[0218] For easier understanding, see Fig.10c , Fig.10c Schematic diagram of a video search scenario provided by an embodiment of the present application. Fig.10c As shown, the video client 3 may be a client used by the first user (for example, user Y3). When user Y3 performs a trigger operation (for example, a click operation) on the search control in the video client 3, a search request carrying a search text may be sent to the business server. The search text here may be "cute pets". The business server may respond to the search request and identify the style tag corresponding to the search text. For example, the style tag corresponding to the search text "cute pets" is "cute pets". Then, one or more associated videos with the style tag "cute pets" (for example, video S111, video S222, video S333, and video S444) may be selected, and these associated videos may be output to the video client 3.

[0219] It is understandable that the first user can perform a trigger operation on one or more videos in at least one associated video to select to watch one or more videos. Figure 8 As shown, in the execution path of number 31-number 32, when the video content consumer (i.e., the first user) chooses to watch a certain video (e.g., video S), the index information of the video S can be obtained from the uplink and downlink content interface server. The index information here can be the URL (Uniform Resource Locator) address corresponding to the video S. Then, the video content consumer can directly obtain the streaming media file (i.e., source file) of the video S from the video content storage server based on the URL address, and play the obtained streaming media file through a local player. It should be understood that the use of streaming media files does not require the media file of the entire video to be downloaded at one time, but the video is compressed to obtain data packets, and the data packets are sent in segments in the network, so that the data packets can be sent like a stream.

[0220] It is understandable that if Figure 8As shown, when the production end is uploading the target video and the video content consumption end is downloading the associated video, the behavior data, freeze, loading time, play clicks, etc. of the production end and the video content consumption end can be reported to the statistics server (not shown in the figure). Among them, the video content consumption end includes an obvious content quality feedback entry. When providing feedback on the content quality, the content reported to the statistics server may include: the type of quality problem, the above-mentioned quality characteristics, the identification information of the video, etc.

[0221] It should be understood that the video processing method in the embodiment of the present application can also be referred to as a short video content tonality style recognition method, and the short video content tonality style recognition method can be applied to Figure 8 The painting style label recognition system in the corresponding embodiment (i.e., the short video content tonality style recognition system).

[0222] It can be seen that the embodiment of the present application can intelligently identify the style label of the target video through the trained style recognition model (i.e., the target network model). It should be noted that the target network model here may include a first network model for extracting the text feature vector of the target video and a second network model for extracting the video feature vector of each video segment of the target video, and the second network model may include K first subnetworks, each first subnetwork corresponding to a video segment. It should be understood that the number of frames of each video segment randomly extracted based on the time sampling strategy may be partially the same or completely different. It can be seen that the present invention can input different video segments into different first subnetworks respectively, so as to extract the video feature vector of each video segment respectively through different first subnetworks, and then in the process of identifying the style label, more abundant features can be extracted, so as to improve the accuracy of the recognition of the style label. Based on this, when the style label obtained by identification is used to recommend videos to users, the accuracy of the video recommendation can be improved, and then the stickiness of the user's consumption video can be improved.

[0223] For further information, see Fig.11 , Fig.11 : is a structural diagram of a video processing device provided by an embodiment of the present application. The video processing device 1 may include: a target acquisition module 10, a first output module 20, a first determination module 30, a second output module 40, and a painting style recognition module 50; further, the video processing device 1 may also include: an initial acquisition module 60, a third output module 70, a second determination module 80, a fourth output module 90, an initial training module 100, a duplicate removal processing module 110, a video review module 120, a distribution activation module 130, and a video push module 140;

[0224] A target acquisition module 10 is used to acquire a target video and a target network model corresponding to the target video; the target network model includes a first network model and a second network model;

[0225] The target acquisition module 10 includes: a video acquisition unit 101, a target acquisition unit 102;

[0226] The video acquisition unit 101 is used to call the video style recognition service through the scheduling center service component, and acquire the target video uploaded by the second user through the video upload interface through the video style recognition service;

[0227] The target acquisition unit 102 is used to acquire a target network model for performing style recognition on a target video through a video style recognition service.

[0228] The specific implementation of the video acquisition unit 101 and the target acquisition unit 102 can be found in the above Figure 3 The description of step S101 in the corresponding embodiment will not be repeated here.

[0229] A first output module 20, used to obtain target text information of a target video, input the target text information into a first network model, and output a text feature vector corresponding to the target text information by the first network model;

[0230] The first output module 20 includes: a text acquisition unit 201, a text segmentation unit 202, a feature extraction unit 203, and a text determination unit 204;

[0231] A text acquisition unit 201 is used to acquire video subject information, video classification information and video tag information of a target video, and use the video subject information, video classification information and video tag information as target text information of the target video;

[0232] The text segmentation unit 202 is used to perform a segmentation process on the target text information to obtain the text segmentation of the target text information, and perform a vector conversion on the text segmentation to obtain a text segmentation vector corresponding to the text segmentation;

[0233] A feature extraction unit 203 is used to input the text word segmentation vector into the first network model, and extract the vector features of the text word segmentation vector by the first network model;

[0234] The text determination unit 204 is used to determine the text position of the text segment in the target text information, and determine the text feature vector corresponding to the target text information based on the text position and the extracted vector features.

[0235] The specific implementation of the text acquisition unit 201, the text segmentation unit 202, the feature extraction unit 203 and the text determination unit 204 can be found in the above Figure 3 The description of step S102 in the corresponding embodiment will not be repeated here.

[0236] The first determination module 30 is used to evenly divide the target video into K video segments, and randomly sample frames for the K video segments based on a time sampling strategy to obtain video segments of each video segment; K is a positive integer, and the number of video frames in each video segment is N; N is a positive integer; one video segment corresponds to one video segment, and the number of video frames in one video segment is less than or equal to N;

[0237] The second output module 40 is used to obtain K first sub-networks in the second network model, input each video slice into a different first sub-network, and output the video feature vector of each video slice by the K first sub-networks;

[0238] Among them, the first subnetwork includes a neural network and a multilayer perceptron; wherein the multilayer perceptron includes an input layer, a hidden layer and an output layer; the neural network is used to extract the initial video features of the input video segment; the input layer is used to use the initial video features as the input features of the hidden layer, and the hidden layer is used to extract the hidden features of the input features; the output layer is used to output the video feature vector of the input video segment according to the hidden features.

[0239] The style recognition module 50 is used to identify the style of the target video based on the text feature vector, the video feature vector of each video segment and the classifier of the target network model to obtain a target style label of the target video; the target style label is used to push associated videos related to the target video to the first user.

[0240] The painting style recognition module 50 includes: a vector projection unit 501, a vector fusion unit 502, a matching degree determination unit 503, and a label determination unit 504;

[0241] A vector projection unit 501, used to project the text feature vector and the video feature vector of each video slice into a target vector space;

[0242] A vector fusion unit 502 is used to perform feature fusion on the text feature vector and the video feature vector of each video slice in the target vector space to obtain a fused vector feature;

[0243] The matching degree determination unit 503 is used to input the fused vector feature into the classifier of the target network model, and the classifier of the target network model performs style recognition on the target video to obtain the matching degree between the fused vector feature and the sample attribute feature in the classifier; the matching degree is used to describe the probability that the target video belongs to the sample label corresponding to the sample attribute feature;

[0244] The label determination unit 504 is used to select the sample label corresponding to the sample attribute feature with the maximum matching probability from the matching degree as the target style label of the target video.

[0245] The specific implementation of the vector projection unit 501, the vector fusion unit 502, the matching degree determination unit 503 and the label determination unit 504 can be found in the above Figure 3 The description of step S105 in the corresponding embodiment will not be repeated here.

[0246] Optionally, an initial acquisition module 60 is used to acquire sample videos and sample style labels of the sample videos for training an initial network model; the initial network model includes a first sample model and a second sample model;

[0247] The third output module 70 is used to obtain sample text information of the sample video, input the sample text information into the first sample model, and the first sample model outputs a sample text vector corresponding to the sample text information;

[0248] The second determination module 80 is used to evenly divide the sample video into K sample segments, and randomly sample the K sample segments based on the time sampling strategy to obtain sample segments of each sample segment; K is a positive integer, and the number of video frames in each sample segment is M; M is a positive integer; one sample segment corresponds to one sample segment, and the number of video frames in one sample segment is less than or equal to M;

[0249] A fourth output module 90 is used to obtain K second sub-networks in the second sample model, input each sample slice into a different second sub-network, and output a sample video vector of each sample slice by the K second sub-networks;

[0250] The initial training module 100 is used to iteratively train the initial network model based on the sample style label, the sample text vector, the sample video vector of each sample segment and the classifier of the initial network model, and use the iteratively trained initial network model as the target network model.

[0251] The initial training module 100 includes: an iterative training unit 1001, a transition determination unit 1002, and a target determination unit 1003;

[0252] An iterative training unit 1001 is used to iteratively train the initial network model based on the sample painting style label, the sample text vector, the sample video vector of each sample slice, and the classifier of the initial network model to obtain a model training result;

[0253] A transition determination unit 1002 is used to use the initial network model after iterative training as a transition network model when the model training result indicates that the model loss function of the initial network model after iterative training does not meet the model convergence condition;

[0254] The target determination unit 1003 is used to adjust the model parameters of the transition network model based on the model loss function that does not meet the model convergence condition, until the adjusted transition network model meets the model convergence condition, and use the transition network model that meets the model convergence condition as the target network model.

[0255] The specific implementation of the iterative training unit 1001, the transition determination unit 1002 and the target determination unit 1003 can be found in the above Figure 7 The description of step S205 in the corresponding embodiment will not be repeated here.

[0256] Optionally, a duplicate removal processing module 110 is used to obtain the target video uploaded by the second user through the dispatch center service component, perform duplicate removal processing on the target video, obtain a duplicate removal processing result, and store the target video in the content database based on the duplicate removal processing result;

[0257] The duplicate removal processing module 110 includes: a feature configuration unit 1101, a distance determination unit 1102, and a video storage unit 1103; optionally, the duplicate removal processing module 110 may further include: a video deletion unit 1104;

[0258] The feature configuration unit 1101 is used to obtain the target video uploaded by the second user through the scheduling center service component, and configure the video fingerprint feature for the target video;

[0259] The distance determination unit 1102 is used to obtain the historical fingerprint feature of each historical video in the content database and determine the vector distance between each historical fingerprint feature and the video fingerprint feature;

[0260] The video storage unit 1103 is used to determine that the video fingerprint feature meets the duplicate elimination processing condition if there is no vector distance greater than the vector threshold associated with the historical video in the vector distance, and store the target video uploaded by the second user in the content database.

[0261] Optionally, the video deletion unit 1104 is used to determine that the video fingerprint feature does not meet the deduplication processing conditions if there is a vector distance less than the vector threshold in the vector distance, and determine that there is a historical video matching the target video in the content database, and delete the target video uploaded by the second user.

[0262] The specific implementation of the feature configuration unit 1101, the distance determination unit 1102, the video storage unit 1103 and the video deletion unit 1104 can be found in the above Figure 7 The description of step S206 in the corresponding embodiment will not be repeated here.

[0263] The video review module 120 is used to review the target video according to the target video and the meta information of the target video read from the content database to obtain the video review result;

[0264] The video review module 120 includes: a video reading unit 1201, a first determination unit 1202, and a second determination unit 1203;

[0265] The video reading unit 1201 is used to read the target video from the content database, and read the first auxiliary information, the second auxiliary information and the third auxiliary information of the target video from the content database; the first auxiliary information is the information entered by the second user when uploading the target video; the second auxiliary information is obtained after the target video is classified by the video classification model; the third auxiliary information is obtained after transcoding the target video;

[0266] The first determining unit 1202 is configured to use the first auxiliary information, the second auxiliary information, and the third auxiliary information as meta information of the target video, and perform a first video review on the target video based on the meta information to obtain an initial video review result;

[0267] The second determination unit 1203 is used to perform a second video review on the target video if the initial video review result indicates that the target video is legal, to obtain the target video review result, and to use the target video review result as the video review result of the target video.

[0268] The specific implementation of the video reading unit 1201, the first determination unit 1202 and the second determination unit 1203 can be found in the above Figure 7 The description of step S207 in the corresponding embodiment will not be repeated here.

[0269] The distribution activation module 130 is used to start the content distribution service when the video review result indicates that the target video is a distributable video, call the content distribution service to obtain the aggregation cluster mapped by the target style label from the video recommendation database, and add the distributable video to the aggregation cluster;

[0270] The video push module 140 is used to obtain a user portrait of the first user, obtain at least one associated video to be pushed to the first user from the aggregation cluster based on the target style tag carried in the user portrait, and push the at least one associated video to the first user.

[0271] The target video is a video played by the first user in the video client;

[0272] The video push module 140 includes: an information acquisition unit 1401, a portrait determination unit 1402;

[0273] The information acquisition unit 1401 is used to acquire a behavior log table of the first user, and extract historical video information associated with the first user from the behavior log table; the historical video information is determined based on the historical user behavior of the first user in the video client;

[0274] The portrait determining unit 1402 is used to obtain a historical style tag of the historical video information, and determine a user portrait for representing the first user based on the historical style tag and the target style tag.

[0275] The specific implementation of the information acquisition unit 1401 and the image determination unit 1402 can be found in the above Figure 7 The description of step S214 in the corresponding embodiment will not be repeated here.

[0276] The specific implementation of the target acquisition module 10, the first output module 20, the first determination module 30, the second output module 40 and the painting style recognition module 50 can be seen in the above Figure 3 The description of step S101 to step S105 in the corresponding embodiment will not be repeated here. Optionally, the specific implementation of the initial acquisition module 60, the third output module 70, the second determination module 80, the fourth output module 90, the initial training module 100, the duplicate removal processing module 110, the video review module 120, the distribution activation module 130 and the video push module 140 can be referred to above. Figure 7 The description of steps S201 to S207 and steps S213 to S214 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here either.

[0277] For further information, see Fig.12 , Fig.12 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.12As shown, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. Optionally, the network interface 1004 may include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. Optionally, the memory 1005 may also be at least one storage device located away from the aforementioned processor 1001. As Fig.12 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0278] In such Fig.12 In the computer device 1000 shown, the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0279] Obtain a target video and a target network model corresponding to the target video; the target network model includes a first network model and a second network model;

[0280] Obtain target text information of a target video, input the target text information into a first network model, and have the first network model output a text feature vector corresponding to the target text information;

[0281] The target video is evenly divided into K video segments, and the K video segments are randomly sampled based on the time sampling strategy to obtain video segments of each video segment; K is a positive integer, and the number of video frames in each video segment is N; N is a positive integer; one video segment corresponds to one video segment, and the number of video frames in one video segment is less than or equal to N;

[0282] Obtain K first sub-networks in the second network model, input each video slice into a different first sub-network, and have the K first sub-networks output a video feature vector for each video slice;

[0283] Based on the text feature vector, the video feature vector of each video segment and the classifier of the target network model, the style of the target video is identified to obtain a target style label of the target video; the target style label is used to push associated videos related to the target video to the first user.

[0284] It should be understood that the computer device 1000 described in the embodiment of the present application can execute the above Figure 3 or Figure 7 The description of the video processing method in the corresponding embodiment can also be performed as described above. Fig.11 The description of the video processing device 1 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here either.

[0285] In addition, it should be pointed out that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the video processing device 1 mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the above-mentioned Figure 3 or Figure 7 The description of the video processing method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0286] In addition, it should be noted that: the embodiment of the present application also provides a computer program product or a computer program, which may include computer instructions, and the computer instructions may be stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor may execute the computer instructions, so that the computer device executes the above Figure 3 or Figure 7 The description of the video processing method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the computer program product or computer program embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0287] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0288] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A video processing method, characterized in that: include: Acquire a target video and a target network model corresponding to the target video; the target network model includes a first network model and a second network model; Acquire target text information of the target video, input the target text information into the first network model, and have the first network model output a text feature vector corresponding to the target text information; The target video is evenly divided into K video segments, and the K video segments are randomly sampled based on a time sampling strategy to obtain video slices of each video segment; K is a positive integer, and the number of video frames in each video segment is N; N is a positive integer; one video segment corresponds to one video slice, and the number of video frames in one video slice is less than or equal to N; Obtain K first subnetworks in the second network model, input each video segment into a different first subnetwork, and have the K first subnetworks output the video feature vector of each video segment respectively; wherein different first subnetworks have different weight parameters, the first subnetwork includes a neural network and a multilayer perceptron, the multilayer perceptron includes an input layer, a hidden layer and an output layer; the neural network is used to extract the initial video features of the input video segment; the input layer is used to use the initial video features as the input features of the hidden layer, and the hidden layer is used to extract the hidden features of the input features; the output layer is used to output the video feature vector of the input video segment according to the hidden features; Projecting the text feature vector and the video feature vector of each video segment to the target vector space; performing feature fusion on the text feature vector and the video feature vector of each video segment in the target vector space to obtain fused vector features; inputting the fused vector features into the classifier of the target network model, and having the classifier of the target network model perform style recognition on the target video to obtain the matching degree between the fused vector features and the sample attribute features in the classifier; the matching degree is used to describe the probability that the target video belongs to the sample label corresponding to the sample attribute feature; from the matching degree, the sample label corresponding to the sample attribute feature with the maximum matching probability is used as the target style label of the target video; the target style label is used to push associated videos related to the target video to the first user; The method further comprises: Acquire the target video uploaded by the second user through the dispatch center service component, and configure a video fingerprint feature for the target video; Obtaining a historical fingerprint feature of each historical video in a content database, and determining a vector distance between each historical fingerprint feature and the video fingerprint feature; If there is no vector distance greater than the vector threshold associated with the historical video in the vector distances, it is determined that the video fingerprint feature meets the duplicate removal processing condition, and the target video uploaded by the second user is stored in the content database; If there is a vector distance less than the vector threshold in the vector distances, it is determined that the video fingerprint feature does not meet the duplicate removal processing condition, and it is determined that there is a historical video matching the target video in the content database, and the target video uploaded by the second user is deleted; Reading the target video from the content database, and reading first auxiliary information, second auxiliary information and third auxiliary information of the target video from the content database; the first auxiliary information is information entered by the second user when uploading the target video; the second auxiliary information is obtained after the target video is classified by a video classification model; the third auxiliary information is obtained after transcoding the target video; Using the first auxiliary information, the second auxiliary information, and the third auxiliary information as meta information of the target video, and performing a first video review on the target video based on the meta information to obtain an initial video review result; If the initial video review result indicates that the target video is legal, a second video review is performed on the target video to obtain a target video review result, and the target video review result is used as the video review result of the target video; When the video review result indicates that the target video is a distributable video, starting a content distribution service, calling the content distribution service to obtain an aggregation cluster mapped by the target style label from a video recommendation database, and adding the distributable video to the aggregation cluster; A user portrait of the first user is obtained, and based on the target style tag carried in the user portrait, at least one associated video to be pushed to the first user is obtained from the aggregation cluster, and the at least one associated video is pushed to the first user.

2. The method according to claim 1, characterized in that The obtaining of a target video and a target network model corresponding to the target video includes: Calling a video style recognition service through a dispatch center service component, and obtaining a target video uploaded by a second user through a video upload interface through the video style recognition service; A target network model for performing style recognition on the target video is obtained through the video style recognition service.

3. The method according to claim 1, characterized in that The step of obtaining target text information of the target video, inputting the target text information into the first network model, and having the first network model output a text feature vector corresponding to the target text information includes: Acquire video subject information, video classification information and video tag information of the target video, and use the video subject information, the video classification information and the video tag information as target text information of the target video; Performing word segmentation processing on the target text information to obtain text segmentation of the target text information, and performing vector conversion on the text segmentation to obtain a text segmentation vector corresponding to the text segmentation; Inputting the text word segmentation vector into the first network model, and extracting vector features of the text word segmentation vector by the first network model; The text position of the text segmentation in the target text information is determined, and based on the text position and the extracted vector features, a text feature vector corresponding to the target text information is determined.

4. The method according to claim 1, characterized in that The method further comprises: Obtaining a sample video and a sample style label of the sample video for training an initial network model; the initial network model includes a first sample model and a second sample model; Acquire sample text information of the sample video, input the sample text information into the first sample model, and have the first sample model output a sample text vector corresponding to the sample text information; The sample video is evenly divided into K sample segments, and the K sample segments are randomly sampled based on a time sampling strategy to obtain sample slices of each sample segment; K is a positive integer, and the number of video frames in each sample segment is M; M is a positive integer; one sample segment corresponds to one sample slice, and the number of video frames in one sample slice is less than or equal to M; Obtain K second sub-networks in the second sample model, input each sample slice into a different second sub-network, and have the K second sub-networks output sample video vectors of each sample slice; Based on the sample style label, the sample text vector, the sample video vector of each sample segment and the classifier of the initial network model, the initial network model is iteratively trained, and the initial network model after iterative training is used as the target network model.

5. The method according to claim 4, wherein the initial network model is iteratively trained based on the sample style label, the sample text vector, the sample video vector of each sample slice, and the classifier of the initial network model, and the initial network model after iterative training is used as the target network model, comprising: Based on the sample painting style label, the sample text vector, the sample video vector of each sample slice, and the classifier of the initial network model, iteratively train the initial network model to obtain a model training result; When the model training result indicates that the model loss function of the initial network model after iterative training does not meet the model convergence condition, using the initial network model after iterative training as a transition network model; Based on the model loss function that does not meet the model convergence condition, the model parameters of the transition network model are adjusted until the adjusted transition network model meets the model convergence condition, and the transition network model that meets the model convergence condition is used as the target network model.

6. The method according to claim 1, characterized in that The target video is a video played by the first user in the video client; The obtaining the user portrait of the first user includes: Acquire a behavior log table of the first user, and extract historical video information associated with the first user from the behavior log table; the historical video information is determined based on historical user behavior of the first user in the video client; A historical painting style tag of the historical video information is obtained, and a user portrait for characterizing the first user is determined based on the historical painting style tag and the target painting style tag.

7. A video processing device, characterized in that: include: A target acquisition module, used to acquire a target video and a target network model corresponding to the target video; the target network model includes a first network model and a second network model; A first output module, used for acquiring target text information of the target video, inputting the target text information into the first network model, and having the first network model output a text feature vector corresponding to the target text information; A first determination module is used to evenly divide the target video into K video segments, and randomly sample frames for the K video segments based on a time sampling strategy to obtain video slices of each video segment; K is a positive integer, and the number of video frames in each video segment is N; N is a positive integer; one video segment corresponds to one video slice, and the number of video frames in one video slice is less than or equal to N; The second output module is used to obtain K first sub-networks in the second network model, input each video segment into different first sub-networks respectively, and the K first sub-networks respectively output the video feature vector of each video segment; wherein different first sub-networks have different weight parameters, the first sub-network includes a neural network and a multi-layer perceptron, and the multi-layer perceptron includes an input layer, a hidden layer and an output layer; the neural network is used to extract the initial video features of the input video segment; the input layer is used to use the initial video features as the input features of the hidden layer, and the hidden layer is used to extract the hidden features of the input features; the output layer is used to output the video feature vector of the input video segment according to the hidden features; A style recognition module is used to project the text feature vector and the video feature vector of each video segment to a target vector space; perform feature fusion on the text feature vector and the video feature vector of each video segment in the target vector space to obtain a fused vector feature; input the fused vector feature into the classifier of the target network model, and perform style recognition on the target video by the classifier of the target network model to obtain a matching degree between the fused vector feature and the sample attribute feature in the classifier; the matching degree is used to describe the probability that the target video belongs to the sample label corresponding to the sample attribute feature; the sample label corresponding to the sample attribute feature with the maximum matching probability in the matching degree is used as the target style label of the target video; the target style label is used to push associated videos related to the target video to the first user; A duplicate removal processing module is used to obtain a target video uploaded by a second user through a dispatch center service component, and configure a video fingerprint feature for the target video; obtain a historical fingerprint feature of each historical video in a content database, and determine a vector distance between each historical fingerprint feature and the video fingerprint feature; if there is no vector distance greater than a vector threshold associated with the historical video in the vector distance, it is determined that the video fingerprint feature meets the duplicate removal processing condition, and the target video uploaded by the second user is stored in the content database; if there is a vector distance less than the vector threshold in the vector distance, it is determined that the video fingerprint feature does not meet the duplicate removal processing condition, and it is determined that there is a historical video matching the target video in the content database, and the target video uploaded by the second user is deleted; A video review module, used for reading the target video from the content database, and reading the first auxiliary information, the second auxiliary information and the third auxiliary information of the target video from the content database; the first auxiliary information is the information entered by the second user when uploading the target video; the second auxiliary information is obtained after the target video is classified by a video classification model; the third auxiliary information is obtained after transcoding the target video; the first auxiliary information, the second auxiliary information and the third auxiliary information are used as meta-information of the target video, and a first video review is performed on the target video based on the meta-information to obtain an initial video review result; if the initial video review result indicates that the target video is legal, a second video review is performed on the target video to obtain a target video review result, and the target video review result is used as the video review result of the target video; A distribution activation module, configured to start a content distribution service when the video review result indicates that the target video is a distributable video, call the content distribution service to obtain the aggregation cluster mapped by the target style label from a video recommendation database, and add the distributable video to the aggregation cluster; The video push module is used to obtain the user portrait of the first user, obtain at least one associated video to be pushed to the first user from the aggregation cluster based on the target style tag carried in the user portrait, and push the at least one associated video to the first user.

8. A computer device, characterized in that: include: Processor and memory; The processor is connected to a memory, wherein the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 6.

Images