Video recognition method, apparatus, device, and storage medium

Abstract

The application discloses a video recognition method and device, equipment and a storage medium, and belongs to the computer technical field. Through the technical scheme provided by the embodiment of the application, a plurality of video frame features can be numbered, and the plurality of feature numbers obtained can represent the corresponding video frame features. In the numbering process, the similarity between the video frame features is combined, that is, if the numbers of two video frame features are the same, it can also represent that the similarity between the two video frame features is high, and further, it can represent that the similarity between the two video frames corresponding to the two video frame features is high. The numbering of the video frame features is used to determine the first repetition information of the video, which quickly reflects the repetition of the video segment in the video. When the first repetition information meets the target condition, the video can be determined as a target type video. The whole process is realized by a computer device, thereby improving the efficiency of video type determination.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a video identification method and device, equipment and a storage medium. BACKGROUND

[0002] With the development of network technology, more and more users will relax by watching videos. Some video playing platforms provide a video sharing function, and users can upload their own videos to the video playing platform to share the videos with other users. Before the videos are presented to other users, the videos often need to be audited. In the videos uploaded by users, there are some loop videos with poor video quality, which need to be identified and deleted in the video auditing process. The loop video refers to a video with a large number of repeated video clips. For example, the original video is only 10 seconds, and the final output video is 60 seconds. The 60-second video is obtained by repeating the 10-second video 6 times.

[0003] In the related art, the uploaded videos of users are often manually audited by auditors to identify the loop videos. However, in the case of a large number of videos, the efficiency of manual auditing is low. SUMMARY

[0004] The embodiments of the present application provide a video identification method, device, equipment and storage medium, which can improve the video identification effect. The technical solution is as follows:

[0005] In one aspect, a video identification method is provided, and the method comprises:

[0006] Obtaining a plurality of video frame features of a video, wherein the video frame features comprise at least one of picture features and audio features of corresponding video frames;

[0007] Numbering the plurality of video frame features to obtain a plurality of feature numbers, wherein the feature numbers of any two video frame features with a similarity meeting a target similarity condition are the same;

[0008] Determining first repetition degree information of the video based on the plurality of feature numbers, wherein the first repetition degree information is used to indicate at least one of picture repetition and audio repetition of video frames in a plurality of video clips of the video;

[0009] In response to the first repetition degree information of the video meeting a first target condition, determining the video as a target type of video.

[0010] In some embodiments, the step of determining the video as the target type of video in response to the first repetition degree information of the video meeting the first target condition comprises any one of the following:

[0011] determine the video as the target type of video in response to a picture repetition ratio of any video segment in the video being greater than or equal to a picture repetition ratio threshold, the picture repetition ratio threshold being associated with a type of the any video segment;

[0012] determine the video as the target type of video in response to an audio repetition ratio of any video segment in the video being greater than or equal to an audio repetition ratio threshold, the audio repetition ratio threshold being associated with a type of the any video segment.

[0013] In some embodiments, before the video is determined as the target type of video in response to the first repetition information of the video meeting the first target condition, the method further comprises:

[0014] input the plurality of video segments into a video classification model, classify the plurality of video segments through the video classification model, and output types of the plurality of video segments.

[0015] In some embodiments, after the video is determined as the target type of video in response to the first repetition information of the video meeting the first target condition, the method further comprises any one of the following:

[0016] divide a total duration of video segments in the video whose picture repetition ratio is greater than or equal to the picture repetition ratio threshold by a duration of the video to obtain a picture repetition ratio of the video;

[0017] divide a total duration of video segments in the video whose audio repetition ratio is greater than or equal to the audio repetition ratio threshold by a duration of the video to obtain an audio repetition ratio of the video.

[0018] In one aspect, a video recognition apparatus is provided, and the apparatus comprises:

[0019] a video frame feature acquisition unit configured to acquire a plurality of video frame features of a video, the video frame features comprising at least one of picture features and audio features of corresponding video frames;

[0020] a numbering unit configured to number the plurality of video frame features to obtain a plurality of feature numbers, the feature numbers of video frame features of any two of the plurality of video frame features meeting a target similarity condition being the same;

[0021] a first repetition information determination unit configured to determine first repetition information of the video based on the plurality of feature numbers, the first repetition information being used to indicate at least one of picture repetition and audio repetition of video frames in a plurality of video segments of the video;

[0022] The video type determination unit is configured to determine the video as a target type of video in response to the first repetition degree information of the video meeting a first target condition.

[0023] In some embodiments, the video frame feature acquisition unit is configured to perform at least one of the following:

[0024] The video classification model is configured to perform feature extraction on the pictures of the plurality of video frames through a feature extraction layer of the video classification model, and obtain a plurality of picture features.

[0025] The audio feature extraction model is configured to perform feature extraction on the audios of the plurality of video frames through a feature extraction layer of the audio feature extraction model, and obtain a plurality of audio features.

[0026] In some embodiments, the numbering unit is configured to determine the arrangement order of each video frame corresponding to each video frame feature in the video as an initial number corresponding to each video frame feature.

[0027] The numbering unit is configured to obtain the plurality of feature numbers based on the similarity between each two video frame features and the plurality of initial numbers.

[0028] In some embodiments, the numbering unit is configured to adjust part of the initial numbers in the plurality of initial numbers based on the similarity between each two video frame features, to obtain the plurality of feature numbers, wherein the part of the initial numbers refers to a number corresponding to a target video frame feature, and the target video frame feature is a video frame feature with a later number order among any two video frame features.

[0029] In some embodiments, the numbering unit is configured to, in response to the similarity between any two video frame features meeting the target similarity condition, adjust an initial number corresponding to a target video frame feature among the any two video frame features to an initial number corresponding to a video frame feature with an earlier number order.

[0030] In some embodiments, the device further comprises:

[0031] The denoising unit is configured to perform denoising processing on the adjusted plurality of initial numbers.

[0032] In some embodiments, the denoising unit is configured to determine a plurality of noise point numbers in the adjusted plurality of initial numbers, and adjust each noise point number based on at least two initial numbers adjacent to the noise point number in the adjusted plurality of initial numbers in terms of number order.

[0033] In some embodiments, the de-noising unit is configured to determine the first number as one of the noise point numbers in response to a difference between the first number and the second number being greater than a first difference threshold and a difference between the first number and the third number being less than or equal to a second difference threshold.

[0034] The first number is any one of the adjusted initial numbers; the second number is an initial number in the adjusted initial numbers and has a number order before the first number; and the third number is an initial number in the adjusted initial numbers and has a number order after the first number.

[0035] In some embodiments, the de-noising unit is configured to adjust each of the noise point numbers to a target number, the target number being an average of the second number and the third number.

[0036] The second number is an initial number in the adjusted initial numbers and has a number order before the noise point number; and the third number is an initial number in the adjusted initial numbers and has a number order after the noise point number.

[0037] In some embodiments, the first repetition degree information determination unit is configured to, for a first video segment and a second video segment in the video, obtain a number of same numbers in a plurality of first feature numbers and a plurality of second feature numbers, the first feature numbers being feature numbers corresponding to video frames in the first video segment, and the second feature numbers being feature numbers corresponding to video frames in the second video segment.

[0038] Based on the number of same numbers, determine second repetition degree information of the first video segment and the second video segment, the second repetition degree information being used to indicate at least one of a picture repetition situation and an audio repetition situation of video frames in the first video segment and video frames in the second video segment.

[0039] Based on a plurality of the second repetition degree information, determine first repetition degree information of the video.

[0040] In some embodiments, the first repetition degree information determination unit is configured to divide the number of same numbers by a target number to obtain a picture repetition ratio or an audio repetition ratio of video frames in the first video segment and video frames in the second video segment, the target number being a number of feature numbers corresponding to the first video segment or the second video segment, the picture repetition ratio being used to represent a picture repetition situation of video frames in the first video segment and video frames in the second video segment, and the audio repetition ratio being used to represent an audio repetition situation of video frames in the first video segment and video frames in the second video segment.

[0041] In some embodiments, the apparatus further comprises:

[0042] a video segment dividing unit, configured to determine a plurality of segment division numbers from the plurality of feature numbers, each of the segment division numbers being smaller than a previous feature number, and a number difference between each of the segment division numbers and the previous feature number being greater than a target difference value, the target difference value being proportional to an average difference value between a plurality of feature numbers in a number order before the segment division number;

[0043] divide the video into the plurality of video segments based on the plurality of segment division numbers.

[0044] In some embodiments, the video type determining unit is configured to perform any one of the following:

[0045] determine the video as the target type of video in response to a picture repetition ratio of any video segment in the video being greater than or equal to a picture repetition ratio threshold value, the picture repetition ratio threshold value being associated with a type of the any video segment;

[0046] determine the video as the target type of video in response to an audio repetition ratio of any video segment in the video being greater than or equal to an audio repetition ratio threshold value, the audio repetition ratio threshold value being associated with a type of the any video segment.

[0047] In some embodiments, the apparatus further comprises:

[0048] a classifying unit, configured to input the plurality of video segments into a video classification model, classify the plurality of video segments by the video classification model, and output types of the plurality of video segments.

[0049] In some embodiments, the apparatus further comprises any one of the following units:

[0050] a picture repetition ratio determining unit, configured to divide a total time length of video segments in the video whose picture repetition ratios are greater than or equal to the picture repetition ratio threshold value by a time length of the video, to obtain a picture repetition ratio of the video;

[0051] an audio repetition ratio determining unit, configured to divide a total time length of video segments in the video whose audio repetition ratios are greater than or equal to the audio repetition ratio threshold value by a time length of the video, to obtain an audio repetition ratio of the video.

[0052] In an aspect, a computer device is provided, which comprises one or more processors and one or more memories, and at least one computer program is stored in the one or more memories, and the computer program is loaded and executed by the one or more processors to implement the video recognition method.

[0053] In an aspect, a computer readable storage medium is provided, which stores at least one computer program, and the computer program is loaded and executed by a processor to implement the video recognition method.

[0054] In an aspect, a computer program product or computer program is provided, which comprises program code stored in a computer readable storage medium, and the program code is read by a processor of a computer device from the computer readable storage medium, and the processor executes the program code to make the computer device execute the video recognition method.

[0055] By the technical solution provided by the embodiments of the present application, the multiple video frame features can be numbered, and the multiple feature numbers obtained can represent the corresponding video frame features. In the numbering process, the similarity between the video frame features is combined, that is, if the numbers of two video frame features are the same, it can represent that the similarity between the two video frame features is high, and further can represent that the similarity between the two video frames corresponding to the two video frame features is high. The numbering of the video frame features is used to determine the first repetition information of the video, which quickly reflects the repetition of the video segment in the video. When the first repetition information meets the target condition, the video can be determined as a video of a target type. The entire process is realized by a computer device, without manual detection, thereby improving the efficiency of video type determination. BRIEF DESCRIPTION OF DRAWINGS

[0056] To more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0057] Figure 1 is a schematic diagram of an implementation environment of a video recognition method provided by an embodiment of the present application;

[0058] Figure 2 is a structural schematic diagram of a convolutional layer provided by an embodiment of the present application;

[0059] Figure 3 is a flowchart of a video recognition method provided by an embodiment of the present application;

[0060] Figure 4 is a flowchart of a video recognition method provided by an embodiment of the present application;

[0061] Figure 5 is a schematic diagram of noise point coding provided by an embodiment of the present application;

[0062] Figure 6 is a schematic diagram of a repeated video segment provided by an embodiment of the present application;

[0063] Figure 7 is a schematic diagram of a repeated video segment provided by an embodiment of the present application;

[0064] Figure 8 is a flowchart of a video recognition method provided by an embodiment of the present application;

[0065] Figure 9 is a flowchart of a video recognition method provided by an embodiment of the present application;

[0066] Figure 10 is a structural schematic diagram of a video recognition device provided by an embodiment of the present application;

[0067] Figure 11 is a structural schematic diagram of a terminal provided by an embodiment of the present application;

[0068] Figure 12 is a structural schematic diagram of a server provided by an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical scheme and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0070] In the present application, the terms "first", "second" and the like are used to distinguish the same items or similar items with basically the same function and action, and it should be understood that there is no logical or time sequence dependency between "first", "second", "nth", and the number and execution order are not limited.

[0071] In the present application, the term "at least one" means one or more, and the term "multiple" means two or more, for example, multiple reference face images means two or more reference face images.

[0072] Artificial Intelligence (AI) is the theory, method, technology and application system of using digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making.

[0073] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0074] Machine Learning (ML) is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is a subject that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge sub-models to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent, and its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0075] Normalization processing: mapping the number series with different value ranges to the interval (0, 1) for easy data processing. In some cases, the normalized value can be directly implemented as a probability.

[0076] Embedded coding: embedded coding represents a corresponding relationship in mathematics, that is, mapping data on X space to Y space through a function F, where the function F is a single function, and the mapping result is structure preserving. Single function means that the mapped data is uniquely corresponding to the pre-mapped data, and structure preserving means that the size relationship of the pre-mapped data is the same as that of the post-mapped data, for example, there are data X1 and X2 before mapping, and Y1 corresponding to X1 and Y2 corresponding to X2 after mapping. If the data X1 > X2 before mapping, then the data Y1 > Y2 after mapping. For words, it is to map words to another space for subsequent machine learning and processing.

[0077] Image recognition: a recognition that does not consider specific instances of an object, but only considers the class of the object (such as people, dogs, cats, birds, etc.) and gives the class to which the object belongs. A typical example is the recognition task in the large general object recognition open source dataset image-net, which recognizes which of the 1000 classes a certain object belongs to.

[0078] The computer device provided by the embodiments of the present application can be implemented as a terminal or a server. The implementation environment composed of the terminal and the server is introduced below.

[0079] Figure 1 is a schematic diagram of an implementation environment of a video recognition method provided by the embodiments of the present application. Referring to Figure 1 , the implementation environment can include a first terminal 110, a second terminal 120, and a server 140. The first terminal 110 is connected to the server 140 through a wireless network or a wired network, and the second terminal 120 is connected to the server 140 through a wireless network or a wired network.

[0080] In some embodiments, the first terminal 110 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a smart television, a smart vehicle device, etc., but is not limited thereto. The first terminal 110 is installed and runs an application program supporting video uploading and video playing.

[0081] In some embodiments, the second terminal 120 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a smart television, a smart vehicle device, etc., but is not limited thereto. The second terminal 120 is installed and runs an application program supporting video auditing.

[0082] In some embodiments, the server 140 is a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The server 140 can execute the video recognition method provided by the embodiments of the present application, recognize the video uploaded by the first terminal 110, and send the recognition result to the second terminal 120.

[0083] In some embodiments, the first terminal 110 refers to one of a plurality of first terminals, and the embodiments of the present application are exemplified by the first terminal 110. The second terminal 120 refers to one of a plurality of second terminals, and the embodiments of the present application are exemplified by the second terminal 120. The first terminal and the second terminal can be the same terminal or different terminals.

[0084] Those skilled in the art can know that the number of the first terminal 110, the second terminal 120 or the server 140 can be more or less. For example, the first terminal is only one, or the first terminal is dozens or hundreds, or more, at this time, the above-mentioned implementation environment also includes other first terminals. The number and type of the first terminal and the second terminal are not limited in the embodiments of the present application.

[0085] In some embodiments, the technical solutions provided by the embodiments of the present application can be applied to a blockchain system, that is, the first terminal 110, the second terminal 120 and the server 140 are all nodes of the blockchain system, and some basic concepts of the blockchain system are as follows.

[0086] The blockchain is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm and other computer technologies. The blockchain is essentially a decentralized database, which is a series of data blocks associated using cryptographic methods. Each data block contains a batch of network transaction information for verifying the validity (anti-fake) of the information and generating the next block. The blockchain can include a blockchain underlying platform, a platform product service layer and an application service layer.

[0087] The blockchain underlying platform can include user management, basic services, smart contracts, and operation monitoring processing modules. Among them, the user management module is responsible for the identity information management of all blockchain participants, including maintaining public and private key generation (account management), key management, and user real identity and blockchain address correspondence maintenance (permission management), etc., and under authorization, supervises and audits the transaction situation of certain real identities, provides risk control rule configuration (risk audit); the basic service module is deployed on all blockchain node devices to verify the validity of business requests, and after consensus, the valid request is recorded to the storage. For a new business request, the basic service first performs interface adaptation analysis and authentication processing (interface adaptation), then encrypts the business information through a consensus algorithm (consensus management), and after encryption, the complete and consistent transmission is transmitted to the shared ledger (network communication), and is recorded and stored; the smart contract module is responsible for contract registration and issuance, contract triggering and contract execution. Developers can define contract logic through a certain programming language, publish it to the blockchain (contract registration), trigger execution according to the logic of the contract terms, complete the contract logic, and also provide contract upgrade and cancellation functions; the operation monitoring module is mainly responsible for the deployment, configuration modification, contract setting, cloud adaptation of the product release process, and the real-time state visualization output of the product running, such as alarm, monitoring network situation, monitoring node device health status, etc.

[0088] The platform product service layer provides basic capabilities and implementation frameworks for typical applications. Developers can add business features based on these basic capabilities to complete the blockchain implementation of business logic. The application service layer provides application services based on the blockchain solution for business participants to use.

[0089] In the embodiments of the present application, the server 140 in the above implementation environment can be used as an execution subject to implement the technical solutions provided by the embodiments of the present application, or the technical solutions provided by the embodiments of the present application can be implemented through interaction between the terminal and the server, such as sending the video uploaded by the first terminal 110 to the second terminal 120 by the server 140 in the above implementation environment, and implementing the technical solutions provided by the embodiments of the present application by the second terminal 120. The present application does not limit this. The following will take the server as an example for description:

[0090] In the embodiments of the present application, the server can extract features of the pictures of the video frames through the video classification model to obtain the picture features of each video frame, and extract features of the audios of the video frames through the audio feature extraction model to obtain the audio features of each video frame. In addition, the server can also classify the video through the video classification model, or classify the video segments in the video.

[0091] To make the technical solutions provided by the embodiments of the present application clearer, the structures of the video classification model and the audio feature extraction model will be described respectively.

[0092] In some embodiments, the video classification model comprises a feature extraction layer and a classification layer, wherein the feature extraction layer comprises a plurality of convolution layers, each convolution layer comprising at least one module, and each module comprising at least one convolution kernel. In some embodiments, each module comprises a plurality of convolution kernels with different sizes. In some embodiments, the feature extraction layer further comprises a pooling layer. The classification layer comprises a pooling layer and a fully connected layer.

[0093] For example, Figure 2 is a structural schematic diagram of a convolution layer 201 of the feature extraction model according to the embodiments of the present application. The convolution layer 201 comprises three modules. In some embodiments, the first module 202 comprises 64 convolution kernels with a size of 1x1, the second module 203 comprises 64 convolution kernels with a size of 3x3, and the third module 204 comprises 256 convolution kernels with a size of 1x1. In some embodiments, the convolution layer 201 further comprises a residual module 205. The residual module 205 is used to perform residual processing on the feature map obtained by convolution and processing, that is, the feature map or video frame input into the convolution layer 201 is added to the feature map obtained after the feature extraction of the convolution layer 201, to obtain a residual processed feature map. The residual processed feature map can retain information to the greatest extent.

[0094] In some embodiments, the video classification model is a ResNet-101 (Residual Network 101) model, and 101 represents the number of layers of the model. The structure of the feature extraction layer in the ResNet-101 is shown in Table 1. The output size in the table represents the size of the feature map output after processing by the corresponding convolution layer. N x N in the composition represents the size of the convolution kernel, and the number after N x N represents the number of convolution kernels.

[0095] Table 1

[0096]

[0097] In some embodiments, the structure of the classification layer in the ResNet-101 is shown in Table 2. K in Table 2 represents the number of categories, and K is a positive integer.

[0098] Table 2

[0099] Layer name Output size Layer type Pooling layer 1×2048 Max pooling layer Fully connected layer 1×K Fully connected layer

[0100] It should be noted that the structure of the audio feature extraction model provided by the embodiments of the present application belongs to the same inventive concept as the structure of the feature extraction layer of the video classification model described above, and will not be described here.

[0101] After introducing the structures of the video classification model and the audio feature extraction model provided by the embodiments of the present application, the training method of the video classification model provided by the embodiments of the present application is described below. The training method of the video classification model includes a data preparation process and a model training process.

[0102] In the data preparation process, the server can capture video pictures from videos of different content types, add video type labels to the captured video pictures based on the video content types corresponding to the captured video pictures, and use the video pictures with the added video labels as sample images of the video classification model. The video content types can include but are not limited to basketball goals, football goals, car accidents, pure text, close-up of faces, life scenes, and others (including non-goal moment commentator pictures). The video classification model trained based on such samples can determine the corresponding video content type based on the input video frame.

[0103] In the model training process, the server inputs the sample images into the video classification model to be trained, and the video classification model extracts features of the sample images through the feature extraction layer, that is, the sample images are convoluted through the multiple convolution layers of the feature extraction layer to obtain picture features of the sample images. The server inputs the picture features of the sample images into the classification layer through the video classification model, and performs pooling processing and full connection processing on the picture features through the classification layer to obtain a predicted label corresponding to the sample image. The predicted label is used to indicate the video content type corresponding to the sample image. The server adjusts the model parameters of the video classification model according to the difference between the predicted label and the video type label of the sample image. In some embodiments, the server uses the gradient descent method based on SGD (Stochastic Gradient Descent) to solve the convolution kernel parameters w and the bias parameters b of the convolution layer of the video classification model. In each iteration process, the server calculates the prediction error (the error between the predicted label and the actual video type label) and propagates it back to the video classification model to determine the gradient and update the parameters of the video classification model. For example, all parameters of the video classification model are set to a learning state, and the video classification model performs forward calculation on an input sample image to obtain a prediction result (predicted label). The prediction result is compared with the actual video type label to calculate the loss value of the model, and the loss value is back-propagated to the video classification model to update the network weight parameters through the stochastic gradient descent method, thereby realizing one weight optimization.

[0104] It should be noted that in the above description, the training of the feature extraction layer and the classification layer of the video classification model is taken as an example. In other possible embodiments, a pre-trained feature extraction layer can be used for feature extraction without retraining the feature extraction layer. For example, the feature extraction layer of ResNet101 pre-trained on the Image-Net dataset can be used. If a pre-trained feature extraction layer is directly used, the parameters of the feature extraction layer can be fixed during the training of the video classification model, and only the parameters of the classification layer can be adjusted.

[0105] In addition, in addition to the video classification model with the above structure, other structures of video classification models can also be used to classify video clips and videos, and the embodiments of the present application do not limit this.

[0106] In addition, in the above description, the training of the feature extraction layer and the classification layer of the video classification model is taken as an example. In other possible embodiments, a pre-trained feature extraction layer can be used for feature extraction without retraining the feature extraction layer. For example, the feature extraction layer of ResNet101 pre-trained on the Image-Net dataset can be used. If a pre-trained feature extraction layer is directly used, the parameters of the feature extraction layer can be fixed during the training of the video classification model, and only the parameters of the classification layer can be adjusted.

[0107] It should be noted that in the following description of the technical solutions provided by the present application, the server is taken as an example of the execution subject. In other possible embodiments, the terminal can also be used as the execution subject to execute the technical solutions provided by the present application, and the embodiments of the present application do not limit the type of execution subject.

[0108] Figure 3 is a flowchart of a video recognition method provided by an embodiment of the present application, referring to Figure 3 , the method comprises:

[0109] 301, the server obtains a plurality of video frame features of a video, the video frame features comprising at least one of picture features and audio features corresponding to the video frame.

[0110] In some embodiments, the picture features are picture Embedding, and the audio features are audio Embedding.

[0111] 302, the server numbers the plurality of video frame features to obtain a plurality of feature numbers, and the feature numbers of any two video frame features with a similarity meeting a target similarity condition are the same.

[0112] In some embodiments, each feature number can identify a video frame feature, and the server can quickly determine the corresponding video frame feature through the feature number.

[0113] 303、The server determines first repetition degree information of the video based on the plurality of feature numbers, the first repetition degree information being used to indicate at least one of a picture repetition case and an audio repetition case of video frames in a plurality of video clips of the video.

[0114] In some embodiments, a picture repetition ratio can be used to represent the picture repetition case, and an audio repetition ratio can be used to represent the audio repetition case.

[0115] 304、In response to the first repetition degree information of the video meeting a first target condition, the server determines the video as a target type of video.

[0116] In some embodiments, the target type of video is a loop video.

[0117] Through the technical solutions provided in the embodiments of the present application, a plurality of video frame features can be numbered, and the plurality of feature numbers can represent the corresponding video frame features. In the numbering process, the similarity between the video frame features is combined, that is, if the numbers of two video frame features are the same, it can be indicated that the similarity between the two video frame features is high, and further, the similarity between the two video frames corresponding to the two video frame features is high. The numbering of the video frame features is used to determine the first repetition degree information of the video, which quickly reflects the repetition case of the video clips in the video. When the first repetition information meets the target condition, the video can be determined as a target type of video. The entire process is implemented by a computer device, without the need for manual detection, thereby improving the efficiency of video type determination.

[0118] Figure 4 is a flowchart of a video recognition method provided by an embodiment of the present application, referring to Figure 4 , the method comprises:

[0119] 401、The server obtains a video.

[0120] In some embodiments, the video is a video uploaded by a user or a video stored in a database maintained by the server, and the embodiments of the present application do not limit this.

[0121] In some embodiments, if the video is a video uploaded by a user, the user can send the video to the server through a terminal, and the server obtains the video.

[0122] For example, a video recording application runs on the terminal, allowing the user to record videos. After recording, the user can upload the video to a server via the terminal, and the server retrieves the video. In some embodiments, the video recording application also has video editing functions, allowing the user to edit the recorded video using the application and send the edited video to the server for retrieval. In some embodiments, the terminal used by the user is the first terminal 110 in the above-described implementation environment.

[0123] In some embodiments, if the video is a video stored in a database maintained by the server, then the server can retrieve the video from the database.

[0124] 402. The server obtains multiple video frame features of the video, and the video frame features include at least one of the picture features and audio features of the corresponding video frame.

[0125] In some embodiments, the server inputs multiple video frames into a video classification model. The feature extraction layer of the video classification model extracts features from the multiple video frames, resulting in multiple image features, which are also the video frame features. In some embodiments, the process of extracting features from multiple video frames is also called embedding encoding, and the resulting image features are called image embeddings. Each image embedding is a vector. That is, an image embedding can represent the image features of the corresponding video frame.

[0126] For example, the server inputs multiple video frames into a video classification model. The feature extraction layer of the video classification model performs convolutional processing on each of the multiple video frames to obtain multiple scene features. For instance, after inputting multiple video frames into the video classification model, the server uses methods such as... Figure 2 The multiple convolutional kernels in the convolutional layer 201 shown perform multiple convolutional processes on each video frame to obtain the image features of each video frame.

[0127] In some embodiments, the server inputs the audio of the plurality of video frames of the video into an audio feature extraction model, extracts features of the audio of the plurality of video frames through a feature extraction layer of the audio feature extraction model, and obtains a plurality of audio features, which are also the plurality of video frame features. In some embodiments, the process of extracting features of the audio of the plurality of video frames is also referred to as the process of performing embedding encoding, and the obtained audio features are also referred to as audio Embedding, each of which is a vector. That is, the audio feature of the corresponding video frame can be represented by an audio Embedding.

[0128] For example, the server obtains audio information from the video, splits the audio information according to the playback time of each video frame to obtain the audio of each video frame. The server performs time-frequency conversion on the audio of each video frame to obtain the frequency domain information corresponding to the audio of each video frame. The server inputs the plurality of frequency domain information into an audio feature extraction model, extracts features of each frequency domain information through a feature extraction layer of the audio feature extraction model, and obtains the audio features of each frequency domain information. The method of performing time-frequency conversion on the audio by the server includes Fourier transform and wavelet transform, etc., which are not limited in the embodiments of the present application. The process of extracting features of the frequency domain information by the audio feature extraction model is also the process of performing convolution processing on the frequency domain information.

[0129] In some embodiments, the server inputs the plurality of video frames into a video classification model, extracts features of the pictures of the plurality of video frames through a feature extraction layer of the video classification model, and obtains a plurality of picture features. The server inputs the audio of the plurality of video frames of the video into an audio feature extraction model, extracts features of the audio of the plurality of video frames through a feature extraction layer of the audio feature extraction model, and obtains a plurality of audio features. The plurality of picture features and the plurality of audio features are also the plurality of video frame features.

[0130] 403、The server numbers the plurality of video frame features to obtain a plurality of feature numbers, and the feature numbers of any two video frame features with a similarity meeting a target similarity condition in the plurality of video frame features are the same.

[0131] In some embodiments, the server determines the arrangement order of the video frame corresponding to each video frame feature in the video as the initial number of each video frame feature. The server obtains the plurality of feature numbers based on the similarity between each two video frame features and the plurality of initial numbers.

[0132] In this implementation, the server can directly number each video frame feature according to the arrangement order of the video frame in the video, and subsequently use the number to quickly determine the corresponding video frame feature, which is efficient.

[0133] To make the above-mentioned embodiments clearer, the following will be divided into two parts to explain the above-mentioned embodiments.

[0134] The first part explains the method of determining the initial number corresponding to each video frame feature by the server according to the arrangement order of the video frame corresponding to each video frame feature in the video.

[0135] In some embodiments, if the video includes N video frames, the server sets the initial number corresponding to the video frame feature of the N video frames as 1, 2, 3…N according to the arrangement order of the N video frames in the video, wherein N is a positive integer. If the arrangement order of a video frame in the video is 5, then the initial number of the video frame is also 5. In some embodiments, the server stores multiple initial numbers in an initial number sequence, and the arrangement order of the initial numbers in the initial number sequence is the arrangement order of the video frames corresponding to the initial numbers in the video. The server can directly obtain the corresponding initial number from the initial number sequence later.

[0136] The second part explains the method of obtaining multiple feature numbers by the server based on the similarity between each two video frame features and multiple initial numbers.

[0137] In some embodiments, the server adjusts part of the numbers in the multiple initial numbers based on the similarity between each two video frame features to obtain multiple feature numbers, wherein the part of the numbers refers to the number corresponding to the target video frame feature, and the target video frame feature is the video frame feature with the later number order in any two video frame features.

[0138] For example, the server obtains the similarity between each two video frame features. In response to the similarity between any two video frame features meeting a target similarity condition, the server adjusts the initial number corresponding to the video frame feature in any two video frame features to the initial number corresponding to the video frame feature with the earlier number order. The server determines the adjusted multiple initial numbers as multiple feature numbers.

[0139] In some embodiments, the server can use Euclidean distance or cosine similarity to represent the similarity between two video frame features.

[0140] For example, if the server uses the Euclidean distance to represent the similarity between two video frame features, and uses the Embedding vector to represent the video frame feature, the server can obtain the Euclidean distance between each two video frame features, which is the straight-line distance, and the Euclidean distance can represent the similarity between two video frame features. The smaller the Euclidean distance, the smaller the straight-line distance between the two video frame features, and the closer the two video frame features; the larger the Euclidean distance, the larger the straight-line distance between the two video frame features, and the greater the difference between the two video frame features. In some embodiments, the server stores the Euclidean distance between each two video frame features in the plurality of video frames in the numbered distance table, and the form of the numbered distance table is shown in Table 3.

[0141] Table 3

[0142] Video frame feature 1 Video frame feature 2 Video frame feature 3 Video frame feature 1 0 1 2 Video frame feature 2 2 0 3 Video frame feature 3 1 2 0

[0143] The values in the numbered distance table represent the Euclidean distance between two video frame features. For example, the Euclidean distance between video frame feature 1 and video frame feature 2 is 1, and the Euclidean distance between video frame feature 1 and video frame feature 3 is 2. By setting the numbered distance table, the server can quickly obtain the Euclidean distance between any two video frame features. It should be noted that the above is an example of taking the Euclidean distance as an integer, and in other possible embodiments, the server can also use a floating-point number, such as 0.6, to represent the Euclidean distance, and the embodiments of the present application do not limit this.

[0144] In response to the Euclidean distance between any two video frame features being less than the distance threshold, that is, the similarity between the two video frame features meets the target similarity condition, the server adjusts the initial number corresponding to the target video frame feature to the initial number corresponding to the video frame feature with the number sequence in front. For example, if one video frame feature is (1, 2) and the initial number corresponding to the video frame feature (1, 2) is 2, and another video frame feature is (1, 3) and the initial number corresponding to the video frame feature (1, 3) is 8. The server obtains the Euclidean distance d between the two video frame features = (1-1) 2 +(2-3) 2 = 1. If the distance threshold is 2, the server can adjust the initial number 8 corresponding to the video frame feature (1, 3) to the initial number 2 corresponding to the video frame feature (1, 2). That is, after adjustment, the initial number 2 corresponding to the video frame feature (1, 2) remains unchanged, and the initial number corresponding to the video frame feature (1, 3) changes to 2. It should be noted that the above is an example of taking the video frame feature size as 1x2, and in other possible embodiments, the size of the video feature is 1xQ, where Q is a positive integer, such as 128, 256, 512, 1024 or 2048, etc. The embodiments of the present application do not limit this.

[0145] For example, if the server uses cosine similarity to represent the similarity between two video frame features, and uses Embedding vectors to represent the video frame features, the server can obtain the cosine similarity between each two video frame features. The smaller the cosine similarity, the smaller the similarity between the two video frame features, and the closer the two video frame features. The greater the cosine similarity, the greater the similarity between the two video frame features, and the greater the difference between the two video frame features. For any two video frame features, the server obtains the cosine similarity between the two video frame features. In response to the cosine similarity between the two video frame features being greater than or equal to a similarity threshold, that is, the similarity between the two video frame features meets the target similarity condition, the server adjusts the initial number corresponding to the target video feature to the initial number corresponding to the video frame feature with the previous numbering order.

[0146] The content of the second part will be described below by an example.

[0147] In some embodiments, the server sets a number update table L to record the adjustment of the plurality of initial numbers. The length of the number update table L is the same as the number of the plurality of initial numbers, and each number in the number update table L corresponds to the adjustment of the first initial number. In some embodiments, the server can initialize all the numbers in the number update table L to 0. After the server adjusts an initial number, the server can adjust the number corresponding to the initial number in the number update table L to 1. For the initial number with the serial number i, if the similarity between the video frame feature corresponding to the initial number with the serial number i and the video frame feature corresponding to the initial number with the serial number j meets the target similarity condition, and the initial number with the serial number j is the initial number with the previous numbering order, the server can adjust the initial number with the serial number i to the initial number with the serial number j. i and j are both positive integers.

[0148] In some embodiments, after the server adjusts the plurality of initial numbers in the manner of the second part, the server can also perform any one of the following steps.

[0149] Step A, the server performs denoising processing on the adjusted plurality of initial numbers. The denoising processing can also be referred to as smoothing processing. The process of the server performing denoising processing on the adjusted plurality of initial numbers is also the process of deleting the noise point number from the adjusted plurality of initial numbers.

[0150] For example, the server determines a plurality of noise point numbers in the adjusted plurality of initial numbers. The server adjusts each noise point number based on at least two initial numbers adjacent to the noise point number in the adjusted plurality of initial numbers in terms of the numbering order.

[0151] In order to make the technical solutions described in the above examples more clearly, the above examples will be explained in two parts.

[0152] The first part explains the method of determining the plurality of noise point numbers by the server from the plurality of initial numbers after adjustment.

[0153] In some embodiments, in response to the difference between the first number and the second number being greater than the first difference threshold, and the difference between the first number and the third number being less than or equal to the second difference threshold, the server determines the first number as one noise point number. Wherein, the first number is any one of the plurality of initial numbers after adjustment. The second number is the initial number before the first number in the order of the plurality of initial numbers after adjustment. The third number is the initial number after the first number in the order of the plurality of initial numbers after adjustment. In some embodiments, the noise point number is also called a glitch. See Figure 5 , 501, 502 and 503 are three glitches, Figure 5 The horizontal coordinate in the above figure is the serial number of the video frame, which represents the arrangement order of the video frame in the video. The vertical coordinate is the number corresponding to the video frame.

[0154] For example, if the first number is the i-th number in the plurality of initial numbers after adjustment, the second number is the i-1-th number in the plurality of initial numbers after adjustment, and the third number is the i+a-th number in the plurality of initial numbers after adjustment. In some embodiments, 0 i-1 -C i th1, and the difference between the first number and the third number is less than or equal to the second difference threshold, that is, i+a -C i th2, where C i is the first number, C i-1 is the second number, and C i+a is the third number, th1 is the first difference threshold, and th2 is the second difference threshold. The first number is determined as one early point number.

[0155] The method of determining a by the server will be explained below. The process of determining a by the server, that is, the process of determining the third number.

[0156] In some embodiments, the server determines a difference between the first number and each of the initial numbers following the first number in the order, and determines the initial number with the smallest difference as the third number. For example, the server determines a difference between the first number and each of the three initial numbers following the first number in the order. For example, the first number is the i-th number in the initial numbers, and the three initial numbers are the i+1-th, i+2-th and i+3-th numbers in the initial numbers. The server determines three differences between the i-th number and the i+1-th, i+2-th and i+3-th numbers, for example, 1, 2 and 3. The server determines the i+1-th number as the third number.

[0157] The second part describes a method for adjusting each noise number based on the order and at least two initial numbers adjacent to each noise number in the adjusted initial numbers.

[0158] In some embodiments, the server adjusts each noise number to a target number, which is an average of the second number and the third number. The second number is an initial number preceding the noise number in the order in the adjusted initial numbers. The third number is an initial number following the noise number in the order in the adjusted initial numbers.

[0159] For example, if the first number is 21, the second number is 4 and the third number is 6, the server can adjust the first number to the target number, i.e., (4+6) / 2=5.

[0160] Step B: The server performs a continuous processing on the adjusted initial numbers to obtain a plurality of feature numbers.

[0161] For example, the server obtains a plurality of key numbers from the adjusted initial numbers. A key number is an initial number different from other initial numbers in the initial numbers. One key number can correspond to a plurality of initial numbers. The server performs a continuous processing on the key numbers to adjust the key numbers to continuous key numbers. The server adjusts each initial number corresponding to a key number to be the same as the key number.

[0162] After the server adjusts the plurality of initial numbers based on the similarity between the features of each two video frames, there can be a case that the plurality of output numbers after adjustment are discontinuous. For example, if the video includes three video frames, before the server adjusts the plurality of initial numbers based on the similarity between the features of each two video frames, the initial numbers corresponding to the features of the three video frames are 1, 2 and 3 respectively, and after the server adjusts the plurality of initial numbers based on the similarity between the features of each two video frames, the initial numbers corresponding to the features of the three video frames can be 1, 1 and 3 respectively, that is, the initial number of the second video frame feature is adjusted to 1. In this case, the obtained initial number sequence 113 becomes discontinuous. The server obtains two key numbers 1 and 3 from the initial number sequence 113, and performs continuous processing on the two key numbers to obtain processed two key numbers 1 and 2. The server adjusts the initial number corresponding to 3 in the initial number sequence 113 to obtain a feature number 112.

[0163] 404、The server divides the video into a plurality of video segments.

[0164] In some embodiments, the server determines a plurality of segment split numbers from the plurality of feature numbers, each segment split number is less than a previous feature number, and a number difference between each segment split number and the previous feature number is greater than a target difference value, the target difference value is proportional to an average difference value between the plurality of feature numbers before the segment split number in the number order. The server divides the video into a plurality of video segments based on the plurality of segment split numbers.

[0165] In order to make the above-mentioned embodiments more clearly, the following will be divided into two parts to explain the above-mentioned embodiments.

[0166] The first part explains the method for the server to determine the segment split number from the plurality of feature numbers.

[0167] In some embodiments, the server determines the difference between each two adjacent feature numbers in the plurality of feature numbers. In response to any feature number in the plurality of feature numbers being less than a previous feature number, the server determines a target difference value corresponding to the feature number. In response to the difference between the feature number and the previous feature number being greater than the target difference value, the server determines the feature number as a segment split number. In some embodiments, the server stores the plurality of determined segment split numbers through a segment split number list, so as to facilitate subsequent query. In some embodiments, the segment split number is also called a drop point, and the segment split number list is also called a drop list. Subsequently, the server can quickly obtain the segment split number through the segment split list, so as to divide the video into a plurality of video segments.

[0168] The following describes a method for determining the target difference value corresponding to the feature number by the server in the above embodiments.

[0169] In some embodiments, the server determines the difference value between the (i-1)th feature number and the ith feature number, where i is a positive integer. That is, if the feature number is the ith feature number, denoted as T i If T=3, the server determines the difference value between the (i-1)th feature number and the three feature numbers adjacent to the ith feature number, i.e., T i-1 , T i-2 and T i-3 , respectively. That is, the difference value between T i-1 and T i-2 , the difference value between T i-2 and T i-3 , and the difference value between T i-3 and T i-4 , are 1, 2 and 3, respectively. The server determines the average difference value of the three difference values, which is 2. In some embodiments, the server multiplies T by the average difference value to obtain the target difference value. Of course, in the above description, T=3 is taken as an example. In other possible embodiments, T can be other positive integers, such as 5. The embodiments of the present application do not limit this.

[0170] In some embodiments, the server adjusts the difference value between the segment division number and the previous feature number to 0, i.e., the fluctuation value of the number at the significant drop point is not used as a reference, so as to avoid that the average value used as a reference is too high due to a new drop point.

[0171] The second part describes a method for dividing a video into a plurality of video segments by the server based on a plurality of segment division numbers.

[0172] In some embodiments, the server determines a plurality of video frames between the video frame corresponding to the first feature number and the video frame corresponding to the first segment division number as the first video segment, determines a plurality of video frames between the video frame corresponding to the last feature number and the video frame corresponding to the last segment division number as the last video segment, and determines a plurality of video frames between the video frames corresponding to each two feature numbers as a video segment. For example, referring to Figure 5 , 504 is a drop point, and the sequence number of the video frame corresponding to the drop point is 270. Through the drop point 504, the server divides the 500 video frames into two video segments 【0, 270】 and 【270, 500】.

[0173] 405、The server determines first repetition degree information of the video based on the plurality of feature numbers, the first repetition degree information being used to indicate at least one of a picture repetition case and an audio repetition case of the video frames in the plurality of video segments of the video.

[0174] In some embodiments, for a first video segment and a second video segment in the video, the server obtains a number of same numbers in the plurality of first feature numbers and the plurality of second feature numbers, the first feature numbers being feature numbers corresponding to the video frames in the first video segment, the second feature numbers being feature numbers corresponding to the video frames in the second video segment. The server determines second repetition degree information of the first video segment and the second video segment based on the number of same numbers, the second repetition degree information being used to indicate at least one of a picture repetition case and an audio repetition case of the video frames in the first video segment and the video frames in the second video segment. The server determines the first repetition degree information of the video based on the plurality of second repetition degree information.

[0175] In order to make the above-mentioned embodiments more clearly, the following will be divided into three parts to explain the above-mentioned embodiments.

[0176] The first part is to explain the method of the server obtaining the number of same numbers in the plurality of first feature numbers and the plurality of second feature numbers.

[0177] In some embodiments, the server sets a first segment storage list to store the repeated video segments, and sets a second segment storage list to store the non-repeated video segments. In some embodiments, the first segment storage list is also referred to as Snon list, and the second segment storage list is also referred to as Sdup list. Wherein, when storing the video segments, the two segment storage lists only need to store the numbers corresponding to the plurality of video frames of the video segments. For the first video segment, the server can directly store the video segment in the first segment storage list. Starting from the second video segment, the server determines the second repetition degree information between each video segment and the video segments stored in the first segment storage list. In response to the second repetition degree information between any video segment and any video segment stored in the first segment storage list meeting the second target condition, the video segment is stored in the second segment storage list. In response to the second repetition degree information between the video segment and any video segment stored in the first segment storage list not meeting the second target condition, the video segment is stored in the first segment storage list. See Figure 5For example, if there are two video segments 【0, 270】 and 【270, 500】 in the video, the server puts the first video segment 【0, 270】 into the first segment storage list. If the second degree of repetition information between the second video segment and the first video segment meets the second target condition, the server puts the second video segment 【270, 500】 into the second segment storage list. In subsequent use, if it is necessary to obtain non-repeated video segments in the video, the server can directly obtain them from the first segment storage list, which is more efficient. In some embodiments, the repeated video segment is also referred to as a cyclic video segment.

[0178] When the server determines the similarity between the two video segments, it compares the feature numbers of the video frames of the two video segments one by one. The server uses a repeated number record table to record the number of repeated feature numbers of the two video segments. In some embodiments, the repeated number record table is also referred to as an app (apply, already appeared number) list. Taking the feature numbers corresponding to the first video segment as the reference, the server compares the feature numbers corresponding to the second video segment with the feature numbers corresponding to the first video segment one by one, and records the comparison results in the repeated number record table. Starting from the first feature number 1 in the feature numbers corresponding to the first video segment, if the first feature number appears once in the feature numbers corresponding to the second video segment, the server sets the value corresponding to the first feature number in the repeated number record table to 1. If the first feature number appears three times in the feature numbers corresponding to the second video segment, the server sets the value corresponding to the first feature number in the repeated number record table to 3. If the first feature number appears zero times in the feature numbers corresponding to the second video segment, the server sets the value corresponding to the first feature number in the repeated number record table to 0. In this way, the repeated number record table corresponding to the first video segment and the second video segment is obtained. The server determines the number of non-zero numbers in the repeated number record table as the number of the same numbers in the two video segments.

[0179] For example, if the feature numbers corresponding to the first video segment are (1, 2, 3, 4, 5, 6, 7) and the feature numbers corresponding to the second video segment are (1, 3, 5, 7, 9, 10, 11), the server compares the feature numbers (1, 2, 3, 4, 5, 6, 7) corresponding to the first video segment with the feature numbers (1, 3, 5, 7, 9, 10, 11) corresponding to the second video segment one by one, and obtains the repeated number record table (1, 0, 1, 0, 1, 0, 1). The server determines the number of non-zero numbers in the repeated number record table (1, 0, 1, 0, 1, 0, 1) as the number of the same numbers in the first video segment and the second video segment.

[0180] The second part describes a method for determining second repetition information of the first video segment and the second video segment based on the same number of quantities of the server.

[0181] In some embodiments, the server divides the same number of quantities by a target number to obtain a picture repetition ratio or an audio repetition ratio of the video frames in the first video segment and the video frames in the second video segment, the target number being the number of feature numbers corresponding to the first video segment or the second video segment. The picture repetition ratio is used to represent the picture repetition of the video frames in the first video segment and the video frames in the second video segment, and the audio repetition ratio is used to represent the audio repetition of the video frames in the first video segment and the video frames in the second video segment.

[0182] For example, since the number of feature numbers corresponding to the first video segment and the second video segment can be different, for the first video segment, the server divides the same number of quantities by a target number corresponding to the first video segment, i.e., divides the same number of quantities by the number of feature numbers corresponding to the first video segment, to obtain the second repetition information of the first video segment. If the feature of the video frame is a picture feature, the second repetition information is a picture repetition ratio of the first video segment and the second video segment, and the picture repetition ratio can represent the picture repetition of the video frames in the first video segment and the video frames in the second video segment. If the feature of the video frame is an audio feature, the second repetition information is an audio repetition ratio of the first video segment and the second video segment, and the audio repetition ratio can represent the audio repetition of the video frames in the first video segment and the video frames in the second video segment.

[0183] For the second video segment, the server divides the same number of quantities by a target number corresponding to the second video segment, i.e., divides the same number of quantities by the number of feature numbers corresponding to the second video segment, to obtain the second repetition information of the second video segment. If the feature of the video frame is a picture feature, the second repetition information is a picture repetition ratio of the first video segment and the second video segment, and the picture repetition ratio can represent the picture repetition of the video frames in the first video segment and the video frames in the second video segment. If the feature of the video frame is an audio feature, the second repetition information is an audio repetition ratio of the first video segment and the second video segment, and the audio repetition ratio can represent the audio repetition of the video frames in the first video segment and the video frames in the second video segment.

[0184] It should be noted that the server can use any one of the second repetition information of the first video segment or the second repetition information of the second video segment as the second repetition information of the first video segment and the second video segment, and the embodiments of the present application do not limit this.

[0185] In some embodiments, for the second target condition in the first part, if the second repetition information is the frame repetition ratio of the first video segment and the second video segment, then the second target condition is that the frame repetition ratio of the video frames in the first video segment and the video frames in the second video segment is greater than or equal to the frame repetition ratio threshold. If the second repetition information is the audio repetition ratio of the first video segment and the second video segment, then the second target condition is that the audio repetition ratio of the video frames in the first video segment and the video frames in the second video segment is greater than or equal to the audio repetition ratio threshold.

[0186] The third part describes a method for determining the first repetition information of the video based on the plurality of second repetition information by the server.

[0187] In some embodiments, the server fills the plurality of second repetition information into the repetition information table, and the filled repetition information table is the first repetition information.

[0188] 406. The server inputs the plurality of video segments into the video classification model, classifies the plurality of video segments by the video classification model, and outputs the types of the plurality of video segments.

[0189] In some embodiments, the server inputs the plurality of video frame features of the plurality of video segments into the video classification model, and performs pooling processing on the plurality of video frame features of each video segment by the video classification model to obtain a segment feature of each video segment. The server performs full connection processing on each segment feature to obtain a prediction sequence of each video segment, and the numbers in the prediction sequence represent the probabilities of the corresponding video segment belonging to different types. The server determines the type of each video segment based on the plurality of prediction sequences.

[0190] For example, the server inputs the plurality of video frame features of each video segment into the classification layer of the video classification model, performs maximum pooling processing on the plurality of video frame features by the pooling layer in the classification layer, splices the plurality of video frame features after the maximum pooling processing to obtain a segment feature of the video segment. The server performs full connection processing on the segment feature of the video segment to obtain a prediction sequence of the video segment, such as (1, 3, 4, 6, 7, 8, 3), and each number in the prediction sequence represents the probability of the corresponding video segment belonging to different types. In some embodiments, the server performs normalization processing on the prediction sequence to obtain a prediction vector, and the numbers in the prediction vector represent the probabilities of the corresponding types of the video segment. The server determines the type with the highest probability as the type of the video segment. Wherein, the normalization processing is that the server maps the prediction sequence to a sequence between 0 and 1. In some embodiments, the server can use the Sigmoid (S-shaped growth curve) or Softmax (soft maximum) function to perform normalization processing on the prediction sequence, and the embodiments of the present application are not limited thereto.

[0191] 407、In response to the first repetition degree information of the video meeting the first target condition, the server determines the video as a target type video.

[0192] In some embodiments, in response to a picture repetition ratio of any video segment in the video being greater than or equal to a picture repetition ratio threshold value associated with the type of the video segment, the server determines the video as a target type video.

[0193] In some embodiments, the target type video is a loop video.

[0194] For example, if the video frame feature is a picture feature, the server determines the picture repetition ratio threshold value corresponding to each video segment according to the type of each video segment. For example, the server obtains the picture repetition ratio threshold value corresponding to each video segment from a picture repetition ratio threshold value table according to the type of each video segment. The picture repetition ratio threshold value table stores a plurality of video types and a picture repetition ratio threshold value corresponding to each video type. A person skilled in the art can set the picture repetition ratio threshold value corresponding to different video types in the picture repetition ratio threshold value table to adapt to more application scenarios. In response to the picture repetition ratio of any video segment in the video being greater than or equal to the picture repetition ratio threshold value, the server determines the video as a target type video. In response to the picture repetition ratio of any video segment in the video being less than the picture repetition ratio threshold value, the server determines the video as a non-target type video.

[0195] In some embodiments, in response to an audio repetition ratio of any video segment in the video being greater than or equal to an audio repetition ratio threshold value associated with the type of the video segment, the server determines the video as a target type video.

[0196] For example, if the video frame feature is an audio feature, the server determines the audio repetition ratio threshold value corresponding to each video segment according to the type of each video segment. For example, the server obtains the audio repetition ratio threshold value corresponding to each video segment from an audio repetition ratio threshold value table according to the type of each video segment. The audio repetition ratio threshold value table stores a plurality of video types and an audio repetition ratio threshold value corresponding to each video type. A person skilled in the art can set the audio repetition ratio threshold value corresponding to different video types in the audio repetition ratio threshold value table to adapt to more application scenarios. In response to the audio repetition ratio of any video segment in the video being greater than or equal to the audio repetition ratio threshold value, the server determines the video as a target type video. In response to the audio repetition ratio of any video segment in the video being less than the audio repetition ratio threshold value, the server determines the video as a non-target type video.

[0197] It should be noted that, in addition to determining the type of the video by the above two implementation manners respectively, the server can also combine the above two implementation manners to determine the type of the video, which will be described below through two examples.

[0198] Example 1, see Figure 6 The video includes six video clips A1-A6. If the server determines that the video clip A1 is a video clip of a life scene, and the video clip A4 has a picture repetition ratio greater than or equal to the picture repetition ratio threshold corresponding to the video clip A1 and an audio repetition ratio greater than or equal to the audio repetition ratio threshold corresponding to the video clip A1 relative to the video clip A1, the server can determine the video as a video of a target type, that is, a loop video.

[0199] Example 2, see Figure 7 The video includes five video clips B1-B5. If the server determines that the video clip B1 is a video clip of a car accident scene, and the video clips B2-B5 have a picture repetition ratio greater than or equal to the picture repetition ratio threshold corresponding to the video clip B1 and an audio repetition ratio less than the audio repetition ratio threshold corresponding to the video clip B1 relative to the video clip B1, the server can determine the video as a video of a non-target type, that is, a non-loop video.

[0200] In the above two examples, the joint recognition method of using audio features and picture features is adopted to effectively utilize multi-modal information and improve the accuracy of video recognition.

[0201] In some embodiments, in addition to determining the type of the video by the above step 407, the server can also combine at least one of the picture repetition ratio and the audio repetition ratio of the video to determine the type of the video.

[0202] In some embodiments, the server divides the total duration of the video clip in the video with a picture repetition ratio greater than or equal to a picture repetition ratio threshold by the duration of the video to obtain the picture repetition ratio of the video. In response to the picture repetition ratio of the video being greater than or equal to a video picture repetition ratio threshold, the server determines the video as a video of a target type, wherein the video picture repetition ratio threshold is associated with the content type of the video. Here, the content type and the target type refer to the types obtained after classification according to different standards, and the target type corresponds to a loop video, which is used to represent the structure type of the video. The content type refers to the type related to the content of the video, such as the video type, which can include but is not limited to a basketball goal, a football goal, a car accident, pure text, a close-up of a face, a life scene, and others (including non-goal moment commentator pictures).

[0203] For example, if the server stores the repeated video clips using the first clip storage list described in step 405 and sets the second clip storage list to store the non-repeated video clips, the server can determine a first total time T Snon of the video clips stored in the first clip storage list, determine a second total time T Sdup of the video clips stored in the second clip storage list, divide the second total time T Sdup by the sum of the time lengths of the video clips stored in the two clip storage lists T total , and obtain a picture repetition ratio of the video, where T total＝ T Snon + T Sdup . For the content type of the video, the server can determine the content type of the video according to the content types of the plurality of video clips in the video. For example, the server can determine the content type of the video as the content type with the largest quantity among the types of the plurality of video clips. The server can obtain the video picture repetition ratio threshold value corresponding to the content type of the video from the video picture repetition ratio threshold value table according to the content type of the video. The video picture repetition ratio threshold value table can store a plurality of video content types and a video picture repetition ratio threshold value corresponding to each video content type. The picture repetition ratio threshold value corresponding to each video content type in the video picture repetition ratio threshold value table can be set by the technician to adapt to more application scenarios. In response to the picture repetition ratio of the video being greater than or equal to the video picture repetition ratio threshold value, the server can determine the video as a video of the target type.

[0204] In some embodiments, the server can divide the total time of the video clips in the video whose audio repetition ratio is greater than or equal to the audio repetition ratio threshold value by the time length of the video to obtain the audio repetition ratio of the video. In response to the audio repetition ratio of the video being greater than or equal to the video audio repetition ratio threshold value, the server can determine the video as a video of the target type, where the video audio repetition ratio threshold value is associated with the content type of the video.

[0205] For example, if the server stores the repeated video clips using the first clip storage list described in step 405 and sets the second clip storage list to store the non-repeated video clips, the server can determine a first total time T Snon of the video clips stored in the first clip storage list, determine a second total time T Sdup of the video clips stored in the second clip storage list, divide the second total time T Sdup by the sum of the time lengths of the video clips stored in the two clip storage lists T total , and obtain an audio repetition ratio of the video, where T total＝ T Snon + T SdupFor the content type of the video, the server can determine the content type of the video according to the content types of the plurality of video segments in the video. For example, the server determines the content type with the largest quantity among the types of the plurality of video segments as the content type of the video. The server obtains the video-audio repetition ratio threshold corresponding to the video from the video-audio repetition ratio threshold table according to the content type of the video. The video-audio repetition ratio threshold table stores a plurality of video content types and the video-audio repetition ratio threshold corresponding to each video content type. The technical personnel can set the audio repetition ratio threshold corresponding to different video content types in the video-audio repetition ratio threshold table to adapt to more application scenarios. In response to the audio repetition ratio of the video being greater than or equal to the video-audio repetition ratio threshold, the server determines the video as a video of the target type.

[0206] In some embodiments, after step 407, the server can further obtain a non-repeating video segment from the video.

[0207] In some embodiments, if the server uses the first segment storage list described in step 405 to store the repeating video segments and sets a second segment storage list to store the non-repeating video segments, the server can directly obtain the non-repeating video segments from the first segment storage list.

[0208] The video identification method provided by the embodiments of the present application will be described below in combination with Figure 8 and the optional implementation of each of steps 401-407 described above.

[0209] See Figure 8The server extracts features of the video to obtain a plurality of picture features and a plurality of audio features. The server inputs the plurality of picture features into a video classification model to identify the type of the video by using the video classification model. The server obtains a picture repetition ratio threshold and an audio repetition ratio threshold of each video segment based on the type of the plurality of video segments. The server obtains a video picture repetition ratio threshold and a video audio repetition ratio threshold of the video based on the type of the video. The server determines the Euclidean distance between each two picture features in the plurality of picture features to obtain a numbered distance table (see Table 3 for form). The server adjusts the initial number of each picture feature based on the numbered distance table. The server performs denoising processing on the adjusted initial number to obtain a plurality of feature numbers. The server determines a plurality of segment division numbers from the plurality of feature numbers, and divides the video into a plurality of video segments based on the plurality of segment division numbers. In some embodiments, the server inputs the plurality of picture features of each video segment into the video classification model to identify the type of each video segment by using the video classification model. The server obtains second repetition information between different video segments, and determines whether the video segment is a repeated video segment or a non-repeated video segment based on the second repetition information. The server determines the first repetition information of the video based on the plurality of second repetition information. The server determines whether the video is of a target type or a non-target type according to the first repetition information. For the audio features, the server can determine the second repetition information of the audio features by using a process similar to the above process, and determine the first repetition information of the target video based on the second repetition information of the audio features. The server fuses the first repetition information obtained based on the picture features and the audio features to obtain a final identification result.

[0210] The above-mentioned various optional embodiments can be combined in any manner to form optional embodiments of the present application, which will not be described one by one here.

[0211] By the technical solutions provided in the embodiments of the present application, a plurality of video frame features can be numbered to obtain a plurality of feature numbers capable of representing the corresponding video frame features. In the numbering process, the similarity between the video frame features is combined, that is, if the numbers of two video frame features are the same, it means that the similarity between the two video frame features is high, and further means that the similarity between the two video frames corresponding to the two video frame features is high. The numbering of the video frame features is used to determine the first repetition information of the video, which quickly reflects the repetition of the video segments in the video. When the first repetition information meets the target condition, the video can be determined as a video of a target type. The entire process is implemented by a computer device, without the need for manual detection, thereby improving the efficiency of video type determination.

[0212] In addition, by adopting the technical solutions provided in the embodiments of the present application, in addition to being capable of identifying the type of the video, the video cycle ratio (the ratio of the time length of the repeated video segment to the total time length of the video) and whether it is a specific type of video segment (whether it is a specific type such as a goal, a car accident moment, etc. is determined according to the classification of the video segment by the video type classification model) can be given. The video can also be compressed, that is, the non-repeated video segment in the output video is output, the storage space occupied by the features in the subsequent identification, detection, and deduplication tasks of the video is reduced, and the proportion of the effective video, or the non-repeated video, is improved.

[0213] Referring to Figure 9 , in addition to being capable of determining the type of the video, by adopting the technical solutions provided in the embodiments of the present application, the repeated ratio of the video, the type of the repeated video segment, and the non-repeated video segment in the video can be determined under the premise that the video is a cyclic video.

[0214] Figure 10 is a structural schematic diagram of a video identification device provided in an embodiment of the present application, referring to Figure 10 , the device comprises a video frame feature acquisition unit 1001, a numbering unit 1002, a first repetition degree information determination unit 1003, and a video type determination unit 1004.

[0215] The video frame feature acquisition unit 1001 is configured to acquire a plurality of video frame features of a video, wherein the video frame features comprise at least one of a picture feature and an audio feature corresponding to a video frame.

[0216] The numbering unit 1002 is configured to number the plurality of video frame features to obtain a plurality of feature numbers, wherein the feature numbers of any two video frame features with a similarity meeting a target similarity condition in the plurality of video frame features are the same.

[0217] The first repetition degree information determination unit 1003 is configured to determine first repetition degree information of the video based on the plurality of feature numbers, wherein the first repetition degree information is used to indicate at least one of a picture repetition condition and an audio repetition condition of the video frame in a plurality of video segments of the video.

[0218] The video type determination unit 1004 is configured to determine the video as a video of a target type in response to the first repetition degree information of the video meeting a first target condition.

[0219] In some embodiments, the video frame feature acquisition unit 1001 is configured to perform at least one of the following:

[0220] inputting the plurality of video frames of the video into a video classification model, performing feature extraction on the pictures of the plurality of video frames by a feature extraction layer of the video classification model to obtain a plurality of picture features.

[0221] The audio of the plurality of video frames of the video is input into an audio feature extraction model, and the audio of the plurality of video frames is subjected to feature extraction through a feature extraction layer of the audio feature extraction model to obtain a plurality of audio features.

[0222] In some embodiments, the numbering unit 1002 is configured to determine, as an initial number corresponding to each video frame feature, an arrangement order of the video frame corresponding to the video frame feature in the video.

[0223] Based on the similarity between each two video frame features and the plurality of initial numbers, a plurality of feature numbers is obtained.

[0224] In some embodiments, the numbering unit 1002 is configured to adjust, based on the similarity between each two video frame features, part of the plurality of initial numbers to obtain the plurality of feature numbers, the part of the plurality of initial numbers referring to a number corresponding to a target video frame feature, the target video frame feature being a video frame feature with a later arrangement order among any two video frame features.

[0225] In some embodiments, the numbering unit 1002 is configured to, in response to the similarity between any two video frame features meeting a target similarity condition, adjust an initial number corresponding to a target video frame feature among the any two video frame features to an initial number corresponding to a video frame feature with an earlier arrangement order.

[0226] In some embodiments, the apparatus further comprises:

[0227] The de-noising unit is configured to perform de-noising processing on the adjusted plurality of initial numbers.

[0228] In some embodiments, the de-noising unit is configured to determine a plurality of noise point numbers in the adjusted plurality of initial numbers, and to adjust each noise point number based on at least two initial numbers adjacent to the noise point number in the arrangement order in the adjusted plurality of initial numbers.

[0229] In some embodiments, the de-noising unit is configured to, in response to a difference between a first number and a second number being greater than a first difference threshold and a difference between the first number and a third number being less than or equal to a second difference threshold, determine the first number as a noise point number.

[0230] wherein the first number is any initial number in the adjusted plurality of initial numbers, the second number is an initial number in the adjusted plurality of initial numbers with an earlier arrangement order than the first number, and the third number is an initial number in the adjusted plurality of initial numbers with a later arrangement order than the first number.

[0231] In some embodiments, the de-noising unit is configured to adjust each noise point number to a target number, the target number being an average of the second number and the third number.

[0232] The second number is an initial number in the adjusted plurality of initial numbers, and the initial number is in the order before the noise number. The third number is an initial number in the adjusted plurality of initial numbers, and the initial number is in the order after the noise number.

[0233] In some embodiments, the first repetition information determination unit 1003 is configured to: obtain a same number of a plurality of first feature numbers and a plurality of second feature numbers for a first video segment and a second video segment in the video, the first feature number being a feature number corresponding to a video frame in the first video segment, and the second feature number being a feature number corresponding to a video frame in the second video segment.

[0234] The second repetition information determination unit 1002 is configured to: determine second repetition information of the first video segment and the second video segment based on the same number, the second repetition information being used to indicate at least one of a picture repetition case and an audio repetition case of the video frame in the first video segment and the video frame in the second video segment.

[0235] The first repetition information determination unit 1003 is configured to: determine the first repetition information of the video based on a plurality of second repetition information.

[0236] In some embodiments, the first repetition information determination unit 1003 is configured to: divide the same number by a target number to obtain a picture repetition ratio or an audio repetition ratio of the video frame in the first video segment and the video frame in the second video segment, the target number being a number of feature numbers corresponding to the first video segment or the second video segment, the picture repetition ratio being used to represent the picture repetition case of the video frame in the first video segment and the video frame in the second video segment, and the audio repetition ratio being used to represent the audio repetition case of the video frame in the first video segment and the video frame in the second video segment.

[0237] In some embodiments, the apparatus further includes:

[0238] The video segment division unit is configured to: determine a plurality of segment division numbers from the plurality of feature numbers, each segment division number being less than a previous feature number, and a number difference between each segment division number and the previous feature number being greater than a target difference, the target difference being in direct proportion to an average difference between a plurality of feature numbers in the order before the segment division number.

[0239] The video division unit is configured to: divide the video into a plurality of video segments based on the plurality of segment division numbers.

[0240] In some embodiments, the video type determination unit 1004 is configured to perform any one of the following:

[0241] In response to a picture repetition ratio of any video segment in the video being greater than or equal to a picture repetition ratio threshold value associated with the type of the any video segment, the video is determined to be a video of a target type.

[0242] In response to an audio repetition ratio of any video segment in the video being greater than or equal to an audio repetition ratio threshold value associated with the type of the any video segment, the video is determined to be a video of a target type.

[0243] In some embodiments, the apparatus further comprises:

[0244] a classification unit configured to input the plurality of video segments into a video classification model, and classify the plurality of video segments through the video classification model to output types of the plurality of video segments.

[0245] In some embodiments, the apparatus further comprises any one of the following units:

[0246] a picture repetition ratio determination unit configured to divide a total time length of video segments in the video, in which a picture repetition ratio is greater than or equal to a picture repetition ratio threshold value, by a time length of the video to obtain a picture repetition ratio of the video.

[0247] an audio repetition ratio determination unit configured to divide a total time length of video segments in the video, in which an audio repetition ratio is greater than or equal to an audio repetition ratio threshold value, by a time length of the video to obtain an audio repetition ratio of the video.

[0248] It should be noted that the video recognition apparatus provided in the above embodiments is only used as an example for the division of the above functional units during video recognition. In actual applications, the above functions can be completed by different functional units according to needs, that is, the internal structure of the computer device is divided into different functional units to complete all or part of the above-described functions. In addition, the apparatus for video recognition provided in the above embodiments and the method for video recognition provided in the embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0249] By the technical solutions provided in the embodiments of the present application, the plurality of video frame features can be numbered, and the plurality of feature numbers obtained can represent the corresponding video frame features. In the numbering process, the similarity between the video frame features is combined, that is, if the numbers of two video frame features are the same, it can be represented that the similarity between the two video frame features is high, and further, it can be represented that the similarity between the two video frames corresponding to the two video frame features is high. The numbering of the video frame features is used to determine the first repetition degree information of the video, which quickly reflects the repetition of the video segment in the video. When the first repetition information meets the target condition, the video can be determined as a video of a target type. The entire process is implemented by a computer device, without manual detection, thereby improving the efficiency of video type determination.

[0250] The embodiments of the present application provide a computer device for executing the above method, which can be implemented as a terminal or a server. The structure of the terminal is introduced as follows:

[0251] Figure 11 FIG. 1 is a structural schematic diagram of a terminal provided by the embodiments of the present application.

[0252] Generally, the terminal 1100 includes one or more processors 1101 and one or more memories 1102.

[0253] The processor 1101 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1101 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 1101 can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1101 can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 1101 can also include an AI (Artificial Intelligence) processor for processing machine learning-related computing operations.

[0254] The memory 1102 can include one or more computer-readable storage media. The computer-readable storage media can be non-transitory. The memory 1102 can also include high-speed random access memory and can include nonvolatile memory, such as one or more magnetic disk storage devices, optical storage devices, flash memory devices, or other memory.

[0255] In some embodiments, the terminal 1100 can optionally also include a peripheral interface 1103 and at least one peripheral device. The processor 1101, the memory 1102, and the peripheral interface 1103 can be connected through buses or signal lines. Each peripheral device can be connected to the peripheral interface 1103 through buses, signal lines, or circuit boards. Specifically, the peripheral devices include at least one of a radio frequency circuit 1104, a display screen 1105, a camera assembly 1106, an audio circuit 1107, a positioning assembly 1108, and a power supply 1109.

[0256] The peripheral interface 1103 can be used to connect at least one peripheral device related to input / output to the processor 1101 and the memory 1102. In some embodiments, the processor 1101, the memory 1102, and the peripheral interface 1103 are integrated on the same chip or circuit board. In some other embodiments, any one or two of the processor 1101, the memory 1102, and the peripheral interface 1103 can be implemented on a separate chip or circuit board, and the embodiments are not limited in this regard.

[0257] The radio frequency circuit 1104 is used to receive and send RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1104 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 1104 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. In some embodiments, the radio frequency circuit 1104 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a coding chipset, a subscriber identity module card, and the like.

[0258] The display 1105 is configured to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display 1105 is a touch display, the display 1105 is further capable of acquiring a touch signal on or above the surface of the display 1105. The touch signal can be input to the processor 1101 as a control signal for processing. At this time, the display 1105 can also be configured to provide a virtual button and / or a virtual keyboard, also known as a soft button and / or a soft keyboard.

[0259] The camera component 1106 is configured to acquire an image or a video. In some embodiments, the camera component 1106 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is disposed on the front panel of the terminal, and the rear-facing camera is disposed on the back of the terminal.

[0260] The audio circuit 1107 can include a microphone and a speaker. The microphone is configured to acquire sound waves of a user and an environment, and convert the sound waves into an electrical signal input to the processor 1101 for processing, or input to the radio frequency circuit 1104 to implement voice communication.

[0261] The positioning component 1108 is configured to locate the current geographical position of the terminal 1100 to implement navigation or LBS (Location Based Service).

[0262] The power supply 1109 is configured to supply power to various components in the terminal 1100. The power supply 1109 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery.

[0263] In some embodiments, the terminal 1100 further includes one or more sensors 1110. The one or more sensors 1110 include, but are not limited to, an acceleration sensor 1111, a gyroscope sensor 1112, a pressure sensor 1113, a fingerprint sensor 1114, an optical sensor 1115, and a proximity sensor 1116.

[0264] The acceleration sensor 1111 can detect the acceleration magnitude in three coordinate axes of a coordinate system established by the terminal 1100.

[0265] The gyroscope sensor 1112 can detect the body direction and rotation angle of the terminal 1100. The gyroscope sensor 1112 can cooperate with the acceleration sensor 1111 to acquire the 3D motion of the user on the terminal 1100.

[0266] The pressure sensor 1113 can be disposed at the side frame of the terminal 1100 and / or the lower layer of the display screen 1105. When the pressure sensor 1113 is disposed at the side frame of the terminal 1100, the holding signal of the user to the terminal 1100 can be detected, and the left-hand or right-hand recognition or the shortcut operation can be performed by the processor 1101 according to the holding signal collected by the pressure sensor 1113. When the pressure sensor 1113 is disposed at the lower layer of the display screen 1105, the controllable control on the UI interface can be controlled by the processor 1101 according to the pressure operation of the user to the display screen 1105.

[0267] The fingerprint sensor 1114 is configured to collect the fingerprint of the user, and the processor 1101 can identify the identity of the user according to the fingerprint collected by the fingerprint sensor 1114, or the fingerprint sensor 1114 can identify the identity of the user according to the collected fingerprint.

[0268] The optical sensor 1115 is configured to collect the ambient light intensity. In an embodiment, the processor 1101 can control the display brightness of the display screen 1105 according to the ambient light intensity collected by the optical sensor 1115.

[0269] The proximity sensor 1116 is configured to collect the distance between the user and the front of the terminal 1100.

[0270] Those skilled in the art can understand that the structure shown in the foregoing embodiments is not a limitation on the terminal 1100, and the terminal 1100 can include more or fewer components than those shown in the drawings, or combine certain components, or adopt a different component arrangement. Figure 11 The structure shown in the foregoing embodiments is not a limitation on the terminal 1100, and the terminal 1100 can include more or fewer components than those shown in the drawings, or combine certain components, or adopt a different component arrangement.

[0271] The foregoing computer device can also be implemented as a server, and the structure of the server will be introduced as follows:

[0272] Figure 12 FIG. 12 is a structural schematic diagram of a server provided by an embodiment of the present application. The server 1200 can have great differences due to different configurations or performances, and can include one or more processors (Central Processing Units, CPUs) 1201 and one or more memories 1202. The one or more memories 1202 store at least one computer program, and the at least one computer program is loaded and executed by the one or more processors 1201 to implement the method provided by the foregoing various method embodiments. Of course, the server 1200 can also have a wired or wireless network interface, a keyboard, an input and output interface, and other components for implementing the functions of the device, and details are not described herein.

[0273] In the example embodiments, a computer readable storage medium, such as a memory including a computer program executable by a processor to implement the video recognition method in the above embodiments, is also provided. For example, the computer readable storage medium can be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0274] In the example embodiments, a computer program product or computer program including program code stored in a computer readable storage medium is also provided, and the processor of a computer device reads the program code from the computer readable storage medium, and the processor executes the program code to cause the computer device to perform the video recognition method.

[0275] In some embodiments, the computer program related to the embodiments of the present application can be deployed to execute on one computer device, or on multiple computer devices located in one place, or on multiple computer devices distributed in multiple places and interconnected through a communication network, which can constitute a blockchain system.

[0276] Those of ordinary skill in the art can understand that all or part of the steps of the above embodiments can be completed by hardware, or by a program instructing relevant hardware, and the program can be stored in a computer readable storage medium, such as a Read-Only Memory, a magnetic disk or an optical disk.

[0277] The above is only an optional embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A video recognition method, characterized in that, The method includes: The video frame features are obtained from multiple video frame features, wherein the video frame features include at least one of the picture features and audio features of the corresponding video frame; The order in which the video frames corresponding to each video frame feature are arranged in the video is used to determine the initial number corresponding to each video frame feature; based on the similarity between any two video frame features and the multiple initial numbers, multiple feature numbers are obtained, wherein any two video frame features whose similarity meets the target similarity condition have the same feature number. For a first video segment and a second video segment in the video, the number of identical numbers among a plurality of first feature numbers and a plurality of second feature numbers is obtained. The first feature number is the feature number corresponding to a video frame in the first video segment, and the second feature number is the feature number corresponding to a video frame in the second video segment. Based on the number of identical numbers, second repetition information of the first video segment and the second video segment is determined. The second repetition information is used to indicate at least one of the image repetition and audio repetition of video frames in the first video segment and the second video segment. Based on the plurality of second repetition information, first repetition information of the video is determined. The first repetition information is used to indicate at least one of the image repetition and audio repetition of video frames in the plurality of video segments of the video. In response to the first repetition information of the video meeting the first target condition, the video is identified as a video of the target type.

2. The method according to claim 1, characterized in that, The multiple video frame features acquired from the video include at least one of the following: Multiple video frames of the video are input into a video classification model. The feature extraction layer of the video classification model is used to extract features from the multiple video frames to obtain multiple image features. The audio of multiple video frames of the video is input into the audio feature extraction model. The feature extraction layer of the audio feature extraction model is used to extract features from the audio of the multiple video frames to obtain multiple audio features.

3. The method according to claim 1, characterized in that, The process of obtaining multiple feature numbers based on the similarity between features of every two video frames and multiple initial numbers includes: Based on the similarity between any two video frame features, a portion of the initial numbers is adjusted to obtain the multiple feature numbers. The portion number refers to the number corresponding to the target video frame feature, which is the video frame feature with the later numbering among any two video frame features.

4. The method according to claim 3, characterized in that, The adjustment of a portion of the initial numbers based on the similarity between features of every two video frames includes: In response to the similarity between any two video frame features meeting the target similarity condition, the initial number corresponding to the target video frame feature among the two video frame features is adjusted to the initial number corresponding to the video frame feature with the earlier numbering order.

5. The method according to claim 4, characterized in that, After adjusting the initial number corresponding to the target video frame feature among any two video frame features to the initial number corresponding to the video frame feature with the earlier numbering order, the method further includes: The adjusted initial numbers are then denoised.

6. The method according to claim 5, characterized in that, The noise reduction process for the adjusted initial numbers includes: Multiple noise point numbers are determined from the adjusted initial numbers; In the adjusted plurality of initial numbers, each noise number is adjusted based on at least two initial numbers whose numbering order is adjacent to each noise number.

7. The method according to claim 6, characterized in that, The step of determining multiple noise point numbers from the adjusted multiple initial numbers includes: In response to the fact that the difference between the first number and the second number is greater than a first difference threshold, and the difference between the first number and the third number is less than or equal to a second difference threshold, the first number is determined as a noise number. Wherein, the first number is any one of the adjusted initial numbers; the second number is the initial number that precedes the first number in the adjusted initial number sequence; and the third number is the initial number that follows the first number in the adjusted initial number sequence.

8. The method according to claim 6, characterized in that, The adjustment of each noise number based on at least two initial numbers adjacent to each noise number in the numbering order includes: Each of the noise point numbers is adjusted to a target number, which is the average of the second number and the third number; Wherein, the second number is the initial number in the adjusted plurality of initial numbers that is numbered before the noise number; the third number is the initial number in the adjusted plurality of initial numbers that is numbered after the noise number.

9. The method according to claim 1, characterized in that, The second repetition information for determining the first video segment and the second video segment based on the number of identical numbers includes: Divide the number of identical numbers by the target number to obtain the screen repetition ratio or audio repetition ratio of video frames in the first video segment and video frames in the second video segment. The target number is the number of feature numbers corresponding to the first video segment or the second video segment. The screen repetition ratio is used to represent the screen repetition situation of video frames in the first video segment and video frames in the second video segment. The audio repetition ratio is used to represent the audio repetition situation of video frames in the first video segment and video frames in the second video segment.

10. The method according to claim 1, characterized in that, Before determining the first repetition information of the video, the method further includes: Multiple segmentation numbers are determined from the plurality of feature numbers, each segmentation number being less than the previous feature number, and the number difference between each segmentation number and the previous feature number being greater than a target difference, wherein the target difference is proportional to the average difference between the plurality of feature numbers whose numbering order precedes the segmentation number; Based on the multiple segmentation numbers, the video is divided into the multiple video segments.

11. A video recognition device, characterized in that, The device includes: A video frame feature acquisition unit is used to acquire multiple video frame features of a video, wherein the video frame features include at least one of the picture features and audio features of the corresponding video frame. The numbering unit is used to determine the arrangement order of the video frames corresponding to each video frame feature in the video, and to determine the initial number corresponding to each video frame feature; based on the similarity between each two video frame features and the multiple initial numbers, multiple feature numbers are obtained, and the feature numbers of any two video frame features whose similarity meets the target similarity condition are the same. The first repetition information determining unit is configured to, for a first video segment and a second video segment in the video, obtain the number of identical numbers among a plurality of first feature numbers and a plurality of second feature numbers, wherein the first feature number is the feature number corresponding to a video frame in the first video segment, and the second feature number is the feature number corresponding to a video frame in the second video segment; determine second repetition information of the first video segment and the second video segment based on the number of identical numbers, wherein the second repetition information is used to indicate at least one of the image repetition and audio repetition of video frames in the first video segment and the second video segment; and determine first repetition information of the video based on the plurality of second repetition information, wherein the first repetition information is used to indicate at least one of the image repetition and audio repetition of video frames in the plurality of video segments of the video. The video type determination unit is configured to determine the video as a target type video in response to the first repetition information of the video meeting the first target condition.

12. The apparatus according to claim 11, characterized in that, The video frame feature acquisition unit is configured to perform at least one of the following: Multiple video frames of the video are input into a video classification model. The feature extraction layer of the video classification model is used to extract features from the multiple video frames to obtain multiple image features. The audio of multiple video frames of the video is input into the audio feature extraction model. The feature extraction layer of the audio feature extraction model is used to extract features from the audio of the multiple video frames to obtain multiple audio features.

13. The apparatus according to claim 11, characterized in that, The numbering unit is used for: Based on the similarity between any two video frame features, a portion of the initial numbers is adjusted to obtain the multiple feature numbers. The portion number refers to the number corresponding to the target video frame feature, which is the video frame feature with the later numbering among any two video frame features.

14. The apparatus according to claim 13, characterized in that, The numbering unit is used for: In response to the similarity between any two video frame features meeting the target similarity condition, the initial number corresponding to the target video frame feature among the two video frame features is adjusted to the initial number corresponding to the video frame feature with the earlier numbering order.

15. The apparatus according to claim 14, characterized in that, The device further includes: The denoising unit is used to denoise the adjusted initial numbers.

16. The apparatus according to claim 15, characterized in that, The noise reduction unit is used for: Multiple noise point numbers are determined from the adjusted initial numbers; In the adjusted plurality of initial numbers, each noise number is adjusted based on at least two initial numbers whose numbering order is adjacent to each noise number.

17. The apparatus according to claim 16, characterized in that, The noise reduction unit is used for: In response to the fact that the difference between the first number and the second number is greater than a first difference threshold, and the difference between the first number and the third number is less than or equal to a second difference threshold, the first number is determined as a noise number. Wherein, the first number is any one of the adjusted initial numbers; the second number is the initial number that precedes the first number in the adjusted initial number sequence; and the third number is the initial number that follows the first number in the adjusted initial number sequence.

18. The apparatus according to claim 16, characterized in that, The noise reduction unit is used for: Each of the noise point numbers is adjusted to a target number, which is the average of the second number and the third number; Wherein, the second number is the initial number in the adjusted plurality of initial numbers that is numbered before the noise number; the third number is the initial number in the adjusted plurality of initial numbers that is numbered after the noise number.

19. The apparatus according to claim 11, characterized in that, The first repeatability information determination unit is used for: Divide the number of identical numbers by the target number to obtain the screen repetition ratio or audio repetition ratio of video frames in the first video segment and video frames in the second video segment. The target number is the number of feature numbers corresponding to the first video segment or the second video segment. The screen repetition ratio is used to represent the screen repetition situation of video frames in the first video segment and video frames in the second video segment. The audio repetition ratio is used to represent the audio repetition situation of video frames in the first video segment and video frames in the second video segment.

20. The apparatus according to claim 11, characterized in that, The device further includes: A video segmentation unit is used to determine multiple segmentation numbers from the multiple feature numbers, wherein each segmentation number is less than the previous feature number, and the number difference between each segmentation number and the previous feature number is greater than a target difference, wherein the target difference is proportional to the average difference between multiple feature numbers whose numbering order precedes the segmentation number; and the video is divided into multiple video segments based on the multiple segmentation numbers.

21. A computer device, characterized in that, The computer device includes one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the computer program is loaded and executed by the one or more processors to implement the video recognition method as described in any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the video recognition method as described in any one of claims 1 to 10.

Images