A video feature extraction method, device, readable storage medium and terminal equipment

Abstract

The application belongs to the technical field of target identification, and particularly relates to a video feature extraction method and device, a computer readable storage medium and a terminal device. The method comprises the following steps: acquiring a video sequence to be processed; performing image feature extraction on each video frame in the video sequence respectively to obtain first image features of the video frames; calculating first video features of the video sequence according to the first image features of the video frames; performing feature optimization on the first image features of the video frames respectively according to the first video features to obtain second image features of the video frames; and calculating second video features of the video sequence according to the second image features of the video frames. Through the application, the quality influence of a video frame with poor quality on the final extracted video features can be effectively weakened, and the robustness of video feature extraction is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of target recognition, and particularly relates to a video feature extraction method and device, a computer readable storage medium, and a terminal device. BACKGROUND

[0002] Video feature extraction refers to extracting feature information from a video sequence so as to recognize the video sequence by using the feature information. With the development of artificial intelligence technology, various videos are increasingly rich, and how to accurately extract video features becomes particularly important.

[0003] In the prior art, the average method is generally used to calculate video features, that is, each video frame has the same contribution to the finally extracted video features. If there are poor-quality video frames such as image blur and overexposure in the video sequence, the quality of the finally extracted video features will be affected, and the robustness is poor. SUMMARY

[0004] Therefore, the embodiments of the present application provide a video feature extraction method, device, computer readable storage medium, and terminal device to solve the problem of poor robustness of the existing video feature extraction method.

[0005] The first aspect of the embodiments of the present application provides a video feature extraction method, which can include:

[0006] obtaining a video sequence to be processed;

[0007] performing image feature extraction on each video frame in the video sequence respectively to obtain first image features of the video frames;

[0008] calculating first video features of the video sequence according to the first image features of the video frames;

[0009] performing feature optimization on the first image features of the video frames respectively according to the first video features to obtain second image features of the video frames;

[0010] calculating second video features of the video sequence according to the second image features of the video frames.

[0011] In a specific implementation manner of the first aspect, the performing feature optimization on the first image features of the video frames respectively according to the first video features to obtain the second image features of the video frames can include:

[0012] calculating a feature similarity between the first video features and first image features of a target video frame; wherein the target video frame is any one of the video frames in the video sequence;

[0013] According to the feature similarity, the first image feature of the target video frame, and the first video feature, a second image feature of the target video frame is calculated.

[0014] In an implementation form of the first aspect, the calculating the second image feature of the target video frame according to the feature similarity, the first image feature of the target video frame, and the first video feature can include:

[0015] determining a first weight and a second weight according to the feature similarity respectively, wherein the first weight is a weight corresponding to the first image feature of the target video frame, the second weight is a weight corresponding to the first video feature, the first weight is positively correlated with the feature similarity, and the second weight is negatively correlated with the feature similarity;

[0016] performing feature fusion on the first image feature of the target video frame and the first video feature according to the first weight and the second weight to obtain the second image feature of the target video frame.

[0017] In an implementation form of the first aspect, the determining the first weight and the second weight according to the feature similarity can include:

[0018] determining the feature similarity as the first weight;

[0019] determining a difference between a preset weight sum and the feature similarity as the second weight.

[0020] In an implementation form of the first aspect, the performing feature fusion on the first image feature of the target video frame and the first video feature according to the first weight and the second weight to obtain the second image feature of the target video frame can include:

[0021] calculating a first weighted feature according to the first weight and the first image feature of the target video frame;

[0022] calculating a second weighted feature according to the second weight and the first video feature;

[0023] calculating the second image feature of the target video frame according to the first weighted feature and the second weighted feature.

[0024] In an implementation form of the first aspect, the calculating the first video feature of the video sequence according to the first image feature of each video frame can include:

[0025] performing average processing on the first image feature of each video frame to obtain the first video feature.

[0026] In a specific implementation manner of the first aspect, the calculating the second video feature of the video sequence according to the second image features of the respective video frames can include:

[0027] averaging the second image features of the respective video frames to obtain the second video feature.

[0028] The second aspect of the embodiments of the present application provides a video feature extraction device, which can include:

[0029] a video sequence acquisition module configured to acquire a video sequence to be processed;

[0030] a feature extraction module configured to perform image feature extraction on each video frame in the video sequence respectively to obtain a first image feature of each video frame;

[0031] a first video feature calculation module configured to calculate a first video feature of the video sequence according to the first image features of the respective video frames;

[0032] a feature optimization module configured to perform feature optimization on the first image features of the respective video frames respectively according to the first video feature to obtain a second image feature of each video frame;

[0033] a second video feature calculation module configured to calculate a second video feature of the video sequence according to the second image features of the respective video frames.

[0034] In a specific implementation manner of the second aspect, the feature optimization module can include:

[0035] a similarity calculation sub-module configured to calculate a feature similarity between the first video feature and a first image feature of a target video frame; wherein the target video frame is any one of the video sequence;

[0036] a feature calculation sub-module configured to calculate a second image feature of the target video frame according to the feature similarity, the first image feature of the target video frame, and the first video feature.

[0037] In a specific implementation manner of the second aspect, the feature calculation sub-module can include:

[0038] a weight determination unit configured to determine a first weight and a second weight respectively according to the feature similarity; wherein the first weight is a weight corresponding to the first image feature of the target video frame, the second weight is a weight corresponding to the first video feature, the first weight is positively correlated with the feature similarity, and the second weight is negatively correlated with the feature similarity;

[0039] The feature fusion unit is configured to perform feature fusion on the first image feature of the target video frame and the first video feature according to the first weight and the second weight, to obtain a second image feature of the target video frame.

[0040] In an implementation form of the second aspect, the weight determination unit can be specifically configured to determine the feature similarity as the first weight, and determine a difference between a preset weight sum and the feature similarity as the second weight.

[0041] In an implementation form of the second aspect, the feature fusion unit can be specifically configured to calculate a first weighted feature according to the first weight and the first image feature of the target video frame, calculate a second weighted feature according to the second weight and the first video feature, and calculate the second image feature of the target video frame according to the first weighted feature and the second weighted feature.

[0042] In an implementation form of the second aspect, the first video feature calculation module can be specifically configured to perform average processing on the first image features of the video frames to obtain the first video feature.

[0043] In an implementation form of the second aspect, the second video feature calculation module can be specifically configured to perform average processing on the second image features of the video frames to obtain the second video feature.

[0044] The third aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the steps of any of the video feature extraction methods.

[0045] The fourth aspect of the embodiments of the present application provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor implements the steps of any of the video feature extraction methods when executing the computer program.

[0046] The fifth aspect of the embodiments of the present application provides a computer program product, which, when executed on a terminal device, causes the terminal device to perform the steps of any of the video feature extraction methods.

[0047] The beneficial effects of the embodiments of the present application compared with the prior art are: the embodiments of the present application obtain a video sequence to be processed; image features of each video frame in the video sequence are extracted respectively to obtain first image features of the each video frame; first video features of the video sequence are calculated according to the first image features of the each video frame; the first image features of the each video frame are respectively optimized according to the first video features to obtain second image features of the each video frame; and second video features of the video sequence are calculated according to the second image features of the each video frame. In the embodiments of the present application, the initial image features (i.e. the first image features) of the each video frame can be optimized through the initial video features (i.e. the first video features) of the video sequence, thereby effectively weakening the quality influence of the video frame with poor quality on the final extracted video features and improving the robustness of the video feature extraction. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0049] Figure 1 An embodiment flow chart of a video feature extraction method in the embodiments of the present application;

[0050] Figure 2 An embodiment flow chart of optimizing the first image features of the each video frame according to the first video features;

[0051] Figure 3 An embodiment complete execution process schematic diagram of a video feature extraction method in the embodiments of the present application;

[0052] Figure 4 An embodiment structure diagram of a video feature extraction device in the embodiments of the present application;

[0053] Figure 5 An embodiment schematic block diagram of a terminal device in the embodiments of the present application. DETAILED DESCRIPTION

[0054] In order to make the objectives, characteristics, advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the embodiments described below are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of the present application.

[0055] It should be understood that the term "comprising" as used in the specification and the appended claims indicates the presence of the recited features, integers, steps, operations, elements, and / or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0056] It should also be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0057] It should be further understood that the term "and / or" as used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0058] As used in the present application specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted as meaning "upon determining" or "in response to determining" or "upon detecting [a described condition or event]" or "in response to detecting [a described condition or event]" depending on the context.

[0059] In addition, in the description of the present application, the terms "first", "second", "third" and the like are only used to distinguish description and cannot be understood as indicating or implying relative importance.

[0060] In the prior art, the average method is generally used to calculate the video features, that is, the contribution of each video frame to the final extracted video features is the same. If there are image blur, overexposure and other poor quality video frames in the video sequence, the quality of the final extracted video features will be affected, and the robustness is poor.

[0061] Therefore, the embodiment of the present application provides a video feature extraction method, device, computer readable storage medium and terminal equipment. In the embodiment of the present application, the image feature optimization can be performed on the initial image features of each video frame based on the initial video features of the video sequence, thereby effectively weakening the influence of the poor quality video frame on the quality of the final extracted video features, and improving the robustness of the video feature extraction.

[0062] It should be noted that the execution subject of the method of the present application is a terminal equipment, which specifically can include but is not limited to mobile phones, tablet computers, desktop computers, notebooks, palmtop computers and robot computing devices.

[0063] Please refer to Figure 1 An embodiment of the video feature extraction method in the embodiment of the present application can include:

[0064] Step S101, obtaining a video sequence to be processed.

[0065] The video sequence is a sequence composed of at least two video frames. In the embodiment of the present application, the video sequence can be collected by a preset camera collection device. For example, a video can be shot by a monitoring camera installed at a preset position, and the video can be taken as a video sequence.

[0066] After the video sequence is collected, it can also be stored in a preset position, so that the video sequence can be obtained in time when video feature extraction is needed. For example, the video sequence can be stored in the memory module of the terminal equipment, and the video sequence can be directly obtained from the memory module of the terminal equipment when video feature extraction is needed. For another example, in order to realize data security backup, the terminal equipment can upload memory data to the cloud server regularly, and the video sequence can be stored in the cloud server. When video feature extraction is needed, a video sequence acquisition request can be sent to the cloud server, and the video sequence can be obtained by analyzing the response message returned by the cloud server.

[0067] In order to improve the efficiency of video feature extraction, the collected video sequence can also be subjected to frame extraction processing. For example, one frame can be extracted from every N frames, thereby obtaining the video sequence to be processed. Wherein, N is an integer greater than 1.

[0068] Step S102, performing image feature extraction on each video frame in the video sequence respectively to obtain the first image feature of each video frame.

[0069] After obtaining the video sequence, the video sequence can be split to obtain each video frame in the video sequence. For each video frame, image feature extraction can be performed to extract the image feature information in the video frame, which is denoted as the first image feature.

[0070] In a specific application, any image feature extraction method in the prior art can be selected according to actual conditions, and the embodiments of the present application do not make specific limitations.

[0071] In a specific implementation manner of the embodiments of the present application, the preset image feature extraction network can be used to perform image feature extraction on each video frame {I1, I2, I3,..., I T} in the video sequence respectively, to obtain corresponding first image features {F1, F2, F3,..., F T} of each video frame. The image feature extraction network is a neural network used for image feature extraction, I t is the tth video frame in the video sequence, and F t is the first image feature of the tth video frame, t is the video frame serial number in the video sequence, 1≤t≤T, and T is the total number of video frames in the video sequence.

[0072] It should be noted that the embodiments of the present application do not make specific limitations on the selection of the image feature extraction network, and it can be set according to actual needs. For example, the image feature extraction network can be any neural network for feature extraction in the prior art, such as a convolutional neural network (CNN) or a recurrent neural network (RNN). Before use, the selected neural network can be trained in advance to obtain the image feature extraction network in the embodiments of the present application, which can be directly used for image feature extraction in the subsequent process.

[0073] Step S103, calculating the first video feature of the video sequence according to the first image features of each video frame.

[0074] In a specific implementation manner of the embodiments of the present application, the first image features of each video frame can be averaged to obtain an initial video feature (denoted as the first video feature), as shown in the following formula:

[0075]

[0076] Wherein, F is the first video feature.

[0077] Step S104, performing feature optimization on the first image features of each video frame respectively according to the first video feature, to obtain second image features of each video frame.

[0078] For ease of understanding, an example of any one video frame in the video sequence (denoted as a target video frame) is taken to explain the calculation process of the second image feature in detail. As shown in the following formula: Figure 2As shown, the calculation process of the second image feature of the target video frame can specifically include:

[0079] In step S1041, a feature similarity between the first video feature and the first image feature of the target video frame is calculated.

[0080] In a specific implementation manner of the embodiment of the present application, the inner product of vectors between the first video feature and the first image feature of the target video frame, and the product of the first vector modulus and the second vector modulus can be calculated respectively, wherein the first vector modulus is the modulus of the vector of the first video feature, and the second vector modulus is the modulus of the vector of the first image feature of the target video frame. Then, the ratio of the inner product to the product can be taken as the feature similarity between the first video feature and the first image feature of the target video frame, as shown in the following formula:

[0081]

[0082] Wherein, || || is the modulus of the vector, s t is the feature similarity between the first video feature and the first image feature of the tth video frame.

[0083] It should be noted that the above similarity calculation process is only an example, and in actual application, any similarity calculation method in the prior art can be selected according to actual conditions, and the embodiment of the present application does not make specific limitation.

[0084] In step S1042, the second image feature of the target video frame is calculated according to the feature similarity, the first image feature of the target video frame and the first video feature.

[0085] In a specific implementation manner of the embodiment of the present application, the first weight and the second weight can be determined according to the feature similarity respectively.

[0086] Wherein, the first weight is the weight corresponding to the first image feature of the target video frame, and the second weight is the weight corresponding to the first video feature. The first weight is positively correlated with the feature similarity, that is, the greater the feature similarity, the greater the first weight, and vice versa, the smaller the feature similarity, the smaller the first weight; the second weight is negatively correlated with the feature similarity, that is, the greater the feature similarity, the smaller the second weight, and vice versa, the smaller the feature similarity, the greater the second weight. For example, the feature similarity can be determined as the first weight, and the difference between the preset weight sum and the feature similarity can be determined as the second weight, as shown in the following formula:

[0087] w t =s t

[0088] w t ′=w sum -s t

[0089] wherein w sum is a weight sum, and a specific value thereof can be set according to an actual situation, and embodiments of the present application do not make a specific limitation thereon, and herein it is preferred to set it as 1, w t is a first weight, w t ′ is a second weight

[0090] After the first weight and the second weight are determined, the first image feature and the first video feature of the target video frame can be fused according to the first weight and the second weight, so as to obtain a second image feature of the target video frame.

[0091] Specifically, a first weighted feature can be calculated according to the first weight and the first image feature of the target video frame, a second weighted feature can be calculated according to the second weight and the first video feature, and the second image feature of the target video frame can be calculated according to the first weighted feature and the second weighted feature, as shown in the following formula:

[0092] h t = w t F t + w t ′F

[0093] or h t = s t F t + (1-s t )F

[0094] wherein w t F t is the first weighted feature, w t ′F is the second weighted feature, and h t is the second image feature of the tth video frame.

[0095] According to the above process, each video frame in the video sequence is traversed, so as to obtain the second image feature of each video frame.

[0096] In step S105, a second video feature of the video sequence is calculated according to the second image feature of each video frame.

[0097] In a specific implementation manner of embodiments of the present application, the second image feature of each video frame can be processed by averaging, so as to obtain a final video feature (denoted as the second video feature), as shown in the following formula:

[0098]

[0099] wherein h is the second video feature obtained after feature optimization.

[0100] Figure 3The figure shows a complete execution process diagram of one embodiment of a video feature extraction method in the embodiment of the present application. As shown in the figure, the embodiment of the present application first acquires a video sequence to be processed, and respectively performs image feature extraction on each video frame in the video sequence to obtain first image features of each video frame; then performs average processing on the first image features of each video frame to obtain first video features, and respectively calculates feature similarities between the first video features and the first image features of each video frame; then determines weights according to the feature similarities, and performs weighted average on the first video features and the first image features according to the weights to obtain second image features of each video frame; finally, performs average processing on the second image features of each video frame to obtain final video features of the video sequence.

[0101] After obtaining the final video features of the video sequence, video-based face detection and recognition, video-based vehicle recognition, video-based pedestrian detection and recognition, and other visual tasks can be performed based on the video features.

[0102] To sum up, the embodiment of the present application acquires a video sequence to be processed; respectively performs image feature extraction on each video frame in the video sequence to obtain first image features of each video frame; calculates first video features of the video sequence according to the first image features of each video frame; respectively performs feature optimization on the first image features of each video frame according to the first video features to obtain second image features of each video frame; and calculates second video features of the video sequence according to the second image features of each video frame. In the embodiment of the present application, the first image features of each video frame can be optimized through the first video features of the video sequence, thereby effectively weakening the quality influence of poor-quality video frames on the final extracted video features, and improving the robustness of video feature extraction.

[0103] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0104] corresponding to the video feature extraction method described in the above embodiment, Figure 4 An embodiment structure diagram of a video feature extraction device provided by the embodiment of the present application is shown.

[0105] In the embodiment, a video feature extraction device can include:

[0106] The video sequence acquisition module 401 is configured to acquire a video sequence to be processed;

[0107] The feature extraction module 402 is configured to perform image feature extraction on each video frame in the video sequence respectively to obtain first image features of the video frames.

[0108] The first video feature calculation module 403 is configured to calculate first video features of the video sequence according to the first image features of the video frames.

[0109] The feature optimization module 404 is configured to perform feature optimization on the first image features of the video frames respectively according to the first video features to obtain second image features of the video frames.

[0110] The second video feature calculation module 405 is configured to calculate second video features of the video sequence according to the second image features of the video frames.

[0111] In a specific implementation manner of the embodiment of the present application, the feature optimization module can include:

[0112] The similarity calculation sub-module is configured to calculate a feature similarity between the first video features and first image features of a target video frame; wherein the target video frame is any one of the video frames in the video sequence.

[0113] The feature calculation sub-module is configured to calculate second image features of the target video frame according to the feature similarity, the first image features of the target video frame and the first video features.

[0114] In a specific implementation manner of the embodiment of the present application, the feature calculation sub-module can include:

[0115] The weight determination unit is configured to determine a first weight and a second weight respectively according to the feature similarity; wherein the first weight is a weight corresponding to the first image features of the target video frame, the second weight is a weight corresponding to the first video features, the first weight is positively correlated with the feature similarity, and the second weight is negatively correlated with the feature similarity.

[0116] The feature fusion unit is configured to perform feature fusion on the first image features of the target video frame and the first video features according to the first weight and the second weight to obtain the second image features of the target video frame.

[0117] In a specific implementation manner of the embodiment of the present application, the weight determination unit can be specifically configured to determine the feature similarity as the first weight, and determine a difference between a preset weight sum and the feature similarity as the second weight.

[0118] In a specific implementation process of the embodiment of the present application, the feature fusion unit can be specifically configured to: calculate a first weighted feature according to the first weight and the first image feature of the target video frame; calculate a second weighted feature according to the second weight and the first video feature; and calculate the second image feature of the target video frame according to the first weighted feature and the second weighted feature.

[0119] In a specific implementation process of the embodiment of the present application, the first video feature calculation module can be specifically configured to: perform average processing on the first image features of the video frames to obtain the first video feature.

[0120] In a specific implementation process of the embodiment of the present application, the second video feature calculation module can be specifically configured to: perform average processing on the second image features of the video frames to obtain the second video feature.

[0121] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described apparatuses, modules and units can refer to the corresponding processes in the foregoing method embodiments, which will not be described herein.

[0122] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in a certain embodiment can be referred to the related description of other embodiments.

[0123] Figure 5 A schematic block diagram of a terminal device provided by an embodiment of the present application is shown, and only the parts related to the embodiments of the present application are shown for the convenience of description.

[0124] As shown in Figure 5 , the terminal device 5 of this embodiment includes a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50. The processor 50 implements the steps in each of the video feature extraction method embodiments described above when executing the computer program 52, such as Figure 1 the steps S101 to S105 shown. Alternatively, the processor 50 implements the functions of each module / unit in the above-described device embodiments when executing the computer program 52, such as Figure 4 the functions of the modules 401 to 405 shown.

[0125] For example, the computer program 52 can be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 52 in the terminal device 5.

[0126] The terminal device 5 can be a mobile phone, a tablet computer, a desktop computer, a notebook computer, a palm computer, a robot, and the like. Those skilled in the art can understand that the terminal device 5 can be any device with a communication function, and the terminal device 5 can also be a device without a communication function. Figure 5 The terminal device 5 is only an example and does not constitute a limitation on the terminal device 5, and can include more or fewer components than shown, or combine certain components, or different components, for example, the terminal device 5 can also include an input / output device, a network access device, a bus, and the like.

[0127] The processor 50 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0128] The memory 51 can be an internal storage unit of the terminal device 5, such as a hard disk or a memory of the terminal device 5. The memory 51 can also be an external storage device of the terminal device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 51 can include both the internal storage unit and the external storage device of the terminal device 5. The memory 51 is used to store the computer program and other programs and data required by the terminal device 5. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0129] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be realized in the form of hardware or software. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0130] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0131] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0132] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / terminal device and method can be implemented by other ways. For example, the above-described apparatus / terminal device embodiments are only schematic, and the division of the modules or units is only a logical function division, and there can be another division way in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0133] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0134] In addition, each of the function units in each of the embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0135] The integrated module / unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be implemented by a computer program instructing related hardware to complete, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer readable storage medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable storage medium does not include electric carrier signals and telecommunication signals.

[0136] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of video feature extraction, characterized by, The method comprises the following steps: acquiring a video sequence to be processed; extracting image features of each video frame in the video sequence respectively to obtain first image features of the each video frame; calculating first video features of the video sequence according to the first image features of the each video frame; calculating a feature similarity between the first video features and first image features of a target video frame, wherein the target video frame is any one of the video frames in the video sequence; determining a first weight and a second weight respectively according to the feature similarity; wherein the first weight is a weight corresponding to the first image features of the target video frame, the second weight is a weight corresponding to the first video features, the first weight is positively correlated with the feature similarity, and the second weight is negatively correlated with the feature similarity; performing feature fusion on the first image features of the target video frame and the first video features according to the first weight and the second weight to obtain second image features of the target video frame; calculating second video features of the video sequence according to the second image features of the each video frame.

2. The video feature extraction method of claim 1, wherein, The method further comprises the following steps: determining the feature similarity as the first weight; determining a difference between a preset weight sum and the feature similarity as the second weight.

3. The video feature extraction method of claim 1, wherein, The method further comprises the following steps: calculating first weighted features according to the first weight and the first image features of the target video frame; calculating second weighted features according to the second weight and the first video features; calculating the second image features of the target video frame according to the first weighted features and the second weighted features.

4. The video feature extraction method of any one of claims 1 to 3, characterized in that, The method further comprises the following steps: performing average processing on the first image features of the each video frame to obtain the first video features.

5. The method of claim 1 to 3, wherein, The method further comprises the following steps: performing average processing on the second image features of the each video frame to obtain the second video features.

6. A video feature extraction apparatus characterized by comprising: The method comprises the following steps: a video sequence acquisition module, configured to acquire a video sequence to be processed; a feature extraction module, configured to extract image features of each video frame in the video sequence respectively to obtain first image features of the each video frame; a first video feature calculation module, configured to calculate first video features of the video sequence according to the first image features of the each video frame; The feature optimization module is configured to calculate a feature similarity between the first video feature and a first image feature of a target video frame, wherein the target video frame is any one of the video frames in the video sequence; determine a first weight and a second weight according to the feature similarity, wherein the first weight is a weight corresponding to the first image feature of the target video frame, the second weight is a weight corresponding to the first video feature, the first weight is positively correlated with the feature similarity, and the second weight is negatively correlated with the feature similarity; and perform feature fusion on the first image feature of the target video frame and the first video feature according to the first weight and the second weight to obtain a second image feature of the target video frame. The second video feature calculation module is configured to calculate a second video feature of the video sequence according to the second image features of the video frames.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: receiving a request for a resource from a client; determining whether the client is authorized to access the resource; and if the client is authorized to access the resource, providing the resource to the client. The computer program, when executed by a processor, implements the steps of the video feature extraction method according to any one of claims 1 to 5.

8. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor, when executing the computer program, implements the steps of the video feature extraction method according to any one of claims 1 to 5.

Images