Video feature extraction method and apparatus, readable storage medium, and terminal device

Abstract

The present application relates to the technical field of target recognition, and in particular to a video feature extraction method and apparatus, a computer-readable storage medium, and a terminal device. The method comprises: acquiring a video sequence to be processed; respectively performing image feature extraction on video frames in the video sequence to obtain first image features of the video frames; on the basis of the first image features of the video frames, calculating a first video feature of the video sequence; on the basis of the first video feature, respectively performing feature optimization on the first image features of the video frames to obtain second image features of the video frames; and on the basis of the second image features of the video frames, calculating a second video feature of the video sequence. The present application can effectively weaken the impact of video frames of poor quality on the quality of finally extracted video features, thereby improving the robustness of video feature extraction.

Description

Video feature extraction method, device, readable storage medium and terminal device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on July 24, 2023, with application number 202310917755.7 and invention name “A video feature extraction method, device, readable storage medium and terminal device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application belongs to the field of target recognition technology, and in particular relates to a video feature extraction method, apparatus, computer-readable storage medium, and terminal device. Background Art

[0003] Video feature extraction refers to extracting feature information from video sequences so that the feature information can be used to identify the video sequence. With the development of artificial intelligence technology, various types of videos are becoming more and more abundant, and how to accurately extract video features has become particularly important.

[0004] In existing technologies, video features are typically calculated using an average approach, meaning that each video frame contributes equally to the final extracted video features. If a video sequence contains poor quality frames, such as those with blurred or overexposed images, this will affect the quality of the extracted video features and reduce robustness. Technical issues

[0005] In view of this, embodiments of the present application provide a video feature extraction method, apparatus, computer-readable storage medium, and terminal device to solve the problem of poor robustness of existing video feature extraction methods. Technical Solutions

[0006] A first aspect of the embodiments of the present application provides a video feature extraction method, which may include:

[0007] Get the video sequence to be processed;

[0008] Performing image feature extraction on each video frame in the video sequence to obtain a first image feature of each video frame;

[0009] Calculating a first video feature of the video sequence according to the first image features of each video frame;

[0010] Performing feature optimization on the first image features of each video frame according to the first video feature to obtain a second image feature of each video frame;

[0011] A second video feature of the video sequence is calculated according to the second image features of the respective video frames.

[0012] In a specific implementation of the first aspect, the performing feature optimization on the first image features of the respective video frames according to the first video features to obtain the second image features of the respective video frames may include:

[0013] Calculating a feature similarity between the first video feature and a first image feature of a target video frame; wherein the target video frame is any video frame in the video sequence;

[0014] A second image feature of the target video frame is calculated according to the feature similarity, the first image feature of the target video frame, and the first video feature.

[0015] In a specific implementation of the first aspect, calculating the second image feature of the target video frame based on the feature similarity, the first image feature of the target video frame, and the first video feature may include:

[0016] Determining a first weight and a second weight respectively according to the feature similarity; wherein the first weight is a weight corresponding to the first image feature of the target video frame, the second weight is a weight corresponding to the first video feature, the first weight is positively correlated with the feature similarity, and the second weight is negatively correlated with the feature similarity;

[0017] According to the first weight and the second weight, feature fusion is performed on the first image feature and the first video feature of the target video frame to obtain the second image feature of the target video frame.

[0018] In a specific implementation of the first aspect, determining the first weight and the second weight according to the feature similarity may include:

[0019] Determining the feature similarity as the first weight;

[0020] The difference between the preset weight sum and the feature similarity is determined as the second weight.

[0021] In a specific implementation of the first aspect, performing feature fusion on the first image feature and the first video feature of the target video frame according to the first weight and the second weight to obtain the second image feature of the target video frame may include:

[0022] Calculating a first weighted feature according to the first weight and a first image feature of the target video frame;

[0023] Calculating a second weighted feature according to the second weight and the first video feature;

[0024] A second image feature of the target video frame is calculated according to the first weighted feature and the second weighted feature.

[0025] In a specific implementation of the first aspect, calculating the first video feature of the video sequence according to the first image feature of each video frame may include:

[0026] Averaging processing is performed on the first image features of the respective video frames to obtain the first video features.

[0027] In a specific implementation of the first aspect, calculating the second video feature of the video sequence according to the second image feature of each video frame may include:

[0028] Averaging processing is performed on the second image features of the respective video frames to obtain the second video features.

[0029] A second aspect of the embodiments of the present application provides a video feature extraction device, which may include:

[0030] A video sequence acquisition module is used to acquire the video sequence to be processed;

[0031] A feature extraction module is used to extract image features from each video frame in the video sequence to obtain a first image feature of each video frame;

[0032] A first video feature calculation module, configured to calculate a first video feature of the video sequence according to the first image feature of each video frame;

[0033] a feature optimization module, configured to perform feature optimization on the first image features of each video frame according to the first video feature to obtain a second image feature of each video frame;

[0034] The second video feature calculation module is configured to calculate the second video feature of the video sequence according to the second image feature of each video frame.

[0035] In a specific implementation of the second aspect, the feature optimization module may include:

[0036] a similarity calculation submodule, configured to calculate a feature similarity between the first video feature and a first image feature of a target video frame; wherein the target video frame is any video frame in the video sequence;

[0037] The feature calculation submodule is configured to calculate a second image feature of the target video frame based on the feature similarity, the first image feature of the target video frame, and the first video feature.

[0038] In a specific implementation of the second aspect, the feature calculation submodule may include:

[0039] a weight determination unit, configured to determine a first weight and a second weight respectively according to the feature similarity; wherein the first weight is a weight corresponding to the first image feature of the target video frame, the second weight is a weight corresponding to the first video feature, the first weight is positively correlated with the feature similarity, and the second weight is negatively correlated with the feature similarity;

[0040] A feature fusion unit is used to perform feature fusion on the first image feature of the target video frame and the first video feature according to the first weight and the second weight to obtain the second image feature of the target video frame.

[0041] In a specific implementation of the second aspect, the weight determination unit may be specifically configured to: determine the feature similarity as the first weight; and determine a difference between a preset weight sum and the feature similarity as the second weight.

[0042] In a specific implementation of the second aspect, the feature fusion unit can be specifically used to: calculate a first weighted feature based on the first weight and the first image feature of the target video frame; calculate a second weighted feature based on the second weight and the first video feature; calculate a second image feature of the target video frame based on the first weighted feature and the second weighted feature.

[0043] In a specific implementation of the second aspect, the first video feature calculation module may be specifically configured to: perform averaging processing on the first image features of the respective video frames to obtain the first video feature.

[0044] In a specific implementation of the second aspect, the second video feature calculation module may be specifically configured to: perform averaging processing on the second image features of the respective video frames to obtain the second video feature.

[0045] A third aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of any one of the above-mentioned video feature extraction methods are implemented.

[0046] The fourth aspect of an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned video feature extraction methods when executing the computer program.

[0047] A fifth aspect of an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the steps of any one of the above-mentioned video feature extraction methods. Beneficial effects

[0048] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the embodiments of the present application obtain a video sequence to be processed; perform image feature extraction on each video frame in the video sequence to obtain a first image feature of each video frame; calculate the first video feature of the video sequence based on the first image feature of each video frame; perform feature optimization on the first image feature of each video frame based on the first video feature to obtain a second image feature of each video frame; calculate the second video feature of the video sequence based on the second image feature of each video frame. In the embodiments of the present application, the initial image features (i.e., first image features) of each video frame can be feature optimized using the initial video features (i.e., first video features) of the video sequence, thereby effectively reducing the quality impact of video frames with poor quality on the quality of the final extracted video features and improving the robustness of video feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0050] FIG1 is a flow chart of an embodiment of a method for extracting video features according to an embodiment of the present application;

[0051] FIG2 is a schematic flow chart of optimizing the first image features of each video frame according to the first video feature;

[0052] FIG3 is a schematic diagram of the complete execution process of an embodiment of a video feature extraction method according to an embodiment of the present application;

[0053] FIG4 is a structural diagram of an embodiment of a video feature extraction device according to an embodiment of the present application;

[0054] FIG5 is a schematic block diagram of a terminal device in an embodiment of the present application. Modes for Carrying Out the Invention

[0055] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0056] It will be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0057] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0058] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0059] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0060] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0061] In existing technologies, video features are typically calculated using an average approach, meaning that each video frame contributes equally to the final extracted video features. If a video sequence contains poor quality frames, such as those with blurred or overexposed images, this will affect the quality of the extracted video features and reduce robustness.

[0062] In view of this, embodiments of the present application provide a video feature extraction method, apparatus, computer-readable storage medium, and terminal device. In these embodiments, the initial image features of each video frame can be optimized using the initial video features of the video sequence, thereby effectively reducing the impact of poor quality video frames on the quality of the ultimately extracted video features and improving the robustness of video feature extraction.

[0063] It should be noted that the execution subject of the method of the present application is a terminal device, which may specifically include but is not limited to mobile phones, tablet computers, desktop computers, notebooks, PDAs, robots and other computing devices.

[0064] Referring to FIG. 1 , an embodiment of a video feature extraction method in an embodiment of the present application may include:

[0065] Step S101: Obtain a video sequence to be processed.

[0066] Wherein, a video sequence is a sequence consisting of at least two video frames. In an embodiment of the present application, the video sequence can be captured by a preset camera capture device. For example, a video can be captured by a surveillance camera installed at a preset position and used as the video sequence.

[0067] After capturing a video sequence, it can be stored in a preset location so that it can be retrieved promptly when video feature extraction is needed. For example, a video sequence can be stored in the memory module of a terminal device. When video feature extraction is needed, the video sequence can be directly retrieved from the memory module of the terminal device. For another example, to achieve secure data backup, the terminal device can periodically upload memory data to a cloud server. The video sequence can be stored in the cloud server. When video feature extraction is needed, a video sequence retrieval request can be sent to the cloud server. The video sequence can be retrieved by parsing the response message returned by the cloud server.

[0068] In order to improve the efficiency of video feature extraction, the collected video sequence can also be subjected to frame extraction, for example, one frame can be extracted from every N frames to obtain a video sequence to be processed, where N is an integer greater than 1.

[0069] Step S102: extract image features from each video frame in the video sequence to obtain a first image feature of each video frame.

[0070] After obtaining the video sequence, the video sequence can be split to obtain each video frame in the video sequence. For each video frame, image feature extraction can be performed, and the image feature information extracted from the video frame is recorded as the first image feature.

[0071] In specific applications, any image feature extraction method in the prior art can be selected according to actual conditions, and the embodiments of the present application do not make specific limitations on this.

[0072] In a specific implementation of the embodiment of the present application, each video frame {I1, I2, I3, ..., I T} respectively perform image feature extraction to obtain the first image features {F1, F2, F3, ..., F T}. Among them, the image feature extraction network is a neural network used to extract image features. t is the tth video frame in the video sequence, F t is the first image feature of the t-th video frame, t is the video frame number in the video sequence, 1≤t≤T, and T is the total number of video frames in the video sequence.

[0073] It should be noted that the embodiment of the present application does not specifically limit the selection of the image feature extraction network and can be set according to actual needs. For example, the image feature extraction network can be any neural network for feature extraction in the prior art, such as a convolutional neural network (CNN) or a recurrent neural network (RNN). Before use, the selected neural network can be pre-trained to obtain the image feature extraction network in the embodiment of the present application, and the image feature extraction network can be directly used when performing subsequent image feature extraction.

[0074] Step S103: Calculate the first video feature of the video sequence according to the first image feature of each video frame.

[0075] In a specific implementation of the embodiment of the present application, the first image features of each video frame may be averaged to obtain an initial video feature (referred to as the first video feature), as shown in the following formula:

[0076] Among them, F is the first video feature.

[0077] Step S104: feature-optimize the first image features of each video frame according to the first video features to obtain second image features of each video frame.

[0078] For ease of understanding, here we take any video frame in the video sequence (referred to as the target video frame) as an example to explain in detail the calculation process of the second image feature. As shown in FIG2 , the calculation process of the second image feature of the target video frame may specifically include:

[0079] Step S1041: Calculate the feature similarity between the first video feature and the first image feature of the target video frame.

[0080] In a specific implementation of the embodiment of the present application, the vector inner product between the first video feature and the first image feature of the target video frame, as well as the product of the first vector modulus and the second vector modulus can be calculated respectively, where the first vector modulus is the modulus of the vector of the first video feature, and the second vector modulus is the modulus of the vector of the first image feature of the target video frame. Then, the ratio of the inner product and the product can be used as the feature similarity between the first video feature and the first image feature of the target video frame, as shown in the following formula:

[0081] Among them, |||| is the modulus of the vector, s t is the feature similarity between the first video feature and the first image feature of the t-th video frame.

[0082] It should be noted that the above similarity calculation process is only an example. In actual applications, any similarity calculation method in the prior art can be selected according to actual conditions, and the embodiments of the present application do not specifically limit this.

[0083] Step S1042: Calculate the second image feature of the target video frame according to the feature similarity, the first image feature of the target video frame, and the first video feature.

[0084] In a specific implementation of the embodiment of the present application, the first weight and the second weight may be determined first according to feature similarity.

[0085] Among them, the first weight is the weight corresponding to the first image feature of the target video frame, and the second weight is the weight corresponding to the first video feature. The first weight is positively correlated with the feature similarity, that is, the greater the feature similarity, the greater the first weight, and conversely, the smaller the feature similarity, the smaller the first weight; the second weight is negatively correlated with the feature similarity, that is, the greater the feature similarity, the smaller the second weight, and conversely, the smaller the feature similarity, the greater the second weight. For example, the feature similarity can be determined as the first weight, and the difference between the preset weight sum and the feature similarity can be determined as the second weight, as shown in the following formula: w t =s t w′ t =w sum -s t

[0086] Among them, w sum The specific value of the weight can be set according to the actual situation. The embodiment of the present application does not specifically limit it. Here, it is preferably set to 1. t is the first weight, w′t The second weight

[0087] After the first weight and the second weight are determined, feature fusion may be performed on the first image feature and the first video feature of the target video frame according to the first weight and the second weight, thereby obtaining the second image feature of the target video frame.

[0088] Specifically, a first weighted feature may be calculated according to the first weight and the first image feature of the target video frame, a second weighted feature may be calculated according to the second weight and the first video feature, and a second image feature of the target video frame may be calculated according to the first weighted feature and the second weighted feature, as shown in the following formula:

[0089] h t =w t F t +w′ t F

[0090] or h t =s t F t +(1-s t )F

[0091] Among them, w t F t is the first weighted feature, w′ t F is the second weighted feature, h t is the second image feature of the t-th video frame.

[0092] By traversing each video frame in the video sequence according to the above process, the second image feature of each video frame can be obtained.

[0093] Step S105: Calculate the second video feature of the video sequence according to the second image feature of each video frame.

[0094] In a specific implementation of the embodiment of the present application, the second image features of each video frame may be averaged to obtain a final video feature (referred to as the second video feature), as shown in the following formula:

[0095] Here, h is the second video feature obtained after feature optimization.

[0096] Figure 3 is a schematic diagram of the complete execution process of an embodiment of a video feature extraction method in an embodiment of the present application. As shown in the figure, the embodiment of the present application first obtains a video sequence to be processed, and extracts image features of each video frame in the video sequence to obtain the first image feature of each video frame; then the first image features of each video frame are averaged to obtain the first video feature, and the feature similarity between the first video feature and the first image feature of each video frame is calculated; then the weight is determined according to the feature similarity, and the first video feature and the first image feature are weighted averaged according to the weight to obtain the second image feature of each video frame; finally, the second image features of each video frame are averaged to obtain the final video feature of the video sequence.

[0097] After obtaining the final video features of the video sequence, visual tasks such as video-based face detection and recognition, video-based vehicle recognition, and video-based pedestrian detection and recognition can be performed based on the video features.

[0098] In summary, the embodiment of the present application obtains a video sequence to be processed; performs image feature extraction on each video frame in the video sequence to obtain a first image feature of each video frame; calculates a first video feature of the video sequence based on the first image feature of each video frame; performs feature optimization on the first image feature of each video frame based on the first video feature to obtain a second image feature of each video frame; and calculates a second video feature of the video sequence based on the second image feature of each video frame. In the embodiment of the present application, the first image feature of each video frame can be feature optimized using the first video feature of the video sequence, thereby effectively reducing the impact of poor quality video frames on the quality of the ultimately extracted video features and improving the robustness of video feature extraction.

[0099] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0100] Corresponding to the video feature extraction method described in the above embodiment, FIG4 shows a structural diagram of an embodiment of a video feature extraction device provided in an embodiment of the present application.

[0101] In this embodiment, a video feature extraction device may include:

[0102] The video sequence acquisition module 401 is used to acquire the video sequence to be processed;

[0103] A feature extraction module 402 is configured to extract image features from each video frame in the video sequence to obtain a first image feature of each video frame;

[0104] A first video feature calculation module 403, configured to calculate a first video feature of the video sequence according to the first image feature of each video frame;

[0105] A feature optimization module 404 is configured to optimize the first image feature of each video frame according to the first video feature to obtain a second image feature of each video frame;

[0106] The second video feature calculation module 405 is configured to calculate the second video feature of the video sequence according to the second image feature of each video frame.

[0107] In a specific implementation of the embodiment of the present application, the feature optimization module may include:

[0108] a similarity calculation submodule, configured to calculate a feature similarity between the first video feature and a first image feature of a target video frame; wherein the target video frame is any video frame in the video sequence;

[0109] The feature calculation submodule is configured to calculate a second image feature of the target video frame based on the feature similarity, the first image feature of the target video frame, and the first video feature.

[0110] In a specific implementation of the embodiment of the present application, the feature calculation submodule may include:

[0111] a weight determination unit, configured to determine a first weight and a second weight respectively according to the feature similarity; wherein the first weight is a weight corresponding to the first image feature of the target video frame, the second weight is a weight corresponding to the first video feature, the first weight is positively correlated with the feature similarity, and the second weight is negatively correlated with the feature similarity;

[0112] A feature fusion unit is used to perform feature fusion on the first image feature of the target video frame and the first video feature according to the first weight and the second weight to obtain the second image feature of the target video frame.

[0113] In a specific implementation of the embodiment of the present application, the weight determination unit can be specifically used to: determine the feature similarity as the first weight; and determine the difference between a preset weight sum and the feature similarity as the second weight.

[0114] In a specific implementation of an embodiment of the present application, the feature fusion unit can be specifically used to: calculate a first weighted feature based on the first weight and the first image feature of the target video frame; calculate a second weighted feature based on the second weight and the first video feature; calculate a second image feature of the target video frame based on the first weighted feature and the second weighted feature.

[0115] In a specific implementation of the embodiment of the present application, the first video feature calculation module can be specifically used to: perform averaging processing on the first image features of each video frame to obtain the first video feature.

[0116] In a specific implementation of the embodiment of the present application, the second video feature calculation module can be specifically used to: perform averaging processing on the second image features of each video frame to obtain the second video feature.

[0117] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0118] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0119] FIG5 shows a schematic block diagram of a terminal device provided in an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0120] As shown in FIG5 , the terminal device 5 of this embodiment includes: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50. When the processor 50 executes the computer program 52, the steps in the above-described video feature extraction method embodiments are implemented, such as steps S101 to S105 shown in FIG1 . Alternatively, when the processor 50 executes the computer program 52, the functions of the modules / units in the above-described apparatus embodiments are implemented, such as the functions of modules 401 to 405 shown in FIG4 .

[0121] Exemplarily, the computer program 52 may be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 52 in the terminal device 5.

[0122] The terminal device 5 may be a computing device such as a mobile phone, tablet computer, desktop computer, notebook computer, PDA, robot, etc. Those skilled in the art will appreciate that FIG5 is merely an example of the terminal device 5 and does not limit the terminal device 5 . The terminal device 5 may include more or fewer components than shown in the figure, or may combine certain components or different components. For example, the terminal device 5 may also include input and output devices, network access devices, buses, etc.

[0123] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0124] The memory 51 can be an internal storage unit of the terminal device 5, such as a hard disk or memory of the terminal device 5. The memory 51 can also be an external storage device of the terminal device 5, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the terminal device 5. Furthermore, the memory 51 can also include both an internal storage unit of the terminal device 5 and an external storage device. The memory 51 is used to store the computer program and other programs and data required by the terminal device 5. The memory 51 can also be used to temporarily store data that has been output or is about to be output.

[0125] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0126] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0127] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0128] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0129] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0130] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0131] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0132] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A video feature extraction method, characterized in that: include: Get the video sequence to be processed; Extracting image features from each video frame in the video sequence to obtain first image features of each video frame; Calculating a first video feature of the video sequence according to the first image features of each video frame; According to the first video feature, feature optimization is performed on the first image feature of each video frame to obtain a second image feature of each video frame; The second video feature of the video sequence is calculated according to the second image features of the respective video frames.

2. The video feature extraction method according to claim 1, characterized in that: The step of optimizing the first image features of each video frame according to the first video feature to obtain the second image features of each video frame includes: Calculating a feature similarity between the first video feature and a first image feature of a target video frame; wherein the target video frame is any video frame in the video sequence; A second image feature of the target video frame is calculated according to the feature similarity, the first image feature of the target video frame, and the first video feature.

3. The video feature extraction method according to claim 2, characterized in that: The calculating the second image feature of the target video frame according to the feature similarity, the first image feature of the target video frame and the first video feature includes: Determine a first weight and a second weight respectively according to the feature similarity; wherein the first weight is a weight corresponding to the first image feature of the target video frame, the second weight is a weight corresponding to the first video feature, the first weight is positively correlated with the feature similarity, and the second weight is negatively correlated with the feature similarity; According to the first weight and the second weight, feature fusion is performed on the first image feature of the target video frame and the first video feature to obtain the second image feature of the target video frame.

4. The video feature extraction method according to claim 3, characterized in that: Determining the first weight and the second weight respectively according to the feature similarity includes: Determining the feature similarity as the first weight; The difference between the preset weight sum and the feature similarity is determined as the second weight.

5. The video feature extraction method according to claim 3, characterized in that: The step of fusing the first image feature of the target video frame and the first video feature according to the first weight and the second weight to obtain the second image feature of the target video frame includes: Calculating a first weighted feature according to the first weight and a first image feature of the target video frame; Calculating a second weighted feature according to the second weight and the first video feature; A second image feature of the target video frame is calculated according to the first weighted feature and the second weighted feature.

6. The video feature extraction method according to any one of claims 1 to 5, characterized in that: The calculating the first video feature of the video sequence according to the first image features of each video frame comprises: The first image features of the respective video frames are averaged to obtain the first video features.

7. The video feature extraction method according to any one of claims 1 to 5, characterized in that: The calculating the second video feature of the video sequence according to the second image feature of each video frame comprises: The second image features of the respective video frames are averaged to obtain the second video features.

8. A video feature extraction device, characterized in that: include: A video sequence acquisition module, used to acquire the video sequence to be processed; A feature extraction module, used to extract image features from each video frame in the video sequence to obtain a first image feature of each video frame; A first video feature calculation module, configured to calculate a first video feature of the video sequence according to a first image feature of each video frame; A feature optimization module, configured to optimize the first image features of each video frame according to the first video feature to obtain a second image feature of each video frame; The second video feature calculation module is used to calculate the second video feature of the video sequence according to the second image features of each video frame.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the video feature extraction method according to any one of claims 1 to 7 are implemented.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the video feature extraction method according to any one of claims 1 to 7 are implemented.

Images