Video feature compression method, device, equipment, medium and product

Abstract

The invention discloses a video feature compression method, device and equipment, a medium and a product. The method comprises the following steps: acquiring a to-be-compressed video, and sampling the to-be-compressed video to obtain a video frame set; establishing a video clip set according to the video frame set, and encoding and compressing each video clip to obtain a target video feature set corresponding to each video clip; and decoding the target video feature set corresponding to each video clip to obtain compressed video features. According to the technical scheme, redundancy of visual information in the long video can be utilized, the context of the long video can be compressed from the fragment level to the video level in a progressive compression mode, necessary details are reserved, and meanwhile the calculation amount is greatly reduced.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of video processing technology, and in particular to a video feature compression method, device, equipment, medium and product. Background Art

[0002] At present, the work of using video language models to understand the content of long videos consists of a visual encoder and a text decoder. The visual encoder processes the original video content to obtain video features, and the text decoder gives answers based on the video features and the questions and instructions provided by the user. With the popularity of large language models in the past two years, current video language models usually use large language models as text decoders. For long videos, the length of the video features encoded by the visual encoder is very long, and it is difficult for the text decoder to fully understand its content. At the same time, the computational efficiency is also very low. In order to address this problem, the existing technical solutions mainly focus on expanding the context length of the text decoder, that is, the large language model (the upper limit of the content length that the large language model can handle) and compressing video features.

[0003] The existing technical solution is to use various feature aggregation methods to reduce the length of video features before inputting into the text decoder, thereby reducing the pressure on the text decoder to process and understand the video features. The disadvantage of the current implementation is that the use of feature aggregation to reduce video features will greatly lose key information in the video, resulting in a large degree of information loss in the video features entering the text decoder, thereby preventing the text decoder from fully and accurately understanding the video content. Summary of the invention

[0004] The embodiments of the present invention provide a video feature compression method, apparatus, device, medium and product, so as to achieve the ability to compress the long video context from the segment level to the video level in a progressive compression manner, thereby significantly reducing the amount of calculation while retaining the necessary details.

[0005] According to one aspect of the present invention, a video feature compression method is provided, comprising:

[0006] Acquire a video to be compressed, and sample the video to be compressed to obtain a video frame set;

[0007] A video segment set is formed according to the video frame set, and each video segment is encoded and compressed to obtain a target video feature set corresponding to each video segment;

[0008] The target video feature set corresponding to each of the video clips is decoded to obtain compressed video features.

[0009] According to another aspect of the present invention, a video feature compression device is provided, the device comprising:

[0010] An acquisition and sampling module, used for acquiring a video to be compressed, and sampling the video to be compressed to obtain a video frame set;

[0011] An encoding and compression module, used to form a video segment set according to the video frame set, and encode and compress each video segment to obtain a target video feature set corresponding to each video segment;

[0012] The decoding module is used to decode the target video feature set corresponding to each of the video clips to obtain compressed video features.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the video feature compression method described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the video feature compression method described in any embodiment of the present invention when executed.

[0018] According to another aspect of the present invention, an embodiment of the present invention further provides a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, the video feature compression method described in any embodiment of the present invention is implemented.

[0019] The embodiment of the present invention obtains a video to be compressed, samples the video to be compressed, obtains a video frame set, forms a video segment set according to the video frame set, encodes and compresses each video segment, obtains a target video feature set corresponding to each video segment, and decodes the target video feature set corresponding to each video segment to obtain a compressed video feature. The technical solution of the present invention can utilize the redundancy of visual information in a long video, compress the long video context from the segment level to the video level in a progressive compression method, and greatly reduce the amount of calculation while retaining necessary details.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 is a flow chart of a video feature compression method in an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of a video feature compression method in an embodiment of the present invention;

[0024] Figure 3 is a structural schematic diagram of a video feature compression device in an embodiment of the present invention;

[0025] Figure 4 It is a structural schematic diagram of an electronic device for implementing the video feature compression method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and their meanings are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0029] Embodiment 1

[0030] Figure 1 is a flow chart of a video feature compression method in an embodiment of the present invention. This embodiment is applicable to the case of video feature compression. The method can be executed by a video feature compression device in an embodiment of the present invention. The device can be implemented in software and / or hardware. Figure 1 As shown, the method specifically comprises the following steps:

[0031] S101: Obtain a video to be compressed, and sample the video to be compressed to obtain a video frame set.

[0032] In this embodiment, the videos to be compressed may be videos of different time lengths. In this embodiment, the videos to be compressed may be divided into different time length levels, and different sampling frequencies may be set for sampling for videos of different time lengths. The video frame set may be a set consisting of all video frames obtained after sampling the videos to be compressed.

[0033] Specifically, for videos of different lengths, this embodiment uses a dynamic video sampling method to ensure the integrity of the sampling. For example, for short videos, dense sampling of 15 frames / second can be used to ensure that dynamic information is sampled; for long videos, sparse sampling of 1 frame / second can be used to reduce inter-frame redundancy. Based on videos of different lengths, different sampling frequencies are used for sampling, and finally all video frames are obtained to form a video frame set.

[0034] S102: assemble a video segment set according to the video frame set, and encode and compress each video segment to obtain a target video feature set corresponding to each video segment.

[0035] It should be noted that the video segment set includes several video segments, and each video segment may be composed of several sampled video frames.

[0036] In this embodiment, the target video feature set may be a set consisting of a plurality of video segment features corresponding to each video segment obtained after performing segment-level compression on the video to be compressed.

[0037] Specifically, for each video frame obtained by sampling the video to be compressed, video segments can be formed by grouping T video frames into a set of video segments. Then, for each video segment, it can be first encoded into M video tokens (i.e., video features) using a video encoder, and then the M video tokens can be compressed into N, where N < M, using a learnable feature compressor to obtain the target video feature set corresponding to each video segment, thus achieving the compression effect.

[0038] S103. Decode the target video feature set corresponding to each video segment to obtain the compressed video features.

[0039] It should be noted that the compressed video features can be the video features obtained after performing segment-level compression and video-level compression on the video to be compressed.

[0040] Specifically, the compressed video segment features can be integrated into a video-level feature and then input into a text decoder for decoding, thereby compressing the number of video-level features to obtain the video-level features.

[0041] In the embodiments of the present invention, by obtaining the video to be compressed, sampling the video to be compressed to obtain a set of video frames, forming a set of video segments according to the set of video frames, encoding and compressing each video segment to obtain the target video feature set corresponding to each video segment, and decoding the target video feature set corresponding to each video segment to obtain the compressed video features. Through the technical solution of the present invention, it is possible to utilize the redundancy of visual information in long videos and compress the long video context from the segment level to the video level in a progressive compression manner, while retaining necessary details and significantly reducing the computational amount.

[0042] Optionally, obtaining the video to be compressed and sampling the video to be compressed to obtain a set of video frames includes:

[0043] Obtain the video to be compressed.

[0044] If the video length of the video to be compressed is less than a preset threshold, sample the video to be compressed based on the first sampling frequency to obtain a set of video frames.

[0045] Among them, the preset threshold can be a video time length threshold preset according to actual requirements or empirical values. If the time length of the video to be compressed exceeds this threshold, the video to be compressed can be considered a long video; if the time length of the video to be compressed does not exceed this threshold, the video to be compressed can be considered a short video. This embodiment does not specifically limit the preset threshold. Exemplarily, the preset threshold can be, for example, 1 hour.

[0046] The first sampling frequency may be a sampling frequency for short videos pre-set according to actual needs or experience values, and this embodiment does not specifically limit the first sampling frequency. Exemplarily, the first sampling frequency may be, for example, 15 frames per second.

[0047] Specifically, for videos of different lengths, this embodiment uses a dynamic video sampling method to ensure the integrity of the sampling. For short videos, that is, videos to be compressed whose length is less than a preset threshold, 15 frames / second dense sampling can be used to ensure that dynamic information is sampled.

[0048] If the video length of the video to be compressed is greater than a preset threshold, the video to be compressed is sampled based on a second sampling frequency to obtain a video frame set.

[0049] The second sampling frequency may be a sampling frequency for long videos pre-set according to actual needs or experience values, and this embodiment does not specifically limit the second sampling frequency. Exemplarily, the second sampling frequency may be, for example, 1 frame / second.

[0050] Specifically, for a long video, that is, a video to be compressed whose time length is greater than a preset threshold, sparse sampling of 1 frame / second can be used to reduce inter-frame redundancy.

[0051] Optionally, a video segment set is formed according to the video frame set, and each video segment is encoded and compressed to obtain a target video feature set corresponding to each video segment, including:

[0052] According to the time sequence, a target number of video frames are selected from the video frame set in sequence to form video segments, thereby obtaining a video segment set.

[0053] It should be noted that the time sequence may be the time sequence in which each video frame appears in the video to be compressed. For example, if video frame A appears at 10 minutes and 56 seconds in the video to be compressed, video frame B appears at 10 minutes and 57 seconds in the video to be compressed, and video frame C appears at 10 minutes and 58 seconds in the video to be compressed, then the order of video frame A, video frame B, and video frame C in forming the video segment is: video frame A, video frame B, and video frame C.

[0054] The target number may be the number of video frames constituting a video clip that is preset according to actual needs or empirical values. This embodiment does not specifically limit this value and it can be represented by T.

[0055] Specifically, after sampling each video frame, T video frames may be selected from the video frame set in chronological order as a group to form a video segment, thereby dividing the video to be compressed into multiple short segments.

[0056] For each video segment, it is encoded by a video encoder to obtain an initial video feature set corresponding to each video segment.

[0057] It can be known that a video encoder is a hardware or software device that compresses and converts video signals into a specific format. The main purpose of a video encoder is to reduce the size of video data while maintaining the video quality as much as possible for easy storage and transmission. In this embodiment, the video encoder can encode the video segment into video tokens, that is, obtain video features. This embodiment does not limit the type and parameters of the video encoder, etc.

[0058] It should be noted that the initial video feature set can be a set composed of several video features obtained by encoding the video segment using a video encoder.

[0059] Specifically, for each video segment, it can first be encoded into M video tokens using a video encoder, that is, the initial video feature set corresponding to each video segment is composed of M video features.

[0060] The initial video feature set corresponding to each video segment is compressed by a feature compressor to obtain a target video feature set corresponding to each video segment.

[0061] It can be known that a feature compressor is a tool used in the fields of machine learning and deep learning to reduce the feature dimension, aiming to reduce the complexity and computational cost of the model while retaining important feature information as much as possible. In this embodiment, the feature compressor can be used to compress the initial video feature set corresponding to each video segment. This embodiment does not limit the type and parameters of the feature compressor, etc.

[0062] Specifically, a learnable feature compressor can be used to compress the M video tokens corresponding to each video segment into N, where N < M, so as to achieve the compression effect.

[0063] Optionally, the target video feature set corresponding to each video segment is decoded to obtain the compressed video features, including:

[0064] The target video feature sets corresponding to each video segment are concatenated to obtain a first video feature set.

[0065] Among them, the first video feature set can be a video feature set obtained by concatenating the target video feature sets corresponding to each video segment.

[0066] In this embodiment, it is assumed that the video to be compressed is divided into V short segments, that is, the video frame set is composed of V video segments, and the target video feature set corresponding to each video segment contains N video features in total. Then, the target video feature set corresponding to each video segment is spliced, that is, the compressed video tokens are spliced ​​together, and finally V×N video tokens are obtained, that is, the first video feature set is obtained.

[0067] The first video feature set is input into a text decoder for decoding to obtain compressed video features.

[0068] In this embodiment, the text decoder may be an LLM (Large Language Model). In this embodiment, the first video feature set may be compressed at the video level to obtain compressed video features.

[0069] Specifically, the compressed video tokens are pieced together to obtain V×N video tokens, which are then input into a text decoder for video-level compression to obtain compressed video features.

[0070] Optionally, the first video feature set is input into a text decoder for decoding to obtain compressed video features, including:

[0071] The first video feature set is input into a text decoder for screening to obtain a second video feature set.

[0072] It should be noted that the second video feature set may be a set consisting of remaining video tokens after the first video feature set is screened by a text decoder and a small number of video tokens are randomly discarded.

[0073] Specifically, after the first video feature set is input into the text decoder, the present embodiment proposes a video-level feature compression scheme to further compress these video tokens. In the actual operation process, a small number of video tokens are randomly discarded in the first few layers of the text decoder (this number of layers can be a hyperparameter, which can be randomly generated, or can be pre-set according to actual conditions, or can be set according to empirical values, and this embodiment does not limit this). Exemplarily, when randomly discarding, the proportion and content of each layer discarded in the first few layers can be different. For example, the first layer randomly discards 10% of the video tokens, the second layer randomly discards 20% of the video tokens, etc. This embodiment does not limit this.

[0074] A similarity calculation is performed between the second video feature set and the text features corresponding to the video to be compressed, and the compressed video features are obtained according to the similarity calculation result.

[0075] In this embodiment, the text feature can be obtained by performing text recognition on the video to be compressed.

[0076] Specifically, in the latter layers of the text decoder (this number of layers can be obtained by subtracting the first few layers in which the video tokens are randomly discarded from the total number of layers of the text decoder, that is, if the total number of layers of the text decoder is L, and the number of the first few layers in which the video tokens are randomly discarded is l1, then the number of the latter layers of the text decoder is l2: l2=L-l1), each video token and text feature are dot-multiplied to calculate the similarity, and tokens with low similarity are discarded, thereby compressing the number of tokens at the video level.

[0077] Optionally, performing similarity calculation on the second video feature set and the text features corresponding to the video to be compressed, and obtaining the compressed video features according to the similarity calculation result, includes:

[0078] Based on the time information, similarity calculation is performed between the second video feature set and the text feature corresponding to the video to be compressed, and a similarity calculation result corresponding to each video feature in the second video feature set is obtained.

[0079] It should be noted that when calculating the similarity between each video feature in the second video feature set and the text feature corresponding to the video to be compressed, the time of the two needs to be aligned. For example, the video feature of the starting video frame of the video to be compressed is aligned with the starting text feature for similarity calculation, and the video feature of the ending video frame of the video to be compressed is aligned with the ending text feature for similarity calculation.

[0080] The video feature whose similarity calculation result is greater than a preset score is determined as a compressed video feature.

[0081] The preset score may be a similarity score preset according to actual needs or experience, and this embodiment does not limit the specific value of the preset score.

[0082] Exemplarily, after calculating the similarity between each video feature in the second video feature set and the text feature corresponding to the video to be compressed, the similarity score obtained is between 0 and 1. Assuming the preset score is 0.7, the video features with a similarity calculation result less than 0.7 are discarded, and the video features with a similarity calculation result greater than 0.7 are retained to obtain the compressed video features.

[0083] As an exemplary description of an embodiment of the present invention, Figure 2 FIG. 1 is a schematic diagram of a video feature compression method in an embodiment of the present invention. Figure 2 As shown, the process of the video feature compression method can be described as follows:

[0084] Input stage: For videos of different time lengths, different sampling frequencies are used for sampling to obtain a set of video frames, and then video segments are formed by grouping T video frames as a group, obtaining video segments 1... video segment v as the input.

[0085] Video encoding stage: For each video segment, a video encoder is first used to encode it into M video tokens.

[0086] Segment-level feature compression stage: A learnable feature compressor is used to compress M video tokens into N, where N < M, so as to achieve the compression effect. Then the compressed video tokens are put together to obtain V × N video tokens.

[0087] Video-level feature compression stage: After entering the text decoder, a video-level feature compression scheme is proposed to further compress these video tokens. In the first few layers of the text decoder, a small number of video tokens are randomly discarded, while in the later layers of the text decoder, the similarity between each video token and the text feature is calculated by dot product, and the tokens with low similarity degree are discarded, thus compressing the number of tokens at the video level.

[0088] Output stage: Finally, the corresponding answer to the input is generated based on the compressed video features and output.

[0089] The embodiment of the present invention utilizes the redundancy of visual information in long videos and compresses the long video context from the segment level to the video level in a progressive compression manner, significantly reducing the amount of computation while retaining necessary details. The specific compression method is divided into two steps: 1. Segment-level compression; the input video feature segments are compressed to a lower dimension through a learnable compressor. 2. Video-level compression; the compressed video segment features are integrated into a video-level feature, and then by calculating the correlation between the video features and the text features, the video-level features are gradually compressed between different layers of the text decoder. The embodiment of the present invention significantly reduces the training and inference costs of the video understanding model, reduces the amount of computation and the video memory occupancy rate by proposing a more efficient lossless video feature compression technology.

[0090] Embodiment 2

[0091] Figure 3 It is a schematic structural diagram of a video feature compression device in the embodiment of the present invention. This embodiment is applicable to the situation of video feature compression. The device can be implemented in software and / or hardware, and the device can be integrated in any device that provides the function of video feature compression, such as Figure 3 As shown, the video feature compression device specifically includes: an acquisition and sampling module 201, an encoding and compression module 202, and a decoding module 203.

[0092] The acquisition and sampling module 201 is used to acquire the video to be compressed and sample the video to be compressed to obtain a video frame set;

[0093] The encoding and compression module 202 is used to form a video segment set according to the video frame set, and encode and compress each video segment to obtain a target video feature set corresponding to each video segment;

[0094] The decoding module 203 is used to decode the target video feature set corresponding to each of the video clips to obtain compressed video features.

[0095] Optionally, the encoding and compression module 202 is specifically used for:

[0096] Selecting a target number of video frames from the video frame set in sequence according to time sequence to form video segments, thereby obtaining a video segment set;

[0097] Encoding each video clip by a video encoder to obtain an initial video feature set corresponding to each video clip;

[0098] The initial video feature set corresponding to each of the video clips is compressed by a feature compressor to obtain a target video feature set corresponding to each of the video clips.

[0099] Optionally, the decoding module 203 includes:

[0100] A splicing unit, configured to splice the target video feature set corresponding to each of the video clips to obtain a first video feature set;

[0101] The decoding unit is used to input the first video feature set into a text decoder for decoding to obtain compressed video features.

[0102] Optionally, the decoding unit includes:

[0103] A screening subunit, configured to input the first video feature set into a text decoder for screening to obtain a second video feature set;

[0104] The similarity calculation subunit is used to calculate the similarity between the second video feature set and the text features corresponding to the video to be compressed, and obtain the compressed video features according to the similarity calculation result.

[0105] Optionally, the similarity calculation subunit is specifically used for:

[0106] Based on the time information, similarity calculation is performed between the second video feature set and the text feature corresponding to the video to be compressed to obtain a similarity calculation result corresponding to each video feature in the second video feature set;

[0107] The video feature whose similarity calculation result is greater than a preset score is determined as a compressed video feature.

[0108] Optionally, the acquisition and sampling module 201 is specifically used for:

[0109] Get the video to be compressed;

[0110] If the video length of the video to be compressed is less than a preset threshold, sampling the video to be compressed based on a first sampling frequency to obtain a video frame set;

[0111] If the video length of the video to be compressed is greater than a preset threshold, the video to be compressed is sampled based on a second sampling frequency to obtain a video frame set.

[0112] The above-mentioned product can execute the video feature compression method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0113] Embodiment 3

[0114] Figure 4 A schematic diagram of the structure of an electronic device 30 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0115] like Figure 4 As shown, the electronic device 30 includes at least one processor 31, and a memory connected to the at least one processor 31, such as a read-only memory (ROM) 32, a random access memory (RAM) 33, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 31 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 32 or the computer program loaded from the storage unit 38 to the random access memory (RAM) 33. In the RAM 33, various programs and data required for the operation of the electronic device 30 can also be stored. The processor 31, the ROM 32, and the RAM 33 are connected to each other through a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0116] A number of components in the electronic device 30 are connected to the I / O interface 35, including: an input unit 36, such as a keyboard, a mouse, etc.; an output unit 37, such as various types of displays, speakers, etc.; a storage unit 38, such as a disk, an optical disk, etc.; and a communication unit 39, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 39 allows the electronic device 30 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0117] The processor 31 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 31 performs the various methods and processes described above, such as the video feature compression method:

[0118] Acquire a video to be compressed, and sample the video to be compressed to obtain a video frame set;

[0119] A video segment set is formed according to the video frame set, and each video segment is encoded and compressed to obtain a target video feature set corresponding to each video segment;

[0120] The target video feature set corresponding to each of the video clips is decoded to obtain compressed video features.

[0121] In some embodiments, the video feature compression method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 30 via the ROM 32 and / or the communication unit 39. When the computer program is loaded into the RAM 33 and executed by the processor 31, one or more steps of the video feature compression method described above may be performed. Alternatively, in other embodiments, the processor 31 may be configured to perform the video feature compression method in any other appropriate manner (e.g., by means of firmware).

[0122] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0124] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0126] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0127] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0128] In one embodiment, the present invention further includes a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, the video feature compression method of any embodiment of the present invention is implemented.

[0129] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0130] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0131] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A video feature compression method, characterized in that: include: Acquire a video to be compressed, and sample the video to be compressed to obtain a video frame set; A video segment set is formed according to the video frame set, and each video segment is encoded and compressed to obtain a target video feature set corresponding to each video segment; The target video feature set corresponding to each of the video clips is decoded to obtain compressed video features.

2. The method according to claim 1, characterized in that A video segment set is formed according to the video frame set, and each video segment is encoded and compressed to obtain a target video feature set corresponding to each video segment, including: Selecting a target number of video frames from the video frame set in sequence according to time sequence to form video segments, thereby obtaining a video segment set; Encoding each video clip by a video encoder to obtain an initial video feature set corresponding to each video clip; The initial video feature set corresponding to each of the video clips is compressed by a feature compressor to obtain a target video feature set corresponding to each of the video clips.

3. The method according to claim 1, characterized in that: Decoding the target video feature set corresponding to each of the video clips to obtain compressed video features, including: splicing the target video feature sets corresponding to each of the video clips to obtain a first video feature set; The first video feature set is input into a text decoder for decoding to obtain compressed video features.

4. The method according to claim 3, characterized in that The first video feature set is input into a text decoder for decoding to obtain compressed video features, including: Inputting the first video feature set into a text decoder for screening to obtain a second video feature set; The second video feature set is subjected to similarity calculation with the text features corresponding to the video to be compressed, and compressed video features are obtained according to the similarity calculation result.

5. The method according to claim 4, characterized in that The second video feature set is subjected to similarity calculation with the text feature corresponding to the video to be compressed, and the compressed video feature is obtained according to the similarity calculation result, including: Based on the time information, similarity calculation is performed between the second video feature set and the text feature corresponding to the video to be compressed to obtain a similarity calculation result corresponding to each video feature in the second video feature set; The video feature whose similarity calculation result is greater than a preset score is determined as a compressed video feature.

6. The method according to claim 1, characterized in that Obtain a video to be compressed, and sample the video to be compressed to obtain a video frame set, including: Get the video to be compressed; If the video length of the video to be compressed is less than a preset threshold, sampling the video to be compressed based on a first sampling frequency to obtain a video frame set; If the video length of the video to be compressed is greater than a preset threshold, the video to be compressed is sampled based on a second sampling frequency to obtain a video frame set.

7. A video feature compression device, characterized in that: include: An acquisition and sampling module, used for acquiring a video to be compressed, and sampling the video to be compressed to obtain a video frame set; An encoding and compression module, used to form a video segment set according to the video frame set, and encode and compress each video segment to obtain a target video feature set corresponding to each video segment; The decoding module is used to decode the target video feature set corresponding to each of the video clips to obtain compressed video features.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the video feature compression method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the video feature compression method according to any one of claims 1 to 6 when executed.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the video feature compression method according to any one of claims 1 to 6.

Images