A video feature compression method, device, equipment, medium and product

Abstract

The application discloses a video feature compression method, device, equipment, medium and product. The method comprises the following steps: acquiring a to-be-compressed video, sampling the to-be-compressed video to obtain a video frame set; assembling a video segment set according to the video frame set, and encoding and compressing each video segment to obtain a target video feature set corresponding to each video segment; and decoding the target video feature set corresponding to each video segment to obtain a compressed video feature. Through the technical scheme of the application, the redundancy of visual information in a long video can be utilized, and the context of the long video is compressed from the segment level to the video level in a progressive compression mode, so that the calculation amount is greatly reduced while necessary details are reserved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of video processing technology, and in particular to a video feature compression method, apparatus, device, medium and product. Background Technology

[0002] Current methods for understanding long video content using video language models typically consist of a visual encoder and a text decoder. The visual encoder processes the raw video content to obtain video features, while the text decoder provides answers based on these features and user-provided questions and instructions. With the rise of large language models in recent years, current video language models often use them as text decoders. However, for long videos, the video features encoded by the visual encoder are extremely long, making it difficult for the text decoder to fully understand the content, and the computational efficiency is also very low. To address this issue, existing solutions mainly focus on two aspects: increasing the context length of the text decoder (i.e., the large language model – the upper limit of the content length that the large language model can process) and compressing video features.

[0003] Existing solutions involve reducing the length of video features using various feature aggregation methods before inputting them into the text decoder, thereby reducing the burden on the text decoder to process and understand the video features. However, the drawback of current implementations is that using feature aggregation to reduce video features results in a significant loss of key information within the video. This leads to substantial information loss in the video features entering the text decoder, hindering the text decoder from fully and accurately understanding the video content. Summary of the Invention

[0004] This invention provides a video feature compression method, apparatus, device, medium, and product to enable the progressive compression of long video context from the segment level to the video level, significantly reducing computational load while preserving necessary details.

[0005] According to one aspect of the present invention, a video feature compression method is provided, comprising:

[0006] The video to be compressed is obtained, and the video to be compressed is sampled to obtain a set of video frames;

[0007] A video segment set is constructed based on the video frame set, and each video segment is encoded and compressed to obtain a target video feature set corresponding to each video segment;

[0008] The target video feature set corresponding to each video segment is decoded to obtain the compressed video features.

[0009] According to another aspect of the present invention, a video feature compression apparatus is provided, the apparatus comprising:

[0010] The acquisition and sampling module is used to acquire the video to be compressed and to sample the video to be compressed to obtain a set of video frames;

[0011] The encoding and compression module is used to construct a video segment set based on the video frame set, and to encode and compress each video segment to obtain a target video feature set corresponding to each video segment;

[0012] The decoding module is used to decode the target video feature set corresponding to each video segment to obtain compressed video features.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the video feature compression method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the video feature compression method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, embodiments of the present invention also provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the video feature compression method described in any embodiment of the present invention.

[0019] This invention acquires a video to be compressed and samples it to obtain a set of video frames. A set of video segments is then constructed based on the set of video frames. Each video segment is encoded and compressed to obtain a set of target video features corresponding to each video segment. Finally, the set of target video features corresponding to each video segment is decoded to obtain the compressed video features. Through this technical solution, the redundancy of visual information in long videos can be utilized to progressively compress the context of long videos from the segment level to the video level, significantly reducing computational load while preserving necessary details.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a video feature compression method according to an embodiment of the present invention;

[0023] Figure 2 This is a schematic diagram of a video feature compression method according to an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of the structure of a video feature compression device according to an embodiment of the present invention;

[0025] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the video feature compression method of this invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and their derivatives, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0029] Example 1

[0030] Figure 1 This is a flowchart of a video feature compression method according to an embodiment of the present invention. This embodiment is applicable to video feature compression. The method can be executed by the video feature compression device of the present invention, which can be implemented in software and / or hardware, such as... Figure 1 As shown, the method specifically includes the following steps:

[0031] S101. Obtain the video to be compressed and sample it to obtain a set of video frames.

[0032] In this embodiment, the video to be compressed can be a video of different durations. This embodiment can divide the video to be compressed into different duration levels, and different sampling frequencies can be set for each video of different durations. The video frame set can be a collection of all video frames obtained after sampling the video to be compressed.

[0033] Specifically, this embodiment uses a dynamic video sampling method to ensure the integrity of the sampling for videos of different durations. For example, for short videos, dense sampling of 15 frames per second can be used to ensure that dynamic information is sampled; for long videos, sparse sampling of 1 frame per second can be used to reduce inter-frame redundancy. Based on videos of different durations, different sampling frequencies are used for sampling, ultimately resulting in a video frame set composed of all video frames.

[0034] S102. Construct a video segment set based on the video frame set, and encode and compress each video segment to obtain the target video feature set corresponding to each video segment.

[0035] It should be noted that the video clip set includes several video clips, and each video clip can be composed of several sampled video frames.

[0036] In this embodiment, the target video feature set can be a set of several video segment features corresponding to each video segment obtained after segment-level compression of the video to be compressed.

[0037] Specifically, for each video frame obtained by sampling the video to be compressed, video segments can be formed by taking T video frames as a group, obtaining a set of video segments. Then, for each video segment, it can first be encoded into M video tokens (i.e., video features) using a video encoder, and then the M video tokens can be compressed into N, where N < M, using a learnable feature compressor, obtaining the target video feature set corresponding to each video segment, thereby achieving the compression effect.

[0038] S103. Decode the target video feature set corresponding to each video segment to obtain the compressed video features.

[0039] It should be noted that the compressed video features can be the video features obtained after performing segment-level compression and video-level compression on the video to be compressed.

[0040] Specifically, the compressed video segment features can be integrated into a video-level feature, and then input into a text decoder for decoding, thereby compressing the number of features at the video level and obtaining the video-level features.

[0041] In the embodiment of the present invention, by obtaining the video to be compressed, sampling the video to be compressed to obtain a set of video frames, forming a set of video segments according to the set of video frames, encoding and compressing each video segment to obtain the target video feature set corresponding to each video segment, and decoding the target video feature set corresponding to each video segment to obtain the compressed video features. Through the technical solution of the present invention, it is possible to utilize the redundancy of visual information in long videos and compress the long video context from the segment level to the video level in a progressive compression manner, while retaining necessary details and significantly reducing the computational amount.

[0042] Optionally, obtaining the video to be compressed and sampling the video to be compressed to obtain a set of video frames includes:

[0043] Obtain the video to be compressed.

[0044] If the video length of the video to be compressed is less than a preset threshold, sample the video to be compressed based on the first sampling frequency to obtain a set of video frames.

[0045] Among them, the preset threshold can be a video time length threshold preset according to actual needs or empirical values. If the time length of the video to be compressed exceeds this threshold, the video to be compressed can be considered a long video; if the time length of the video to be compressed does not exceed this threshold, the video to be compressed can be considered a short video. This embodiment does not specifically limit the preset threshold. Exemplarily, the preset threshold can be, for example, 1 hour.

[0046] The first sampling frequency can be a sampling frequency for short videos that is preset according to actual needs or empirical values. This embodiment does not specifically limit the first sampling frequency. For example, the first sampling frequency can be 15 frames per second.

[0047] Specifically, for videos of different durations, this embodiment uses a dynamic video sampling method to ensure the integrity of the sampling. For short videos, i.e., videos whose duration is less than a preset threshold, dense sampling of 15 frames per second can be used to ensure that dynamic information is sampled.

[0048] If the length of the video to be compressed is greater than a preset threshold, the video to be compressed is sampled based on the second sampling frequency to obtain a set of video frames.

[0049] The second sampling frequency can be a sampling frequency for long videos that is preset according to actual needs or empirical values. This embodiment does not specifically limit the second sampling frequency. For example, the second sampling frequency can be 1 frame / second.

[0050] Specifically, for long videos, i.e. videos whose duration exceeds a preset threshold, sparse sampling of 1 frame / second can be used to reduce inter-frame redundancy.

[0051] Optionally, a video segment set is constructed based on the video frame set, and each video segment is encoded and compressed to obtain the target video feature set corresponding to each video segment, including:

[0052] In chronological order, a target number of video frames are selected from the video frame set to form video segments, resulting in a video segment set.

[0053] It should be noted that the chronological order can refer to the order in which each video frame appears in the video to be compressed. For example, if video frame A appears at 10 minutes and 56 seconds, video frame B appears at 10 minutes and 57 seconds, and video frame C appears at 10 minutes and 58 seconds, then the order in which video frames A, B, and C are assembled into a video segment is: video frame A, video frame B, and video frame C.

[0054] The target number can be the number of video frames that make up a video segment, which can be preset according to actual needs or experience. In this embodiment, this value is not specifically limited and can be represented by T.

[0055] Specifically, after sampling each video frame, T video frames can be selected sequentially from the video frame set according to time order to form a group and assemble a video segment, thus dividing the video to be compressed into multiple short segments.

[0056] For each video segment, it is encoded by a video encoder to obtain an initial video feature set corresponding to each video segment.

[0057] It can be known that a video encoder is a hardware or software device that compresses and converts video signals into a specific format. The main purpose of a video encoder is to reduce the size of video data while maintaining the video quality as much as possible for easy storage and transmission. In this embodiment, the video encoder can encode a video segment into video tokens, that is, obtain video features. This embodiment does not limit the type and parameters of the video encoder.

[0058] It should be noted that the initial video feature set can be a set composed of several video features obtained by encoding a video segment using a video encoder.

[0059] Specifically, for each video segment, it can first be encoded by a video encoder into M video tokens, that is, the initial video feature set corresponding to each video segment is composed of M video features.

[0060] The initial video feature set corresponding to each video segment is compressed by a feature compressor to obtain a target video feature set corresponding to each video segment.

[0061] It can be known that a feature compressor is a tool used in the fields of machine learning and deep learning to reduce the feature dimension, aiming to reduce the complexity and computational cost of the model while retaining important feature information as much as possible. In this embodiment, the feature compressor can be used to compress the initial video feature set corresponding to each video segment. This embodiment does not limit the type and parameters of the feature compressor.

[0062] Specifically, a learnable feature compressor can be used to compress the M video tokens corresponding to each video segment into N, where N < M, so as to achieve the compression effect.

[0063] Optionally, the target video feature set corresponding to each video segment is decoded to obtain the compressed video features, including:

[0064] The target video feature sets corresponding to each video segment are concatenated to obtain a first video feature set.

[0065] Among them, the first video feature set can be a video feature set obtained by concatenating the target video feature sets corresponding to each video segment.

[0066] In this embodiment, it is assumed that the video to be compressed is divided into V short segments, that is, the set of video frames is assembled into V video segments, and the target video feature set corresponding to each video segment contains N video features. Then, the target video feature sets corresponding to each video segment are concatenated, that is, the compressed video tokens are concatenated together, and finally V×N video tokens are obtained, which is the first video feature set.

[0067] The first set of video features is input into a text decoder for decoding to obtain compressed video features.

[0068] In this embodiment, the text decoder can be an LLM (Large Language Model). In this embodiment, the first video feature set can be compressed at the video level to obtain compressed video features.

[0069] Specifically, the compressed video tokens are concatenated to obtain V×N video tokens, which are then input into a text decoder for video-level compression to obtain the compressed video features.

[0070] Optionally, the first video feature set is input into a text decoder for decoding to obtain compressed video features, including:

[0071] The first video feature set is input into the text decoder for filtering to obtain the second video feature set.

[0072] It should be noted that the second video feature set can be the set of remaining video tokens after the first video feature set has been filtered by a text decoder and a small number of video tokens have been randomly discarded.

[0073] Specifically, after inputting the first set of video features into the text decoder, this embodiment proposes a video-level feature compression scheme to further compress these video tokens. In actual operation, a small number of video tokens are randomly discarded in the first few layers of the text decoder (this number of layers can be a hyperparameter, randomly generated, preset according to actual conditions, or set based on empirical values; this embodiment does not limit this). For example, the proportion and content of the discarded tokens in each of the first few layers can be different; for instance, 10% of the video tokens are randomly discarded in the first layer, 20% are randomly discarded in the second layer, etc. This embodiment does not limit this.

[0074] The similarity between the second set of video features and the text features corresponding to the video to be compressed is calculated, and the compressed video features are obtained based on the similarity calculation results.

[0075] In this embodiment, the text features can be obtained by performing text recognition on the video to be compressed.

[0076] Specifically, in the later layers of the text decoder (this number can be obtained by subtracting the earlier layers where video tokens are randomly discarded from the total number of layers of the text decoder; that is, if the total number of layers of the text decoder is L, and the number of the earlier layers where video tokens are randomly discarded is l1, then the number of the later layers of the text decoder is l2: l2 = L - l1), the similarity between each video token and the text feature is calculated by multiplying them, and tokens with low similarity are discarded, thereby compressing the number of tokens at the video level as well.

[0077] Optionally, the similarity between the second video feature set and the text features corresponding to the video to be compressed is calculated, and the compressed video features are obtained based on the similarity calculation results, including:

[0078] Based on time information, the similarity between the second video feature set and the text features corresponding to the video to be compressed is calculated to obtain the similarity calculation result for each video feature in the second video feature set.

[0079] It should be noted that when calculating the similarity between each video feature in the second video feature set and the corresponding text feature of the video to be compressed, the time of the two needs to be aligned. For example, the video features of the starting video frame of the video to be compressed are aligned with the starting text feature for similarity calculation, and the video features of the ending video frame of the video to be compressed are aligned with the ending text feature for similarity calculation.

[0080] Video features whose similarity calculation results are greater than a preset score are identified as compressed video features.

[0081] The preset score can be a similarity score set in advance based on actual needs or experience. In this embodiment, the specific value of the preset score is not limited.

[0082] For example, after calculating the similarity between each video feature in the second video feature set and the text feature corresponding to the video to be compressed, the similarity score obtained is between 0 and 1. Assuming the preset score is 0.7, video features with similarity calculation results less than 0.7 are discarded, and video features with similarity calculation results greater than 0.7 are retained to obtain the compressed video features.

[0083] As an exemplary description of an embodiment of the present invention Figure 2 This is a schematic diagram of a video feature compression method according to an embodiment of the present invention. Figure 2 As shown, the process of the video feature compression method can be described as follows:

[0084] Input stage: For videos of different time lengths, different sampling frequencies are used for sampling to obtain a set of video frames. Then, video segments are formed by grouping T video frames as a group, resulting in video segments 1... video segment v, which are used as the input.

[0085] Video encoding stage: For each video segment, it is first encoded into M video tokens using a video encoder.

[0086] Segment-level feature compression stage: A learnable feature compressor is used to compress the M video tokens into N, where N < M, so as to achieve the compression effect. After that, the compressed video tokens are concatenated together to obtain V × N video tokens.

[0087] Video-level feature compression stage: After entering the text decoder, a video-level feature compression scheme is proposed to further compress these video tokens. In the first few layers of the text decoder, a small number of video tokens are randomly discarded, while in the later layers of the text decoder, the similarity between each video token and the text feature is calculated by dot product, and the tokens with low similarity degree are discarded, thus compressing the number of tokens at the video level as well.

[0088] Output stage: Finally, the corresponding answer to the input is generated based on the compressed video features and output.

[0089] The embodiment of the present invention utilizes the redundancy of visual information in long videos and compresses the long video context from the segment level to the video level in a progressive compression manner, significantly reducing the computational amount while retaining necessary details. The specific compression method is divided into two steps: 1. Segment-level compression; the input video feature segments are compressed to a lower dimension through a learnable compressor. 2. Video-level compression; the compressed video segment features are integrated into a video-level feature, and then the video-level features are gradually compressed between different layers of the text decoder by calculating the correlation between the video features and the text features. The embodiment of the present invention significantly reduces the training and inference costs of the video understanding model, reduces the amount of computation and the video memory occupancy rate by proposing a more efficient lossless video feature compression technology.

[0090] Embodiment 2

[0091] Figure 3 It is a schematic structural diagram of a video feature compression device in the embodiment of the present invention. This embodiment is applicable to the situation of video feature compression. The device can be implemented in a software and / or hardware manner and can be integrated into any device that provides the function of video feature compression, such as Figure 3 As shown, the video feature compression device specifically includes: an acquisition and sampling module 201, an encoding and compression module 202, and a decoding module 203.

[0092] The acquisition and sampling module 201 is used to acquire the video to be compressed and to sample the video to be compressed to obtain a set of video frames.

[0093] The encoding and compression module 202 is used to construct a video segment set based on the video frame set, and to encode and compress each video segment to obtain a target video feature set corresponding to each video segment;

[0094] The decoding module 203 is used to decode the target video feature set corresponding to each video segment to obtain compressed video features.

[0095] Optionally, the encoding and compression module 202 is specifically used for:

[0096] In chronological order, a target number of video frames are selected sequentially from the video frame set to form video segments, thus obtaining a video segment set.

[0097] For each video segment, an encoding method is used to obtain an initial set of video features corresponding to each video segment;

[0098] The initial video feature set corresponding to each video segment is compressed using a feature compressor to obtain the target video feature set corresponding to each video segment.

[0099] Optionally, the decoding module 203 includes:

[0100] The splicing unit is used to splice the target video feature sets corresponding to each video segment to obtain a first video feature set;

[0101] The decoding unit is used to input the first video feature set into the text decoder for decoding to obtain compressed video features.

[0102] Optionally, the decoding unit includes:

[0103] The filtering subunit is used to input the first video feature set into the text decoder for filtering to obtain the second video feature set;

[0104] The similarity calculation subunit is used to calculate the similarity between the second video feature set and the text features corresponding to the video to be compressed, and to obtain the compressed video features based on the similarity calculation results.

[0105] Optionally, the similarity calculation subunit is specifically used for:

[0106] Based on time information, the similarity between the second video feature set and the text features corresponding to the video to be compressed is calculated to obtain the similarity calculation result for each video feature in the second video feature set.

[0107] Video features whose similarity calculation results are greater than a preset score are identified as compressed video features.

[0108] Optionally, the acquisition and sampling module 201 is specifically used for:

[0109] Get the video to be compressed;

[0110] If the video length of the video to be compressed is less than a preset threshold, the video to be compressed is sampled based on the first sampling frequency to obtain a set of video frames;

[0111] If the length of the video to be compressed is greater than a preset threshold, the video to be compressed is sampled based on the second sampling frequency to obtain a set of video frames.

[0112] The above-mentioned products can perform the video feature compression method provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects of performing the method.

[0113] Example 3

[0114] Figure 4 A schematic diagram of an electronic device 30 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0115] like Figure 4 As shown, the electronic device 30 includes at least one processor 31 and a memory, such as a read-only memory (ROM) 32 or a random access memory (RAM) 33, communicatively connected to the at least one processor 31. The memory stores computer programs executable by the at least one processor. The processor 31 can perform various appropriate actions and processes based on the computer program stored in the ROM 32 or loaded from storage unit 38 into the RAM 33. The RAM 33 can also store various programs and data required for the operation of the electronic device 30. The processor 31, ROM 32, and RAM 33 are interconnected via a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0116] Multiple components in electronic device 30 are connected to I / O interface 35, including: input unit 36, such as keyboard, mouse, etc.; output unit 37, such as various types of monitors, speakers, etc.; storage unit 38, such as disk, optical disk, etc.; and communication unit 39, such as network card, modem, wireless transceiver, etc. Communication unit 39 allows electronic device 30 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0117] Processor 31 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 31 performs the various methods and processes described above, such as video feature compression methods:

[0118] The video to be compressed is obtained, and the video to be compressed is sampled to obtain a set of video frames;

[0119] A video segment set is constructed based on the video frame set, and each video segment is encoded and compressed to obtain a target video feature set corresponding to each video segment;

[0120] The target video feature set corresponding to each video segment is decoded to obtain the compressed video features.

[0121] In some embodiments, the video feature compression method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 30 via ROM 32 and / or communication unit 39. When the computer program is loaded into RAM 33 and executed by processor 31, one or more steps of the video feature compression method described above may be performed. Alternatively, in other embodiments, processor 31 may be configured to perform the video feature compression method by any other suitable means (e.g., by means of firmware).

[0122] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0124] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0125] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0126] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0127] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0128] In one embodiment, the present invention further includes a computer program product, which includes a computer program that, when executed by a processor, implements the video feature compression method of any embodiment of the present invention.

[0129] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0131] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A video feature compression method, characterized in that, include: The video to be compressed is obtained, and the video to be compressed is sampled to obtain a set of video frames; A video segment set is constructed based on the video frame set, and each video segment is encoded and compressed to obtain a target video feature set corresponding to each video segment; Decode the target video feature set corresponding to each video segment to obtain the compressed video features; Specifically, a video segment set is constructed based on the video frame set, and each video segment is encoded and compressed to obtain a target video feature set corresponding to each video segment, including: In chronological order, a target number of video frames are selected sequentially from the video frame set to form a video segment set; wherein, the chronological order is the order in which each video frame appears in the video to be compressed. For each video segment, an encoding method is used to obtain an initial set of video features corresponding to each video segment; The initial video feature set corresponding to each video segment is compressed using a feature compressor to obtain the target video feature set corresponding to each video segment; Specifically, the target video feature set corresponding to each video segment is decoded to obtain compressed video features, including: The target video feature sets corresponding to each video segment are concatenated to obtain the first video feature set; The first video feature set is input into a text decoder for decoding to obtain compressed video features; The first video feature set is input into a text decoder for decoding to obtain compressed video features, including: The first video feature set is input into a text decoder for filtering to obtain the second video feature set; The similarity between the second video feature set and the text features corresponding to the video to be compressed is calculated, and the compressed video features are obtained based on the similarity calculation results.

2. The method according to claim 1, characterized in that, The similarity between the second video feature set and the text features corresponding to the video to be compressed is calculated, and the compressed video features are obtained based on the similarity calculation results, including: Based on time information, the similarity between the second video feature set and the text features corresponding to the video to be compressed is calculated to obtain the similarity calculation result for each video feature in the second video feature set. Video features whose similarity calculation results are greater than a preset score are identified as compressed video features.

3. The method according to claim 1, characterized in that, The video to be compressed is acquired, and the video to be compressed is sampled to obtain a set of video frames, including: Get the video to be compressed; If the video length of the video to be compressed is less than a preset threshold, the video to be compressed is sampled based on the first sampling frequency to obtain a set of video frames; If the length of the video to be compressed is greater than a preset threshold, the video to be compressed is sampled based on the second sampling frequency to obtain a set of video frames.

4. A video feature compression device, characterized in that, include: The acquisition and sampling module is used to acquire the video to be compressed and to sample the video to be compressed to obtain a set of video frames; The encoding and compression module is used to construct a video segment set based on the video frame set, and to encode and compress each video segment to obtain a target video feature set corresponding to each video segment; The decoding module is used to decode the target video feature set corresponding to each video segment to obtain compressed video features; Specifically, the encoding and compression module is used for: In chronological order, a target number of video frames are selected sequentially from the video frame set to form a video segment set; wherein, the chronological order is the order in which each video frame appears in the video to be compressed. For each video segment, an encoding method is used to obtain an initial set of video features corresponding to each video segment; The initial video feature set corresponding to each video segment is compressed using a feature compressor to obtain the target video feature set corresponding to each video segment; The decoding module includes: The splicing unit is used to splice the target video feature sets corresponding to each video segment to obtain a first video feature set; A decoding unit is used to input the first video feature set into a text decoder for decoding to obtain compressed video features; The decoding unit includes: The filtering subunit is used to input the first video feature set into the text decoder for filtering to obtain the second video feature set; The similarity calculation subunit is used to calculate the similarity between the second video feature set and the text features corresponding to the video to be compressed, and to obtain the compressed video features based on the similarity calculation results.

5. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the video feature compression method according to any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the video feature compression method according to any one of claims 1-3.

7. A computer program product comprising a computer program that, when executed by a processor, implements the video feature compression method according to any one of claims 1-3.

Images