Video compression method and device, electronic equipment and storage medium

Abstract

The invention provides a video compression method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the frame-by-frame disassembly of a source video uploaded by a user, and generating a continuous frame sequence; performing image semantic recognition on each video frame in the frame sequence to obtain a semantic word set corresponding to each video frame; according to the semantic word sets, key frames and non-key frames in each video frame are judged frame by frame, the semantic word sets corresponding to the key frames are compared, and repeated frames identical to the key frames are determined; and deleting the non-key frames and the repeated frames, and generating a compressed video to complete compression of the source video. Compared with the prior art, the content of each frame can be analyzed in detail; key information in the video can be identified, non-key frames and repeated frames can be deleted, and redundant information in the video can be remarkably reduced; therefore, the video compression efficiency can be improved, and the video compression quality can be optimized.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a video compression method and device, an electronic device, and a storage medium. Background Art

[0002] At present, video compression technology mainly relies on video encoding algorithms. This technology does not perform special processing for the characteristics of video content, but uses a unified method for compression. Therefore, it cannot optimize processing for different types of video content, which limits the further improvement of compression effect. Existing compression technology faces a major contradiction: while pursuing a high compression ratio to reduce storage and transmission costs, it is difficult to maintain high video quality. Although lossless compression can ensure the integrity of data, its compression ratio is limited and it is difficult to meet the demand for high compression rate. In contrast, although lossy compression can achieve a higher compression ratio, it will inevitably lead to the loss of part of the video data, thereby affecting the quality of the decompressed video, which is manifested as blurred images, loss of details, color distortion and other problems.

[0003] In addition, existing video compression technology requires complex calculations and algorithm processing of the entire video, which limits the compression efficiency. In some specific application scenarios, such as car insurance inspection, there are high requirements for video compression efficiency, but current compression technology is difficult to meet these requirements. Therefore, how to improve the efficiency of video compression while ensuring compression quality has become an urgent problem to be solved. Summary of the invention

[0004] The present invention provides a method and device for video compression, an electronic device and a storage medium, the main purpose of which is to solve the problem that the quality of current video compression is not high and the compression efficiency is low.

[0005] According to a first aspect of the present disclosure, there is provided a method for video compression, comprising:

[0006] Disassemble the source video uploaded by the user frame by frame and generate a continuous frame sequence;

[0007] Performing image semantic recognition on each video frame in the frame sequence to obtain a semantic word set corresponding to each video frame;

[0008] According to the semantic word set, key frames and non-key frames in each of the video frames are determined frame by frame, and the semantic word sets corresponding to the key frames are compared to determine repeated frames that are the same as the key frames;

[0009] The non-key frames and the repeated frames are deleted, and a compressed video is generated to complete the compression of the source video.

[0010] In some embodiments, the method further comprises:

[0011] Based on the preset scene video, a video knowledge graph is constructed, and based on the knowledge graph, a scene word set is generated.

[0012] In some embodiments, performing image semantic recognition on each video frame in the frame sequence to obtain a semantic word set corresponding to each video frame includes:

[0013] Performing image enhancement processing on each of the video frames, and performing object segmentation processing on the video frames after the image enhancement processing to obtain a component frame of each segmented component in the video frame;

[0014] Using a component recognition model to perform semantic recognition on the component frames, and generating a component description word set corresponding to each of the video frames;

[0015] The component description word set is subjected to semantic generalization processing to obtain a semantic word set corresponding to each of the video frames.

[0016] In some embodiments, performing image enhancement processing on each of the video frames, and performing object segmentation processing on the video frames after the image enhancement processing to obtain a component frame of each segmented component in the video frame includes:

[0017] Performing edge extraction processing on each of the video frames to obtain an edge binary image;

[0018] Passivating the edge binary image to obtain a video frame after image enhancement processing;

[0019] Based on an image segmentation algorithm, object segmentation processing is performed on the video frame after the image enhancement processing to obtain a component frame of each of the segmented components.

[0020] In some embodiments, the semantic generalization processing is performed on the component description word set to obtain a semantic word set corresponding to each of the video frames, including:

[0021] Calculating the size of the circumscribed matrix of each of the segmentation components in the video frame;

[0022] Using the minimum circumscribed matrix as a mask, and sequentially moving the mask in each of the video frames according to a preset step length;

[0023] Acquire the frame image in the mask, perform semantic recognition on the frame image using a component recognition model, and generate a set of supplementary words for the video frame;

[0024] The supplementary word set is merged with the component description word set to generate the semantic word set.

[0025] In some embodiments, judging the key frames and non-key frames in each of the video frames frame by frame according to the semantic word set, comparing the semantic word sets corresponding to the key frames, and determining the repeated frames identical to the key frames, includes:

[0026] Calculating the similarity between each word in the semantic word set and each word in the scene word set, and calculating the sum of the similarities of each word;

[0027] Determine a target semantic word set whose sum of similarities exceeds a first threshold, and use a video frame corresponding to the target semantic word set as the key frame, wherein the sum of similarities between the semantic word set of the key frame and the scene word set is the highest;

[0028] The similarity between the key frame and each of the video frames is calculated, and the video frames whose similarity exceeds a second threshold are determined to be the repeated frames.

[0029] According to a second aspect of the present disclosure, a video compression apparatus is provided, comprising:

[0030] A disassembly unit, used to disassemble the source video uploaded by the user frame by frame and generate a continuous frame sequence;

[0031] A recognition unit, configured to perform image semantic recognition on each video frame in the frame sequence to obtain a semantic word set corresponding to each video frame;

[0032] A determination unit, configured to determine the key frames and non-key frames in each of the video frames frame by frame according to the semantic word set, compare the semantic word sets corresponding to the key frames, and determine the repeated frames identical to the key frames;

[0033] The first generating unit is used to delete the non-key frames and the repeated frames and generate a compressed video to complete the compression of the source video.

[0034] In some embodiments, the apparatus further comprises:

[0035] The second generating unit is used to construct a video knowledge graph based on a preset scene video, and to generate a scene word set based on the knowledge graph.

[0036] In some embodiments, the identification unit comprises:

[0037] A segmentation module, used to perform image enhancement processing on each of the video frames, and perform object segmentation processing on the video frames after the image enhancement processing, to obtain a component frame of each segmented component in the video frame;

[0038] A recognition module, used to perform semantic recognition on the component frames using a component recognition model, and generate a component description word set corresponding to each of the video frames;

[0039] The generalization module is used to perform semantic generalization processing on the component description word set to obtain a semantic word set corresponding to each of the video frames.

[0040] In some embodiments, the segmentation module is further configured to:

[0041] Performing edge extraction processing on each of the video frames to obtain an edge binary image;

[0042] Passivating the edge binary image to obtain a video frame after image enhancement processing;

[0043] Based on an image segmentation algorithm, object segmentation processing is performed on the video frame after the image enhancement processing to obtain a component frame of each of the segmented components.

[0044] In some embodiments, the generalization module is further configured to:

[0045] Calculating the size of the circumscribed matrix of each of the segmentation components in the video frame;

[0046] Using the minimum circumscribed matrix as a mask, and sequentially moving the mask in each of the video frames according to a preset step length;

[0047] Acquire the frame image in the mask, perform semantic recognition on the frame image using a component recognition model, and generate a set of supplementary words for the video frame;

[0048] The supplementary word set is merged with the component description word set to generate the semantic word set.

[0049] In some embodiments, the determining unit includes:

[0050] A calculation module, used to calculate the similarity between each word in the semantic word set and each word in the scene word set, and calculate the sum of the similarities of each word;

[0051] A first determination module is used to determine a target semantic word set whose sum of similarities exceeds a first threshold, and use a video frame corresponding to the target semantic word set as the key frame, wherein the sum of similarities between the semantic word set of the key frame and the scene word set is the highest;

[0052] The second determination module is used to calculate the similarity between the key frame and each of the video frames, and determine the video frames whose similarity exceeds a second threshold as the repeated frames.

[0053] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0054] at least one processor; and

[0055] a memory communicatively connected to the at least one processor; wherein,

[0056] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0057] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0058] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method as described in the first aspect above.

[0059] The present disclosure provides a method and device for video compression, an electronic device and a storage medium, which disassemble the source video uploaded by the user frame by frame and generate a continuous frame sequence; perform image semantic recognition on each video frame in the frame sequence to obtain a semantic word set corresponding to each video frame; according to the semantic word set, determine the key frame and non-key frame in each video frame frame by frame, compare the semantic word set corresponding to the key frame, and determine the repeated frame that is the same as each key frame; delete the non-key frame and the repeated frame, and generate a compressed video to complete the compression of the source video. Compared with the related art, the embodiment of the present disclosure can analyze the content of each frame in detail by disassembling the source video frame by frame, so as to achieve accurate control of the video content; perform image semantic recognition frame by frame, and can identify the key information in the video, providing an accurate basis for subsequent key frame judgment; delete non-key frames and repeated frames, and can significantly reduce the redundant information in the video, so as to improve the compression efficiency and make the generated compressed video more compact; thus, the embodiment of the present disclosure can not only improve the video compression efficiency, but also optimize the video compression quality.

[0060] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0062] Figure 1 A flowchart of a video compression method provided by an embodiment of the present disclosure;

[0063] Figure 2 A flowchart of another video compression method provided by an embodiment of the present disclosure;

[0064] Figure 3 An original image of a video frame provided in an embodiment of the present disclosure;

[0065] Figure 4 A video frame after edge extraction provided by an embodiment of the present disclosure;

[0066] Figure 5 A video frame after passivation processing provided by an embodiment of the present disclosure;

[0067] Figure 6 A segmented license plate component frame image provided by an embodiment of the present disclosure;

[0068] Figure 7 A schematic diagram of the structure of a video compression device provided by an embodiment of the present disclosure;

[0069] Figure 8 A schematic diagram of the structure of another video compression device provided by an embodiment of the present disclosure;

[0070] Fig. 9 A schematic block diagram of an exemplary electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0071] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0072] The following describes the video compression method and apparatus, electronic device, and storage medium according to the embodiments of the present disclosure with reference to the accompanying drawings.

[0073] Figure 1 A flowchart of a video compression method provided in an embodiment of the present disclosure.

[0074] like Figure 1 As shown, the method comprises the following steps:

[0075] Step 101: Decompose the source video uploaded by the user frame by frame and generate a continuous frame sequence.

[0076] In an embodiment of the present disclosure, after the user uploads the source video to a designated processing system or platform, in order to achieve compression of the source video, the system will first start a video disassembly processing flow. This process requires the video, a continuous dynamic image stream, to be carefully divided into its basic constituent units - individual static image frames. These image frames are extracted one by one in the order in which they originally appeared in the video, and a continuous frame sequence is generated. This frame sequence is like a string of pearls, each pearl represents a frame in the video, they are closely connected, and together record the complete process of the video from the beginning to the end. Through this processing method, the system can provide a clear, orderly and easy-to-manage image frame set for subsequent video content analysis, processing or editing operations.

[0077] Step 102: Perform image semantic recognition on each video frame in the frame sequence to obtain a semantic word set corresponding to each video frame.

[0078] In the embodiment of the present disclosure, after the source video is disassembled frame by frame and a continuous frame sequence is generated, the next stage is immediately entered: image semantic recognition of each video frame. This step aims to deeply understand the information and meaning carried by each frame of the image, rather than just processing at the pixel level.

[0079] Image semantic recognition technology uses advanced machine learning and deep learning algorithms to perform a detailed analysis of each video frame. During this process, the system will identify various objects, scenes, actions in the frame and the relationships between them. For example, the system can identify whether there are people, animals, buildings, natural landscapes and other elements in the frame, whether these elements are in motion, what the spatial relationship between them is, whether a certain scene or story clue is formed, etc.

[0080] Based on these analysis results, the system generates a corresponding semantic word set for each video frame. This set contains a series of keywords or phrases that can accurately describe the content of the frame. These semantic words not only reflect the specific elements in the frame, but also reflect the semantic associations and overall context between them. For example, a frame containing a beach, sunshine, beach volleyball, and a laughing crowd may have a corresponding semantic word set including "beach scenery", "sunny", "beach volleyball game", "joyful atmosphere", etc.

[0081] Through such an image semantic recognition process, the system can provide richer, more accurate and easy-to-understand semantic information for subsequent video content analysis, classification, retrieval or editing operations. This can not only improve the efficiency and accuracy of video processing, but also provide users with a more intelligent and personalized video service experience.

[0082] Step 103 , judging the key frames and non-key frames in each of the video frames frame by frame according to the semantic word set, comparing the semantic word sets corresponding to the key frames, and determining the repeated frames identical to the key frames.

[0083] In an embodiment of the present disclosure, after obtaining the semantic word set corresponding to each video frame, the system will then determine whether each video frame is a key frame frame by frame based on the semantic information. First, the system will analyze the semantic word set of each video frame to evaluate the amount of information it contains, its uniqueness, and the degree of difference from the previous and next frames. For example, if the semantic word set of a frame contains a large number of unique and informative words, and these words are significantly different from the semantic word sets of adjacent frames, then the frame is likely to be determined as a key frame. Key frames usually carry important information in the video, such as scene transitions, the appearance or disappearance of characters, the occurrence of important events, etc.

[0084] On the contrary, if the semantic word set of a frame is highly similar to that of adjacent frames and contains little or no new information, then the frame is likely to be judged as a non-key frame. Non-key frames usually do not contain key information in the video, but exist as transitions or supplements between key frames.

[0085] After determining the keyframes, the system further compares the semantic word sets corresponding to these keyframes to identify whether there are duplicate frames with exactly the same content as a keyframe. This step is crucial for removing redundant information in the video. The system determines whether two keyframes are similar enough to be considered duplicates by calculating the similarity or degree of match between the semantic word sets. If the semantic word sets of two keyframes are highly consistent and their positions in the video are close, then the system is likely to mark one of them as a duplicate frame.

[0086] Through such frame-by-frame judgment and comparison process, the system can accurately identify key frames and repeated frames in the video, providing strong support for subsequent video compression, editing or analysis operations. This not only helps to improve the efficiency and accuracy of video processing, but also provides users with a smoother, more compact and information-rich video experience.

[0087] In other embodiments of the present disclosure, the amount of information contained in the video frame may be compared with a preset scene, that is, an evaluation standard may be generated based on the preset scene video.

[0088] Step 104, deleting the non-key frames and the repeated frames, and generating a compressed video to complete the compression of the source video.

[0089] In the embodiment of the present disclosure, after successfully identifying and distinguishing the key frames, non-key frames and repeated frames in the video, the system immediately enters the final stage of video compression. The core task of this stage is to remove non-key frames and frames with repeated content that contribute less to the overall content and situational expression of the video from the video, so as to minimize the amount of video data while ensuring the integrity and coherence of the video information. The system will traverse the entire frame sequence and check the status of each frame one by one. For frames marked as non-key frames, the system will remove them from the frame sequence. These frames often do not contain key information in the video, such as important scene transitions, character movements or emotional expressions, so their absence will not significantly affect the overall understanding of the video. Then, the system will traverse the remaining frame sequence again, find and delete those frames that are determined to be repeated frames. These frames are highly consistent with the key frames in content and usually appear at similar time points, so their existence will only increase the redundancy of the video without providing new information or visual experience to the audience.

[0090] After deleting non-key frames and duplicate frames, the system will reorganize the remaining frame sequence to ensure that the smoothness and continuity of the video are not affected. During this process, the system may insert some transition frames or adjust the playback order of frames as needed to ensure that the video remains visually smooth and logically coherent. The system will use video compression algorithms and technologies to encode the processed frame sequence into a compressed video file. This compressed video file not only has a smaller data volume and occupies less storage space, but also retains the key information and overall context of the video, providing viewers with a high-quality viewing experience.

[0091] The present disclosure provides a method for video compression, which disassembles the source video uploaded by the user frame by frame and generates a continuous frame sequence; performs image semantic recognition on each video frame in the frame sequence to obtain a semantic word set corresponding to each video frame; according to the semantic word set, the key frame and non-key frame in each video frame are judged frame by frame, and the semantic word set corresponding to the key frame is compared to determine the repeated frame that is the same as each key frame; the non-key frame and the repeated frame are deleted, and a compressed video is generated to complete the compression of the source video. Compared with the related art, the embodiment of the present disclosure can analyze the content of each frame in detail by disassembling the source video frame by frame, so as to achieve accurate control of the video content; performs image semantic recognition frame by frame, and can identify the key information in the video, providing an accurate basis for subsequent key frame judgment; deletes non-key frames and repeated frames, and can significantly reduce the redundant information in the video, thereby improving the compression efficiency and making the generated compressed video more compact; thus, the embodiment of the present disclosure can not only improve the video compression efficiency, but also optimize the video compression quality.

[0092] In order to clearly illustrate the embodiment of the present disclosure, this embodiment provides a flowchart of another video compression method.

[0093] like Figure 2 As shown, the method comprises the following steps:

[0094] Step 201: construct a video knowledge graph based on a preset scene video, and generate a scene word set based on the knowledge graph.

[0095] Specifically, in step 201, in order to more effectively identify important information in the video frame, a standard verification scene video (preset scene video) can be created in a specific scene, and then a corresponding standard verification word set, i.e., a scene word set, can be established. For example, in the scene of car insurance inspection, an insurance video knowledge graph is constructed to collect user input prompt words, such as: vehicle type (such as "family car"), brand model (such as "XX brand HQ7002BEV37"), license plate (such as "X·Q12345"), etc., and the vehicle inspection model of the vehicle model is found in the vehicle model library (as shown in the figure below). Based on the video knowledge graph, the text content and model image content are mapped to a scene word set that is related to the theme of this scene and has similar meaning.

[0096] Step 202: Decompose the source video uploaded by the user frame by frame and generate a continuous frame sequence.

[0097] Step 203: perform image enhancement processing on each of the video frames, and perform object segmentation processing on the video frames after the image enhancement processing to obtain a component frame of each segmented component in the video frame.

[0098] Step 2031: perform edge extraction processing on each of the video frames to obtain an edge binary map.

[0099] Step 2032: perform passivation processing on the edge binary image to obtain a video frame after image enhancement processing.

[0100] Step 2033: Based on an image segmentation algorithm, perform object segmentation processing on the video frame after the image enhancement processing to obtain a component frame of each of the segmented components.

[0101] Specifically in step 203, when processing video data, in order to improve the accuracy of subsequent analysis or visual effects, it is necessary to perform image enhancement processing on each video frame, and on this basis, further implement object segmentation processing on the video frame after image enhancement processing, and the ultimate goal is to obtain the component frame corresponding to each independent segmentation component in the video frame. By performing edge extraction processing on each video frame, the boundary between the object and the background in the image, or the boundary between objects, can be identified and highlighted. By applying algorithms such as Canny edge detection, Sobel operator or Laplacian operator, the edge information in the image can be accurately captured and an edge binary map is generated. In this binary map, edge pixels are assigned a specific value, while non-edge pixels are assigned another value, thereby forming a sharp contrast, providing clear edge information for subsequent processing.

[0102] Passivation is an image enhancement technique that aims to smooth image edges and reduce the influence of noise and fine structures. The purpose of passivation of the obtained edge binary image is to enhance the overall contrast of the image by blurring or smoothing while ensuring that the edge information is basically intact, so that the boundary between the object and the background is clearer, providing more favorable conditions for subsequent object segmentation. Specific methods may include using filters such as Gaussian blur and mean blur, and adjusting the blur radius and intensity to achieve the best image enhancement effect.

[0103] Based on advanced image segmentation algorithms, object segmentation is performed on the enhanced video frames. Object segmentation is a complex task in computer vision, which requires the algorithm to accurately distinguish different objects in the image and separate them from the background or other objects. This process can be, but is not limited to, through deep learning models such as Mask R-CNN, UNet, etc. These models can be trained to recognize complex image features and achieve high-precision pixel-level segmentation. In this step, each successfully segmented object or component will generate an independent component frame, which not only retains the key information in the original video frame, but also displays the details of the video content in a more structured form, which greatly facilitates subsequent analysis, recognition or synthesis.

[0104] Specifically, obtain an edge binary image (an image containing only black and white colors, with black point color value 0 and white point color value 255); 2) Smooth the edge. The skeleton image obtained based on the general edge detection algorithm has serious jagged effects and discontinuous edge pixels. In order to facilitate the next step of recognition, it needs to be smoothed according to the following algorithm:

[0105] Scan each pixel of the image. If it is a highlight point Pi (brightness value 255), define the radiation radius as Δ and the attenuation factor as α. Calculate the influence of the highlight point on the surrounding points Pj according to the following formula:

[0106] vi j=255-255*di*α, where di is the distance from Pj to Pi, equal to (Pj.x-Pi.x)2+(Pj.y-Pi.y)2, and the brightness value of point Pj is the sum of the influence of all highlight points on it, that is, Vj=∑Vi j, where when Vj is greater than 255, it is 255. Figure 3 is the original image of the video frame, Figure 4 For the video frame after edge extraction, Figure 5 This is the video frame after passivation processing; zooming in on the details shows that the license plate part in the image has formed a continuous edge from a discontinuous edge after passivation, which facilitates the subsequent judgment of the component shape and obtains the accurate area of ​​the target component.

[0107] The diffusion method is used to segment the image content into multiple objects. The specific method is: define the diffusion damping coefficient ζ (value range: 0~1), starting from any highlight point Pi (brightness Vi=255, coordinates xi, yi), its 8 adjacent points are (xi-1, yi), (xi-1, yi-1), (xi-1, yi+1), (xi, yi-1), (xi, yi+1), (xi+1, yi-1), (xi+1, yi), (xi+1, yi+1). Detect the brightness value Vj of the adjacent point. If the brightness of the adjacent point Vj / 255>=ζ, continue to search for adjacent points. Otherwise, it means that the edge of the object has been reached, stop searching, repeat the above process, and achieve image object segmentation. This method is similar to gas diffusion, so it is called diffusion segmentation. It should be noted that diffusion segmentation depends on passivation. If passivation is not performed, there will be more edge discontinuities, causing the diffusion to stop prematurely, resulting in fragmented image segmentation and difficulty in achieving the expected goal. Figure 6 is a segmented license plate component frame image. For the aforementioned test image in this embodiment, when ζ=0.5, the above algorithm can more accurately segment key vehicle parts such as the front grille (including the license plate), tires, and headlights.

[0108] Step 204: perform semantic recognition on the component frames using a component recognition model to generate a component description word set corresponding to each of the video frames.

[0109] Specifically, in step 204, in the process of processing and analyzing the video frame, in order to further mine and utilize the semantic information of the video content, a component recognition model can be used to perform in-depth semantic recognition on the component frame obtained after the object segmentation process. This step aims to map the visual elements in each component frame to a specific semantic concept, thereby generating a component description word set that can accurately describe the content of the component frame.

[0110] Specifically, the component recognition model is an intelligent model based on deep learning or machine learning technology. It is optimized with a large amount of training data and can recognize various objects, components or features in the image and understand the semantic meaning they represent. When this model is applied to the component frame, it analyzes the pixel information, shape features, texture details, etc. in the frame one by one, matches them with the semantic knowledge stored in the model, and finally determines the semantic category corresponding to each component. In this process, the component recognition model not only recognizes what the objects in the component frame are, but also further understands the roles these objects play in the video content and the semantic relationships between them. For example, if a component frame contains a person, the model may recognize details such as whether the person is male or female, what kind of clothes he is wearing, what actions he is taking, and extract this information in the form of descriptive words.

[0111] As the component recognition model conducts in-depth analysis and semantic recognition of component frames, a rich set of component description words will be generated for each video frame. This set contains keywords or phrases that describe each component in the frame. They not only reflect the visual features of the components, but also reveal the semantic connections and overall context between the components. Such a set of description words provides powerful semantic support for subsequent operations such as video content understanding, retrieval, editing or synthesis, making video processing and analysis more intelligent and efficient.

[0112] After completing the object segmentation, determine the segmentation range, decompose the original image into multiple objects, and complete the "image-to-text" conversion through the large model to obtain the textual meaning of the segmented objects. According to the knowledge graph, the description text is regularized into a word set. The specific approach is: complete the "image-to-text" conversion through the large model, generate a text description of each object, segment the text description, filter the function words, and regularize the remaining nouns, verbs and other content words with reference to the knowledge graph to form a word set for the object.

[0113] Step 205: perform semantic generalization processing on the component description word set to obtain a semantic word set corresponding to each of the video frames.

[0114] Step 2051, calculating the size of the external matrix of each of the segmentation components in the video frame.

[0115] Step 2052: Use the minimum circumscribed matrix as a mask, and move the mask in sequence in each of the video frames according to a preset step length.

[0116] Step 2053, obtaining the frame image in the mask, performing semantic recognition on the frame image using a component recognition model, and generating a set of supplementary words for the video frame.

[0117] Step 2054: merge the supplementary word set with the component description word set to generate the semantic word set.

[0118] Specifically, in step 205, in order to further improve the comprehensiveness and accuracy of the semantic understanding of the video frame, it is necessary to perform semantic generalization processing on the initially generated component description word set, so as to obtain a richer and more detailed semantic word set.

[0119] First, it is necessary to determine the minimum external matrix (i.e., the minimum rectangular area surrounding the component) of each component segmented by the previous processing in the video frame. This step is the basis for the subsequent mask setting and movement, ensuring that all important component areas can be covered during semantic generalization while avoiding unnecessary repeated calculations. The size of the external matrix not only reflects the physical size of the component, but also provides a basis for the subsequent selection of the mask size.

[0120] These minimum circumscribed matrices are used as the initial mask and slid across the entire video frame at a preset step size. This preset step size needs to be determined based on a comprehensive consideration of factors such as the resolution of the video frame, the density of components, and the accuracy requirements for semantic recognition. By gradually moving the mask, more potential semantic information areas in the frame can be captured. These areas may not completely correspond to the initially segmented components, but are also important for understanding the video content.

[0121] For each mask position after movement, the frame image area covered by it is extracted, and the trained component recognition model is used again to perform semantic recognition on the area. The purpose of this step is to mine the semantic information in the video frame from a more fine-grained perspective, especially those that may not be fully captured due to inaccurate segmentation or occlusion between components. Through this process, a series of supplementary word sets corresponding to the mask position can be generated, which reflect the semantic characteristics of the image content in the mask area.

[0122] The supplementary word set obtained from each mask position is merged with the initial component description word set. This process needs to take into account the repetitiveness and semantic relevance of the vocabulary to ensure that the final semantic word set generated is both comprehensive and refined. When merging, methods such as deduplication, synonym merging, and clustering based on semantic similarity can be used to optimize the quality and practicality of the word set. Through this step, a semantic word set containing richer semantic information can be obtained, providing strong support for the in-depth understanding and application of video content.

[0123] Step 206: Calculate the similarity between each word in the semantic word set and each word in the scene word set, and calculate the sum of the similarities of each word.

[0124] Step 207, determining a target semantic word set whose sum of similarities exceeds a first threshold, and using a video frame corresponding to the target semantic word set as the key frame, wherein the sum of similarities between the semantic word set of the key frame and the scene word set is the highest.

[0125] Step 208: Calculate the similarity between the key frame and each of the video frames, and determine the video frames whose similarity exceeds a second threshold as the repeated frames.

[0126] Specifically, in steps 206 to 208, during the video processing and analysis, in order to identify key frames and potential repeated frames that are highly relevant to a specific scene, it is necessary to further compare and analyze the previously generated semantic word set.

[0127] By comparing the semantic word set of the video frame with the predefined scene word set. The scene word set usually contains a series of words closely related to a specific scene or event, which can reflect the core features and key information of the scene. In order to measure the similarity between each word in the semantic word set and each word in the scene word set, a variety of semantic similarity calculation methods can be used, such as cosine similarity based on word vectors, semantic distance based on WordNet, etc.

[0128] For each word in the semantic word set, it is necessary to calculate its similarity with all words in the scene word set, and accumulate these similarity values ​​to obtain the sum of the similarities of the word. This process not only considers the relevance of a single word to the scene, but also enhances the significance of the overall semantic match by accumulation.

[0129] After obtaining the sum of similarities of each semantic word set, a first threshold needs to be set to filter out semantic word sets that are highly relevant to the target scene. This threshold can be adjusted according to actual application scenarios and requirements to ensure that the filtered key frames are representative and not too redundant.

[0130] For those semantic word sets whose sum of similarities exceeds the first threshold, their corresponding video frames are regarded as key frames. Among multiple key frames, the video frame corresponding to the semantic word set with the highest sum of similarities is further selected as the final key frame. This ensures that the key frames we select can best represent the core features and key information of the target scene.

[0131] After determining the key frame, it is necessary to further analyze the similarity between video frames to identify potential duplicate frames. This step can be achieved by calculating the semantic similarity between the key frame and each other video frame. Similarly, a variety of semantic similarity calculation methods can be used, such as cosine similarity based on word vectors.

[0132] After obtaining the similarity between the key frame and other video frames, a second threshold needs to be set to determine which video frames are duplicated. The selection of this threshold needs to be adjusted according to the actual application scenario and requirements to ensure that duplicate frames can be accurately identified. For those video frames whose similarity with the key frame exceeds the second threshold, they are regarded as duplicate frames and can be merged or deleted in subsequent processing.

[0133] Step 209, deleting the non-key frames and the repeated frames, and generating a compressed video to complete the compression of the source video.

[0134] Specifically, in step 209, after successfully identifying and distinguishing the key frames, non-key frames and repeated frames in the video, the system immediately enters the final stage of video compression. The core task of this stage is to remove non-key frames and frames with repeated content that contribute less to the overall content and situational expression of the video from the video, so as to minimize the amount of video data while ensuring the integrity and coherence of the video information. The system will traverse the entire frame sequence and check the status of each frame one by one. For frames marked as non-key frames, the system will remove them from the frame sequence. These frames often do not contain key information in the video, such as important scene transitions, character movements or emotional expressions, so their absence will not have a significant impact on the overall understanding of the video. Then, the system will traverse the remaining frame sequence again to find and delete those frames that are determined to be repeated frames. These frames are highly consistent with the key frames in content and usually appear at similar time points, so their existence will only increase the redundancy of the video without providing new information or visual experience to the audience.

[0135] After deleting non-key frames and duplicate frames, the system will reorganize the remaining frame sequence to ensure that the smoothness and continuity of the video are not affected. During this process, the system may insert some transition frames or adjust the playback order of frames as needed to ensure that the video remains visually smooth and logically coherent. The system will use video compression algorithms and technologies to encode the processed frame sequence into a compressed video file. This compressed video file not only has a smaller data volume and occupies less storage space, but also retains the key information and overall context of the video, providing viewers with a high-quality viewing experience.

[0136] In summary, through such a processing flow, the system can successfully complete the compression task of the source video and provide users with a more efficient, convenient and easy-to-share video file. This not only helps to reduce the cost of video storage and transmission, but also promotes the widespread dissemination and rapid consumption of videos, creating a broader space and possibility for the creation, distribution and appreciation of video content.

[0137] It should be noted that the embodiments of the present disclosure may include multiple steps. For the convenience of description, these steps are numbered, but these numbers do not limit the execution time slots or execution order between the steps; these steps can be implemented in any order, and the embodiments of the present disclosure do not limit this.

[0138] Corresponding to the above-mentioned video compression method, the present invention also provides a video compression device. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment, and will not be repeated in the present invention.

[0139] Figure 7 A schematic diagram of the structure of a video compression device provided by an embodiment of the present disclosure is shown in FIG. Figure 7 As shown, including:

[0140] The disassembly unit 31 is used to disassemble the source video uploaded by the user frame by frame and generate a continuous frame sequence;

[0141] The recognition unit 32 is used to perform image semantic recognition on each video frame in the frame sequence to obtain a semantic word set corresponding to each video frame;

[0142] A determination unit 33, configured to determine the key frames and non-key frames in each of the video frames frame by frame according to the semantic word set, compare the semantic word sets corresponding to the key frames, and determine the repeated frames identical to the key frames;

[0143] The first generating unit 34 is used to delete the non-key frames and the repeated frames and generate a compressed video to complete the compression of the source video.

[0144] The present disclosure provides a video compression device, which disassembles the source video uploaded by the user frame by frame and generates a continuous frame sequence; performs image semantic recognition on each video frame in the frame sequence to obtain a semantic word set corresponding to each video frame; according to the semantic word set, the key frame and non-key frame in each video frame are judged frame by frame, and the semantic word set corresponding to the key frame is compared to determine the repeated frame that is the same as each key frame; the non-key frame and the repeated frame are deleted, and a compressed video is generated to complete the compression of the source video. Compared with the related art, the embodiment of the present disclosure can analyze the content of each frame in detail by disassembling the source video frame by frame, so as to achieve accurate control of the video content; the image semantic recognition is performed frame by frame, so as to identify the key information in the video, and provide an accurate basis for the subsequent key frame judgment; the non-key frame and the repeated frame are deleted, so as to significantly reduce the redundant information in the video, so as to improve the compression efficiency and make the generated compressed video more compact; thus, the embodiment of the present disclosure can not only improve the video compression efficiency, but also optimize the video compression quality.

[0145] Furthermore, in a possible implementation of this embodiment, as Figure 8 As shown, the device also includes:

[0146] The second generating unit 35 is used to construct a video knowledge graph based on a preset scene video, and generate a scene word set based on the knowledge graph.

[0147] Furthermore, in a possible implementation of this embodiment, as Figure 8 As shown, the identification unit 32 includes:

[0148] The segmentation module 321 is used to perform image enhancement processing on each of the video frames, and perform object segmentation processing on the video frames after the image enhancement processing to obtain a component frame of each segmented component in the video frame;

[0149] The recognition module 322 is used to perform semantic recognition on the component frame using a component recognition model to generate a component description word set corresponding to each video frame;

[0150] The generalization module 323 is used to perform semantic generalization processing on the component description word set to obtain a semantic word set corresponding to each of the video frames.

[0151] Furthermore, in a possible implementation of this embodiment, the segmentation module 321 is further configured to:

[0152] Performing edge extraction processing on each of the video frames to obtain an edge binary image;

[0153] Passivating the edge binary image to obtain a video frame after image enhancement processing;

[0154] Based on an image segmentation algorithm, object segmentation processing is performed on the video frame after the image enhancement processing to obtain a component frame of each of the segmented components.

[0155] Furthermore, in a possible implementation of this embodiment, the generalization module 323 is further used to:

[0156] Calculating the size of the circumscribed matrix of each of the segmentation components in the video frame;

[0157] Using the minimum circumscribed matrix as a mask, and sequentially moving the mask in each of the video frames according to a preset step length;

[0158] Acquire the frame image in the mask, perform semantic recognition on the frame image using a component recognition model, and generate a set of supplementary words for the video frame;

[0159] The supplementary word set is merged with the component description word set to generate the semantic word set.

[0160] Furthermore, in a possible implementation of this embodiment, as Figure 8 As shown, the determining unit 33 includes:

[0161] A calculation module 331, used to calculate the similarity between each word in the semantic word set and each word in the scene word set, and calculate the sum of the similarities of each word;

[0162] A first determination module 332 is used to determine a target semantic word set whose sum of similarities exceeds a first threshold, and use a video frame corresponding to the target semantic word set as the key frame, wherein the sum of similarities between the semantic word set of the key frame and the scene word set is the highest;

[0163] The second determination module 333 is configured to calculate the similarity between the key frame and each of the video frames, and determine the video frames whose similarity exceeds a second threshold as the repeated frames.

[0164] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principle is the same, which is not limited in this embodiment.

[0165] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0166] Fig. 9 A schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0167] like Fig. 9As shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 402 or a computer program loaded from a storage unit 408 to a RAM (Random Access Memory) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An I / O (Input / Output) interface 405 is also connected to the bus 404.

[0168] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0169] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as a method for video compression. For example, in some embodiments, the method for video compression may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to execute the aforementioned video compression method in any other appropriate manner (for example, by means of firmware).

[0170] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a special purpose or general purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0171] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0172] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a RAM, a ROM, an EPROM (Electrically Programmable Read-Only-Memory) or a flash memory, an optical fiber, a CD-ROM (Compact Dis sc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0173] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0174] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0175] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0176] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0177] The various numerical numbers such as first and second involved in the present disclosure are only for the convenience of description and are not used to limit the scope of the embodiments of the present disclosure, but also indicate the order of precedence.

[0178] At least one in the present disclosure may also be described as one or more, and a plurality may be two, three, four or more, which is not limited in the present disclosure. In the embodiments of the present disclosure, for a technical feature, the technical features in the technical feature are distinguished by "first", "second", "third", "A", "B", "C" and "D", etc., and there is no order of precedence or size between the technical features described by the "first", "second", "third", "A", "B", "C" and "D".

[0179] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0180] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A video compression method, characterized in that: include: Disassemble the source video uploaded by the user frame by frame and generate a continuous frame sequence; Performing image semantic recognition on each video frame in the frame sequence to obtain a semantic word set corresponding to each video frame; According to the semantic word set, key frames and non-key frames in each of the video frames are determined frame by frame, and the semantic word sets corresponding to the key frames are compared to determine repeated frames that are the same as the key frames; The non-key frames and the repeated frames are deleted, and a compressed video is generated to complete the compression of the source video.

2. The method according to claim 1, characterized in that The method further comprises: Based on the preset scene video, a video knowledge graph is constructed, and based on the knowledge graph, a scene word set is generated.

3. The method according to claim 1, characterized in that The performing image semantic recognition on each video frame in the frame sequence to obtain a semantic word set corresponding to each video frame includes: Performing image enhancement processing on each of the video frames, and performing object segmentation processing on the video frames after the image enhancement processing to obtain a component frame of each segmented component in the video frame; Using a component recognition model to perform semantic recognition on the component frames, and generating a component description word set corresponding to each of the video frames; The component description word set is subjected to semantic generalization processing to obtain a semantic word set corresponding to each of the video frames.

4. The method according to claim 3, characterized in that The step of performing image enhancement processing on each of the video frames and performing object segmentation processing on the video frames after the image enhancement processing to obtain a component frame of each segmented component in the video frame includes: Performing edge extraction processing on each of the video frames to obtain an edge binary image; Passivating the edge binary image to obtain a video frame after image enhancement processing; Based on an image segmentation algorithm, object segmentation processing is performed on the video frame after the image enhancement processing to obtain a component frame of each of the segmented components.

5. The method according to claim 3, characterized in that: The semantic generalization processing is performed on the component description word set to obtain a semantic word set corresponding to each video frame, including: Calculating the size of the circumscribed matrix of each of the segmentation components in the video frame; Using the minimum circumscribed matrix as a mask, and sequentially moving the mask in each of the video frames according to a preset step length; Acquire the frame image in the mask, perform semantic recognition on the frame image using a component recognition model, and generate a set of supplementary words for the video frame; The supplementary word set is merged with the component description word set to generate the semantic word set.

6. The method according to claim 1, characterized in that The step of determining the key frames and non-key frames in each of the video frames frame by frame according to the semantic word set, comparing the semantic word sets corresponding to the key frames, and determining repeated frames identical to the key frames includes: Calculating the similarity between each word in the semantic word set and each word in the scene word set, and calculating the sum of the similarities of each word; Determine a target semantic word set whose sum of similarities exceeds a first threshold, and use a video frame corresponding to the target semantic word set as the key frame, wherein the sum of similarities between the semantic word set of the key frame and the scene word set is the highest; The similarity between the key frame and each of the video frames is calculated, and the video frames whose similarity exceeds a second threshold are determined to be the repeated frames.

7. A video compression device, characterized in that: include: A disassembly unit, used to disassemble the source video uploaded by the user frame by frame and generate a continuous frame sequence; A recognition unit, configured to perform image semantic recognition on each video frame in the frame sequence to obtain a semantic word set corresponding to each video frame; A determination unit, configured to determine the key frames and non-key frames in each of the video frames frame by frame according to the semantic word set, compare the semantic word sets corresponding to the key frames, and determine the repeated frames identical to the key frames; The first generating unit is used to delete the non-key frames and the repeated frames and generate a compressed video to complete the compression of the source video.

8. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images