Video encoding method and system based on dynamic optimization of background frames

Abstract

The application provides a video coding method and system based on dynamic optimization of background frames, which comprises the following steps: performing background analysis on a video to be coded to obtain a background frame, performing difference analysis on the video to be coded and the background frame to obtain a difference image; performing target detection on the difference image to obtain a target object, performing motion analysis on the target object to obtain a motion vector; coding a current video frame according to the background frame and the motion vector to obtain a segmented coding stream, and updating the background frame according to the motion vector; continuously performing the step of coding the current video frame according to the background frame and the motion vector according to the updated background frame until all video frames in the video to be coded are coded; and combining the segmented coding streams to obtain a video coding stream. In the embodiment of the application, the current video frame is coded according to the background frame and the motion vector, and the background in the current video frame does not need to be repeatedly coded, so that the video coding efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of video coding technology, and in particular to a video coding method and system based on dynamic optimization of background frames. Background Technology

[0002] Video images have a large amount of data. When interacting with video images, it is necessary to encode and decode the video images. The main function of video encoding is to compress and encode video pixel data into a video bitstream, thereby reducing the amount of video data and achieving the purpose of reducing network bandwidth and storage space during transmission.

[0003] In existing video encoding processes, the entire video is typically encoded, which often results in the encoding of a large amount of redundant background information, thereby reducing video encoding efficiency. Summary of the Invention

[0004] The purpose of this invention is to provide a video coding method and system based on dynamic optimization of background frames, so as to solve the problem of low video coding efficiency in the prior art.

[0005] The present invention is implemented as follows: a video coding method based on dynamic optimization of background frames, the method comprising:

[0006] Obtain the video to be encoded, and segment the video to be encoded to obtain a segmented video;

[0007] The segmented video is subjected to background analysis to obtain background frames, and the frames to be encoded in the segmented video are compared with the corresponding background frames to obtain difference images.

[0008] Target detection is performed on the difference image to obtain the target object, and motion analysis is performed on the target object to obtain the motion vector;

[0009] The current video frame is encoded based on the background frame and the motion vector to obtain a segmented encoded stream, and the segmented encoded streams are combined to obtain a video encoded stream.

[0010] Preferably, motion analysis is performed on the target object to obtain motion vectors, including:

[0011] In the same segmented video, the frame to be encoded containing the target object is set as an associated frame, and an association set is constructed based on the associated frame;

[0012] Key point identification is performed on the target object in different associated frames to obtain object key points, and a key point vector is generated based on the object key points;

[0013] Generate reference key points based on the object key points, and generate a reference point vector based on the reference key points;

[0014] A difference matrix is ​​constructed based on the reference point vector and the key point vector, and the motion vector is determined based on the difference matrix.

[0015] Preferably, determining the motion vector based on the difference matrix includes:

[0016] Calculate the median parameter and absolute deviation parameter between the matrix parameters in the first and second directions of the difference matrix, respectively;

[0017] Anomaly detection is performed on the matrix parameters based on the median parameter and the absolute deviation parameter to obtain abnormal parameters, and the abnormal parameters are then corrected.

[0018] Calculate the average value of the matrix parameters in the first direction after correction to obtain the first motion parameter, and calculate the average value of the matrix parameters in the second direction after correction to obtain the second motion parameter;

[0019] The motion vector is generated based on the first motion parameter and the second motion parameter.

[0020] Preferably, generating reference key points based on the object key points includes:

[0021] The object key points corresponding to the same key point identifier in different associated frames are combined to obtain a key point set, and the similarity between different object key points in the same key point set is calculated to obtain the key point similarity.

[0022] The object key points are determined based on the key point similarity, the key point weights in the key point set are determined, and the object key point corresponding to the key point with the largest key point weight is determined as the benchmark key point.

[0023] Preferably, the formula used to encode the current video frame based on the background frame and the motion vector includes:

[0024]

[0025] in, This refers to the segmented encoded stream. This represents the entropy coding function. Represents the quantization function. This represents the image block of the current video frame. Indicates a specified natural number. Represents vector weights, This represents the k-th motion vector. This represents the residual vector corresponding to the k-th motion vector. This represents the background frame corresponding to the k-th current video frame. Indicates encoding parameters;

[0026]

[0027] in, This represents the background frame corresponding to the i-th current video frame. This represents the i-th motion vector. This represents the sum of squares operation. This represents the natural exponential function. This represents the smoothing coefficient.

[0028] Preferably, background analysis is performed on the segmented video to obtain background frames, including:

[0029] The frame to be encoded is divided into time windows to obtain time windows, and the pixel variation coefficient between the time windows at the same position between different frames to be encoded is calculated.

[0030] The time window is modified according to the pixel variation coefficient to obtain the modified window, and the pixel weight is determined according to the pixel variation coefficient.

[0031] The background frame is obtained by weighting the correction window according to the pixel weights.

[0032] Preferably, the difference analysis is performed between the frame to be encoded in the segmented video and the corresponding background frame to obtain a difference image, including:

[0033] Calculate the pixel difference between the pixel blocks of the frame to be encoded and the background frame, and determine the difference blocks based on the pixel difference;

[0034] The difference image is generated based on the difference blocks.

[0035] Preferably, after encoding the current video frame based on the background frame and the motion vector to obtain a segmented encoded stream, the method further includes:

[0036] The motion deviation value is calculated based on the motion vector, and the difference between the motion deviation value between the current video frame and the previous video frame and the next video frame is calculated respectively to obtain the first deviation difference and the second deviation difference.

[0037] If the first deviation difference is greater than the first difference threshold and the second deviation difference is less than the second difference threshold, then the motion vector is determined to satisfy the background update condition.

[0038] Another objective of this invention is to provide a video coding system based on dynamic optimization of background frames, the system comprising:

[0039] The video segmentation module is used to acquire the video to be encoded and segment the video to be encoded to obtain a segmented video;

[0040] The difference analysis module is used to perform background analysis on the segmented video to obtain a background frame, and to perform difference analysis between the frame to be encoded in the segmented video and the corresponding background frame to obtain a difference image.

[0041] The motion analysis module is used to perform target detection on the difference image to obtain the target object, and to perform motion analysis on the target object to obtain the motion vector;

[0042] The video encoding module is used to encode the current video frame based on the background frame and the motion vector to obtain a segmented encoded stream;

[0043] The background dynamic optimization module is used to update the background frame according to the motion vector when the motion vector meets the background update condition; and to continue to perform the step of encoding the current video frame according to the background frame and the motion vector according to the updated background frame, until all video frames in the video to be encoded are encoded.

[0044] The encoding and combining module is used to combine the segmented encoded streams to obtain a video encoded stream.

[0045] Preferably, the motion analysis module is further configured to: in the same segmented video, set the frames to be encoded containing the target object as associated frames, and construct an association set based on the associated frames;

[0046] Key point identification is performed on the target object in different associated frames to obtain object key points, and a key point vector is generated based on the object key points;

[0047] Generate reference key points based on the object key points, and generate a reference point vector based on the reference key points;

[0048] A difference matrix is ​​constructed based on the reference point vector and the key point vector, and the motion vector is determined based on the difference matrix.

[0049] Another object of the present invention is to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0050] Another objective of this invention is to provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0051] In this embodiment of the invention, segmenting the video to be encoded facilitates background analysis of the segmented video. By performing background analysis on the segmented video, background frames corresponding to each segmented video can be effectively generated. By performing difference analysis between the frame to be encoded and the corresponding background frame, the difference image between the frame to be encoded and the background frame can be effectively determined. By performing motion analysis on the target object, the motion vector corresponding to the target object can be effectively obtained. Encoding the current video frame using the background frame and motion vector can effectively achieve the video encoding effect of the current video frame without repeatedly encoding the background in the current video frame, thus improving the video encoding efficiency. When the motion vector meets the background update condition, the background frame is updated using the motion vector, effectively achieving a dynamic optimization effect on the background frame. Based on the dynamically optimized background frame, the encoding step of the current video frame continues to be performed, effectively improving the accuracy of video encoding. Attached Figure Description

[0052] Figure 1 This is a flowchart of the video coding method based on dynamic optimization of background frames provided in the first embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the structure of a video coding system based on dynamic optimization of background frames provided in the second embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the third embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0056] To illustrate the technical solution described in this invention, specific embodiments are described below.

[0057] Example 1

[0058] Please see Figure 1 This is a flowchart of a video coding method based on dynamic optimization of background frames provided in the first embodiment of the present invention. This video coding method based on dynamic optimization of background frames can be applied to any device or system. The video coding method based on dynamic optimization of background frames includes the following steps:

[0059] Step S10: Obtain the video to be encoded and segment the video to be encoded to obtain a segmented video;

[0060] The process involves entity recognition of the video to be encoded, identifying moving entities in the video based on the entity recognition results, deleting the moving entities from the video frames of the video to be encoded, calculating the pixel difference between adjacent video frames after the removal of the moving entities, and segmenting the adjacent video frames if any pixel difference is greater than a preset difference, thus obtaining a segmented video.

[0061] Step S20: Perform background analysis on the segmented video to obtain background frames, and perform difference analysis on the frame to be encoded in the segmented video and the corresponding background frame to obtain a difference image;

[0062] By performing background analysis on the segmented video, background frames corresponding to each segmented video can be effectively generated. By performing difference analysis between the frame to be encoded and the corresponding background frame, the difference images between the frame to be encoded and the background frame can be determined. During the encoding process of the frame to be encoded, the encoding results of the background frame can be reused for the same background images between the frame to be encoded and the background frame, without the need for further encoding, thereby improving the video encoding efficiency.

[0063] Optionally, background analysis is performed on the segmented video to obtain background frames, including:

[0064] The frame to be encoded is segmented into time windows to obtain time windows, and the pixel variation coefficient between time windows at the same position between different frames to be encoded is calculated. The size and shape of the time windows can be set according to requirements. By calculating the pixel variation coefficient between time windows at the same position between different frames to be encoded, the pixel differences between images at the same position can be effectively analyzed. In this step, the window similarity between the time window and other time windows at the same position is calculated respectively. The total similarity is calculated based on the window similarity. The quotient of the total similarity between adjacent time windows is calculated to obtain the pixel variation coefficient.

[0065] The time window is modified according to the pixel variation coefficient to obtain a modified window, and the pixel weight is determined according to the pixel variation coefficient. If the pixel variation coefficient is greater than the variation threshold, the average window between the current time window and the adjacent time windows is calculated, and the average window is used to replace the current window to obtain the modified window. The pixel variation coefficient is numerically mapped according to a preset mapping relationship to obtain the pixel weight.

[0066] The background frame is obtained by weighting the correction window according to the pixel weights.

[0067] Further, performing difference analysis on the frame to be encoded in the segmented video and the corresponding background frame to obtain a difference image includes: calculating the pixel difference between the pixel blocks of the frame to be encoded and the background frame, determining the difference block based on the pixel difference, and generating the difference image based on the difference block, wherein if any pixel difference is greater than a pixel threshold, the pixel block corresponding to the pixel difference is determined as a difference block.

[0068] Step S30: Target detection is performed on the difference image to obtain the target object, and motion analysis is performed on the target object to obtain the motion vector;

[0069] Among these methods, by performing motion analysis on the target object, the motion vector corresponding to the target object can be effectively obtained. Optionally, performing motion analysis on the target object to obtain the motion vector includes:

[0070] In the same segmented video, the frame to be encoded containing the target object is set as an associated frame, and an association set is constructed based on the associated frame;

[0071] Key point identification is performed on the target object in different associated frames to obtain object key points, and key point vectors are generated based on the object key points; wherein, the target object is identified as an entity to obtain the entity type, the entity type is matched with an identification rule lookup table to obtain key point identification rules, key point identification is performed on the target object based on the key point identification rules to obtain object key points, the identification rule lookup table stores the correspondence between different entity types and corresponding key point identification rules, a key point matrix is ​​constructed based on the object key points, and the key point matrix is ​​vectorized to obtain key point vectors;

[0072] Based on the object's key points, reference key points are generated, and reference point vectors are generated based on the reference key points; wherein, by generating reference key points, the subsequent construction of reference point vectors and difference matrices is effectively facilitated;

[0073] A difference matrix is ​​constructed based on the reference point vector and the key point vector, and the motion vector is determined based on the difference matrix; wherein, the determination of the motion vector is effectively guaranteed by constructing the difference matrix using the reference point vector and the key point vector.

[0074] Further, generating reference key points based on the object key points includes:

[0075] The object key points corresponding to the same key point identifier in different associated frames are combined to obtain a key point set, and the similarity between different object key points in the same key point set is calculated to obtain the key point similarity.

[0076] The object key points are determined based on the key point similarity, the key point weights in the key point set are determined, and the object key point corresponding to the maximum key point weight is determined as the benchmark key point; wherein, the sum of the key point similarities between the object key points is calculated to obtain the key point weight.

[0077] Furthermore, determining the motion vector based on the difference matrix includes:

[0078] Calculate the median parameter and absolute deviation parameter between the matrix parameters in the first and second directions of the difference matrix, respectively;

[0079] The matrix parameters are anomaly detected based on the median parameter and the absolute deviation parameter to obtain anomaly parameters, and the anomaly parameters are corrected. Specifically, the parameter difference between the matrix parameters and the median parameter is calculated. If the parameter difference is greater than the absolute deviation parameter, the matrix parameter corresponding to the parameter difference is determined as an anomaly parameter. The anomaly parameter is corrected according to a preset correction value, which can be set according to requirements or the absolute deviation parameter.

[0080] The average value of the matrix parameters in the first direction after correction is calculated to obtain the first motion parameter. The average value of the matrix parameters in the second direction after correction is calculated to obtain the second motion parameter. The motion vector is then generated based on the first motion parameter and the second motion parameter.

[0081] Step S40: Encode the current video frame according to the background frame and the motion vector to obtain a segmented encoded stream;

[0082] Optionally, the formula used to encode the current video frame based on the background frame and the motion vector includes:

[0083]

[0084] in, This refers to the segmented encoded stream. This represents the entropy coding function. Represents the quantization function. This represents the image block of the current video frame. Indicates a specified natural number. Represents vector weights, This represents the k-th motion vector. This represents the residual vector corresponding to the k-th motion vector. This represents the vector of the k-th key point. Indicates encoding parameters;

[0085]

[0086] in, This represents the background frame corresponding to the i-th current video frame. This represents the i-th motion vector. This represents the sum of squares operation. This represents the natural exponential function. This represents the smoothing coefficient.

[0087] Optionally, after encoding the current video frame based on the background frame and the motion vector to obtain a segmented encoded stream, the method further includes:

[0088] The motion deviation value is calculated based on the motion vector, and the difference between the motion deviation value between the current video frame and the previous video frame and the next video frame is calculated respectively to obtain the first deviation difference and the second deviation difference; wherein, the motion vector is mapped according to the preset deviation relationship to obtain the motion deviation value;

[0089] If the first deviation difference is greater than the first difference threshold and the second deviation difference is less than the second difference threshold, then the motion vector is determined to meet the background update condition. The first and second difference thresholds can be set as needed. If the first deviation difference is greater than the first difference threshold and the second deviation difference is less than the second difference threshold, then it is determined that the motion deviation between the current video frame and the previous video frame is large, and the motion deviation between the current video frame and the next video frame is small. The background frame can be updated based on the motion vector in the current video frame to further reduce the motion deviation between the background frame and the next video frame, thereby improving video encoding efficiency.

[0090] Step S50: When the motion vector satisfies the background update condition, the background frame is updated according to the motion vector. Then, the step of encoding the current video frame according to the background frame and the motion vector is continued according to the updated background frame until all video frames in the video to be encoded are encoded.

[0091] In this process, motion vectors are input into a pre-trained vector decoder for vector decoding to obtain a motion image. Based on the key point positions corresponding to the motion vectors, the motion image is overlaid with a background frame to obtain an updated background frame. Updating the background frame through motion vectors effectively achieves dynamic optimization of the background frame. In this step, sample vectors are obtained and input into the vector decoder for vector decoding to obtain sample images. The loss is calculated based on the sample images and the standard image, and the parameters of the vector decoder are updated based on the loss until the vector decoder converges to obtain the pre-trained vector decoder.

[0092] Step S60: Combine the segmented encoded streams to obtain a video encoded stream.

[0093] In this embodiment, segmenting the video to be encoded facilitates background analysis of the segmented video. By performing background analysis on the segmented video, background frames corresponding to each segmented video can be effectively generated. By performing difference analysis between the frame to be encoded and the corresponding background frame, the difference image between the frame to be encoded and the background frame can be effectively determined. By performing motion analysis on the target object, the motion vector corresponding to the target object can be effectively obtained. Encoding the current video frame using the background frame and motion vector can effectively achieve the video encoding effect of the current video frame without repeatedly encoding the background in the current video frame, thus improving the video encoding efficiency. When the motion vector meets the background update condition, the background frame is updated using the motion vector, effectively achieving a dynamic optimization effect on the background frame. Based on the dynamically optimized background frame, the encoding step of the current video frame continues to be executed, effectively improving the accuracy of video encoding.

[0094] Example 2

[0095] Please see Figure 2 This is a schematic diagram of the structure of a video coding system 100 based on dynamic optimization of background frames provided in the second embodiment of the present invention, including:

[0096] The video segmentation module 10 is used to acquire the video to be encoded and segment the video to obtain a segmented video. Specifically, entity recognition is performed on the video to be encoded, and moving entity objects in the video are identified based on the entity recognition results. These moving entity objects are then deleted from the video frames of the video to be encoded. After the removal of the moving entity objects, the pixel difference between adjacent video frames is calculated. If any pixel difference is greater than a preset difference, the adjacent video frames are segmented to obtain a segmented video.

[0097] The difference analysis module 11 is used to perform background analysis on the segmented video to obtain background frames, and to perform difference analysis between the frame to be encoded in the segmented video and the corresponding background frame to obtain a difference image. Specifically, by performing background analysis on the segmented video, background frames corresponding to each segmented video can be effectively generated. By performing difference analysis between the frame to be encoded and the corresponding background frame, the difference image between the frame to be encoded and the background frame is determined. During the encoding process of the frame to be encoded, for the same background image between the frame to be encoded and the background frame, the encoding result of the background frame is reused, eliminating the need for further encoding, thereby improving video encoding efficiency.

[0098] Optionally, the difference analysis module 11 is further configured to: perform time window segmentation on the frame to be encoded to obtain time windows, and calculate the pixel variation coefficient between the time windows at the same position between different frames to be encoded;

[0099] The time window is modified according to the pixel variation coefficient to obtain the modified window, and the pixel weight is determined according to the pixel variation coefficient.

[0100] The background frame is obtained by weighting the correction window according to the pixel weights.

[0101] Furthermore, the difference analysis module 11 is also used to: calculate the pixel difference between the pixel blocks of the frame to be encoded and the background frame, determine the difference blocks based on the pixel difference, and generate the difference image based on the difference blocks.

[0102] The motion analysis module 12 is used to perform target detection on the difference image to obtain the target object, and to perform motion analysis on the target object to obtain the motion vector.

[0103] Optionally, the motion analysis module 12 is further configured to: in the same segmented video, set the frame to be encoded containing the target object as an associated frame, and construct an association set based on the associated frame;

[0104] Key point identification is performed on the target object in different associated frames to obtain object key points, and a key point vector is generated based on the object key points;

[0105] Generate reference key points based on the object key points, and generate a reference point vector based on the reference key points;

[0106] A difference matrix is ​​constructed based on the reference point vector and the key point vector, and the motion vector is determined based on the difference matrix.

[0107] Furthermore, the motion analysis module 12 is also used to: calculate the median parameter and absolute deviation parameter between the matrix parameters in the first and second directions of the difference matrix, respectively;

[0108] Anomaly detection is performed on the matrix parameters based on the median parameter and the absolute deviation parameter to obtain abnormal parameters, and the abnormal parameters are then corrected.

[0109] Calculate the average value of the matrix parameters in the first direction after correction to obtain the first motion parameter, and calculate the average value of the matrix parameters in the second direction after correction to obtain the second motion parameter;

[0110] The motion vector is generated based on the first motion parameter and the second motion parameter.

[0111] Furthermore, the motion analysis module 12 is also used to: combine the object key points corresponding to the same key point identifier in different associated frames to obtain a key point set, and calculate the similarity between different object key points in the same key point set to obtain key point similarity;

[0112] The object key points are determined based on the key point similarity, the key point weights in the key point set are determined, and the object key point corresponding to the key point with the largest key point weight is determined as the benchmark key point.

[0113] The video encoding module 13 is used to encode the current video frame based on the background frame and the motion vector to obtain a segmented encoded stream.

[0114] Optionally, the formula used to encode the current video frame based on the background frame and the motion vector includes:

[0115]

[0116] in, This refers to the segmented encoded stream. This represents the entropy coding function. Represents the quantization function. This represents the image block of the current video frame. Indicates a specified natural number. Represents vector weights, This represents the k-th motion vector. This represents the residual vector corresponding to the k-th motion vector. This represents the background frame corresponding to the k-th current video frame. Indicates encoding parameters;

[0117]

[0118] in, This represents the background frame corresponding to the i-th current video frame. This represents the i-th motion vector. This represents the sum of squares operation. This represents the natural exponential function. This represents the smoothing coefficient.

[0119] The background dynamic optimization module 14 is used to update the background frame according to the motion vector when the motion vector meets the background update condition; and to continue the step of encoding the current video frame according to the background frame and the motion vector, until all video frames in the video to be encoded are encoded. Specifically, the motion vector is input to a vector decoder for vector decoding to obtain a motion image. Based on the key point positions corresponding to the motion vector, the motion image is overlaid with the background frame to obtain an updated background frame. In this step, updating the background frame through the motion vector effectively achieves a dynamic optimization effect on the background frame.

[0120] Optionally, the background dynamic optimization module 14 is further configured to: calculate motion deviation values ​​based on the motion vector, and calculate the differences between the motion deviation values ​​between the current video frame and the previous video frame and the next video frame respectively, to obtain a first deviation difference and a second deviation difference;

[0121] If the first deviation difference is greater than the first difference threshold and the second deviation difference is less than the second difference threshold, then the motion vector is determined to satisfy the background update condition.

[0122] The encoding combination module 15 is used to combine the segmented encoded streams to obtain a video encoded stream.

[0123] In this embodiment, segmenting the video to be encoded facilitates background analysis of the segmented video. By performing background analysis on the segmented video, background frames corresponding to each segmented video can be effectively generated. By performing difference analysis between the frame to be encoded and the corresponding background frame, the difference image between the frame to be encoded and the background frame can be effectively determined. By performing motion analysis on the target object, the motion vector corresponding to the target object can be effectively obtained. Encoding the current video frame using the background frame and motion vector can effectively achieve the video encoding effect of the current video frame without repeatedly encoding the background in the current video frame, thus improving the video encoding efficiency. When the motion vector meets the background update condition, the background frame is updated using the motion vector, effectively achieving a dynamic optimization effect on the background frame. Based on the dynamically optimized background frame, the encoding step of the current video frame continues to be executed, effectively improving the accuracy of video encoding.

[0124] Example 3

[0125] Figure 3 This is a structural block diagram of an electronic device 2 provided in the third embodiment of this application. For example... Figure 3As shown, the electronic device 2 in this embodiment includes a processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the processor 20, such as a program for a video coding method based on dynamic optimization of background frames. When the processor 20 executes the computer program 22, it implements the steps in the various embodiments of the video coding method based on dynamic optimization of background frames described above.

[0126] For example, the computer program 22 may be divided into one or more modules, which are stored in the memory 21 and executed by the processor 20 to complete this application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 22 in the electronic device 2. The electronic device may include, but is not limited to, the processor 20 and the memory 21.

[0127] The processor 20 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0128] The memory 21 can be an internal storage unit of the electronic device 2, such as a hard disk or memory. The memory 21 can also be an external storage device of the electronic device 2, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 21 can include both internal and external storage units of the electronic device 2. The memory 21 is used to store the computer program and other programs and data required by the electronic device. The memory 21 can also be used to temporarily store data that has been output or will be output.

[0129] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0130] If an integrated module is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. This computer-readable storage medium can be non-volatile or volatile. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the contents of a computer-readable storage medium may be appropriately added to or subtracted from the contents as required by the legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, a computer-readable storage medium may not include electrical carrier signals and telecommunication signals.

[0131] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A video coding method based on dynamic optimization of background frames, characterized in that, The method includes: Obtain the video to be encoded, and segment the video to be encoded to obtain a segmented video; The segmented video is subjected to background analysis to obtain background frames, and the frames to be encoded in the segmented video are compared with the corresponding background frames to obtain difference images. Target detection is performed on the difference image to obtain the target object, and motion analysis is performed on the target object to obtain the motion vector; The current video frame is encoded based on the background frame and the motion vector to obtain a segmented encoded stream, and the background frame is updated based on the motion vector when the motion vector satisfies the background update condition. The process of encoding the current video frame based on the updated background frame and the motion vector continues until all video frames in the video to be encoded have been encoded. The segmented encoded streams are combined to obtain a video encoded stream.

2. The video coding method based on dynamic optimization of background frames as described in claim 1, characterized in that, Motion analysis is performed on the target object to obtain motion vectors, including: In the same segmented video, the frame to be encoded containing the target object is set as an associated frame, and an association set is constructed based on the associated frame; Key point identification is performed on the target object in different associated frames to obtain object key points, and a key point vector is generated based on the object key points; Generate reference key points based on the object key points, and generate a reference point vector based on the reference key points; A difference matrix is ​​constructed based on the reference point vector and the key point vector, and the motion vector is determined based on the difference matrix.

3. The video coding method based on dynamic optimization of background frames as described in claim 2, characterized in that, Determining the motion vector based on the difference matrix includes: Calculate the median parameter and absolute deviation parameter between the matrix parameters in the first and second directions of the difference matrix, respectively; Anomaly detection is performed on the matrix parameters based on the median parameter and the absolute deviation parameter to obtain abnormal parameters, and the abnormal parameters are then corrected. Calculate the average value of the matrix parameters in the first direction after correction to obtain the first motion parameter, and calculate the average value of the matrix parameters in the second direction after correction to obtain the second motion parameter; The motion vector is generated based on the first motion parameter and the second motion parameter.

4. The video coding method based on dynamic optimization of background frames as described in claim 2, characterized in that, Generate baseline key points based on the object key points, including: The object key points corresponding to the same key point identifier in different associated frames are combined to obtain a key point set, and the similarity between different object key points in the same key point set is calculated to obtain the key point similarity. The object key points are determined based on the key point similarity, the key point weights in the key point set are determined, and the object key point corresponding to the key point with the largest key point weight is determined as the benchmark key point.

5. The video coding method based on dynamic optimization of background frames as described in claim 2, characterized in that, The formula used to encode the current video frame based on the background frame and the motion vector includes: in, This refers to the segmented encoded stream. This represents the entropy coding function. Represents the quantization function. This represents the image block of the current video frame. Indicates a specified natural number. Represents vector weights, This represents the k-th motion vector. This represents the residual vector corresponding to the k-th motion vector. This represents the background frame corresponding to the k-th current video frame. Indicates encoding parameters; in, This represents the background frame corresponding to the i-th current video frame. This represents the i-th motion vector. This represents the sum of squares operation. This represents the natural exponential function. This represents the smoothing coefficient.

6. The video coding method based on dynamic optimization of background frames as described in claim 1, characterized in that, Background analysis is performed on the segmented video to obtain background frames, including: The frame to be encoded is divided into time windows to obtain time windows, and the pixel variation coefficient between the time windows at the same position between different frames to be encoded is calculated. The time window is modified according to the pixel variation coefficient to obtain the modified window, and the pixel weight is determined according to the pixel variation coefficient. The background frame is obtained by weighting the correction window according to the pixel weights.

7. The video coding method based on dynamic optimization of background frames as described in claim 1, characterized in that, Perform difference analysis between the frame to be encoded in the segmented video and the corresponding background frame to obtain a difference image, including: Calculate the pixel difference between the pixel blocks of the frame to be encoded and the background frame, and determine the difference blocks based on the pixel difference; The difference image is generated based on the difference blocks.

8. The video coding method based on dynamic optimization of background frames as described in claim 1, characterized in that, After encoding the current video frame based on the background frame and the motion vector to obtain a segmented encoded stream, the process further includes: The motion deviation value is calculated based on the motion vector, and the difference between the motion deviation value between the current video frame and the previous video frame and the next video frame is calculated respectively to obtain the first deviation difference and the second deviation difference. If the first deviation difference is greater than the first difference threshold and the second deviation difference is less than the second difference threshold, then the motion vector is determined to satisfy the background update condition.

9. A video coding system based on dynamic optimization of background frames, characterized in that, The system includes: The video segmentation module is used to acquire the video to be encoded and segment the video to be encoded to obtain a segmented video; The difference analysis module is used to perform background analysis on the segmented video to obtain a background frame, and to perform difference analysis between the frame to be encoded in the segmented video and the corresponding background frame to obtain a difference image. The motion analysis module is used to perform target detection on the difference image to obtain the target object, and to perform motion analysis on the target object to obtain the motion vector; The video encoding module is used to encode the current video frame based on the background frame and the motion vector to obtain a segmented encoded stream; The background dynamic optimization module is used to update the background frame according to the motion vector when the motion vector meets the background update condition; and to continue to perform the step of encoding the current video frame according to the background frame and the motion vector according to the updated background frame, until all video frames in the video to be encoded are encoded. The encoding and combining module is used to combine the segmented encoded streams to obtain a video encoded stream.

10. The video coding system based on dynamic optimization of background frames as described in claim 9, characterized in that, The motion analysis module is also used for: In the same segmented video, the frame to be encoded containing the target object is set as an associated frame, and an association set is constructed based on the associated frame; Key point identification is performed on the target object in different associated frames to obtain object key points, and a key point vector is generated based on the object key points; Generate reference key points based on the object key points, and generate a reference point vector based on the reference key points; A difference matrix is ​​constructed based on the reference point vector and the key point vector, and the motion vector is determined based on the difference matrix.

Images