Video encoding method, system, device and storage medium based on image similarity

Abstract

The present invention relates to the technical field of video encoding, and discloses a video encoding method, system, device and storage medium based on image similarity, wherein the method determines whether the YUV image data is a new key frame by calculating the image similarity between each frame of YUV image data and the first key frame, and if so, uses the new key frame as the second key frame of the next image group, and determines whether it is necessary to dynamically adjust the exposure parameters of the second key frame according to the image difference between the first key frame and the second key frame; in response to the need, generates exposure parameters according to a preset algorithm, renders the second key frame according to the exposure parameters, and performs video encoding according to each image group and the corresponding rendered image data, and obtains the encoding file corresponding to each image group. The present invention dynamically adjusts the GOP and exposure parameters based on the similarity of the image, and ensures the video fluency and clarity while reducing the size of the encoding file.

Description

Technical Field

[0001] The present invention relates to the technical field of video encoding, and in particular to a video encoding method, system, device and storage medium based on image similarity. Background Art

[0002] In modern video communication and media entertainment industries, efficient video coding technology is the key to providing high-quality video content. With the popularity of multi-viewpoint video and high-resolution content, traditional video coding methods face many challenges, especially in bandwidth utilization and video quality control.

[0003] Traditional video coding technologies, such as H.264 or HEVC, although providing efficient compression algorithms, often lack flexibility when dealing with scene switching. Without accurate detection and processing of scene changes, the video encoder may not be able to insert I frames (i.e. key frames) in a timely manner, resulting in two completely unrelated scenes being incorrectly encoded as the same GOP (a group of continuous pictures), thus affecting compression efficiency and video quality. In addition, the coding strategy with a fixed GOP length fails to take into account the dynamic changes of video content, which is particularly evident in video streams with frequent dynamic scene changes. In most existing coding systems, the bit rate and quantization parameters are usually set before the start of encoding and remain unchanged throughout the encoding process. This static configuration method ignores the complexity of video content changes and cannot dynamically allocate appropriate resources for different scenes. In addition, due to different shooting scenes, the exposure parameters will also be different. Therefore, various exposure parameters need to be adjusted for video switching scenes. This adjustment requires people familiar with coding technology to slice and import parameters, which increases manual labor and labor costs. In addition, due to limited GPUs, it is often necessary to wait for the previous task to end before the next task can be performed, resulting in low coding efficiency. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a video encoding method, system, device and storage medium based on image similarity, which dynamically adjusts GOP parameters through image similarity and dynamically adjusts image exposure parameters through image difference, so as to achieve the technical effect of reducing video encoding complexity and improving video encoding efficiency.

[0005] In a first aspect, the present invention provides a video encoding method based on image similarity, the method comprising:

[0006] Acquire video data to be encoded, convert the video data into YUV image data, and use the first frame of YUV image data as the first key frame of the current image group;

[0007] By calculating the image similarity between each frame of YUV image data and the first key frame, determining whether the YUV image data is a new key frame, and if so, taking the new key frame as the second key frame of the next image group, until each image group corresponding to the video data is obtained;

[0008] Convert the video data into RGB image data to obtain RGB image data corresponding to the second key frame of each image group, and determine whether it is necessary to dynamically adjust the exposure parameter of the second key frame according to the image difference between the first key frame and the second key frame;

[0009] In response to the need for dynamic adjustment, an exposure parameter is generated according to a preset algorithm, image rendering is performed on the RGB image data corresponding to the second key frame according to the exposure parameter, and video encoding is performed according to each image group and the corresponding rendered RGB image data to obtain an encoded file corresponding to each image group;

[0010] In response to the fact that dynamic adjustment is not required, video encoding is performed according to each image group and the corresponding GRB image data to obtain an encoded file corresponding to each image group;

[0011] Each encoding file is encapsulated to obtain a video encoding file corresponding to the video data to be encoded.

[0012] Furthermore, the step of determining whether the YUV image data is a new key frame by calculating the image similarity between each frame of YUV image data and the first key frame includes:

[0013] Using an MSE algorithm, a first similarity between each component of the YUV image data and each component of the first key frame is calculated, wherein the components include a Y component, a U component, and a V component;

[0014] Using the SSIM algorithm, calculating a second similarity between the YUV image data and the first key frame;

[0015] If the first similarity of any component is greater than a first similarity threshold, or the second similarity is greater than the second similarity threshold, the YUV image data is used as a new key frame.

[0016] Furthermore, the step of determining whether it is necessary to dynamically adjust the exposure parameters of the second key frame based on the image difference between the first key frame and the second key frame includes:

[0017] Acquire a first average brightness and a first contrast of the first key frame, a second average brightness and a second contrast of the second key frame, and a covariance between the first key frame and the second key frame;

[0018] Determining whether it is necessary to dynamically adjust the Y component of the second key frame according to a comparison relationship between a brightness difference between the second average brightness and the first average brightness and a brightness threshold;

[0019] Determining whether it is necessary to dynamically adjust the U component of the second key frame according to a comparison relationship between a contrast difference between the second contrast and the first contrast and a contrast threshold;

[0020] According to the comparison relationship between the covariance and the covariance threshold, it is determined whether it is necessary to dynamically adjust the V component of the second key frame.

[0021] Furthermore, the step of generating exposure parameters according to a preset algorithm includes:

[0022] Based on an image enhancement algorithm, component parameters of each component of the second key frame are calculated to obtain a component linear relationship;

[0023] In response to the need to dynamically adjust the Y component of the second key frame, generating an adjustment parameter of the Y component according to the brightness difference, and generating adjustment parameters of the U component and the V component according to the linear relationship of the components;

[0024] In response to the need to dynamically adjust the U component of the second key frame, generating an adjustment parameter of the U component according to the contrast difference, and generating adjustment parameters of the Y component and the V component according to the component linear relationship;

[0025] In response to the need to dynamically adjust the V component of the second key frame, generating an adjustment parameter of the V component according to the covariance, and generating adjustment parameters of the Y component and the U component according to the component linear relationship;

[0026] The adjustment parameters of each component are taken together as the exposure parameters.

[0027] Furthermore, the step of calculating the component parameters of each component of the second key frame based on the image enhancement algorithm to obtain the linear relationship of the components includes:

[0028] Based on the image enhancement algorithm, a reflection image model is constructed, and component parameters of each component of the second key frame are input into the reflection image model to obtain reflection image components corresponding to each component;

[0029] The greatest common divisor among the reflection image components is calculated, each reflection image component is expressed as the product of the greatest common divisor and the corresponding component coefficient, and the component linear relationship among the components is determined according to the component coefficients.

[0030] Furthermore, the reflection image model is expressed by the following formula:

[0031] Log[R(x,y)]=Log[S(x,y)]-Log[S(x,y)*G(x,y)]

[0032] Where R(x,y) represents the reflected image, S(x,y) represents the original image, G(x,y) represents the Gaussian kernel, and x and y represent the coordinates of the point in the image;

[0033] The reflected image component is expressed by the following formula:

[0034] LogRY=A*GCD

[0035] LogRU=B*GCD

[0036] LogRV=C*GCD

[0037] Wherein, LogRY, LogRU and LogRV represent the reflected image components of Y component, U component and V component respectively, GCD represents the greatest common divisor, A, B, C represent the component coefficients of Y component, U component and V component respectively.

[0038] Furthermore, before the step of converting the video data into RGB image data, the method further includes:

[0039] Based on the number of distributed devices, each image group of the video data is sliced, and the sliced ​​image groups and corresponding exposure parameters are sent to the corresponding distributed devices, so that the distributed devices perform video encoding on the sliced ​​image groups.

[0040] In a second aspect, the present invention provides a video encoding system based on image similarity, the system comprising:

[0041] A data conversion module, used for acquiring video data to be encoded, converting the video data into YUV image data, and using the first frame of YUV image data as the first key frame of the current image group;

[0042] a key frame determination module, configured to determine whether the YUV image data is a new key frame by calculating the image similarity between each frame of YUV image data and the first key frame, and if so, use the new key frame as the second key frame of the next image group until each image group corresponding to the video data is obtained;

[0043] a dynamic adjustment determination module, configured to convert the video data into RGB image data, obtain RGB image data corresponding to the second key frame of each image group, and determine whether it is necessary to dynamically adjust the exposure parameters of the second key frame according to the image difference between the first key frame and the second key frame;

[0044] a video encoding module, configured to generate exposure parameters according to a preset algorithm in response to a need for dynamic adjustment, perform image rendering on the RGB image data corresponding to the second key frame according to the exposure parameters, and perform video encoding according to each image group and the corresponding rendered RGB image data to obtain an encoding file corresponding to each image group;

[0045] In response to the fact that dynamic adjustment is not required, video encoding is performed according to each image group and the corresponding GRB image data to obtain an encoded file corresponding to each image group;

[0046] The file encapsulation module is used to encapsulate various coded files to obtain a video coded file corresponding to the video data to be encoded.

[0047] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0048] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0049] The present invention provides a video encoding method, system, device and storage medium based on image similarity. The present invention dynamically adjusts GOP based on image similarity, which can reduce the file size while ensuring the video fluency and clarity. It determines whether the exposure parameters need to be dynamically adjusted through the difference in images, and provides a method for generating the exposure parameters, which further ensures the effect of video encoding. At the same time, through the local area network mode, parallel image enhancement and parallel encoding technology are performed, which can effectively reduce the difficulty and complexity of large-scale video encoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a schematic flow chart of a video encoding method based on image similarity in an embodiment of the present invention;

[0051] Figure 2 is a structural schematic diagram of a video encoding system based on image similarity in an embodiment of the present invention;

[0052] Figure 3 It is a diagram of the internal structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] See also Figure 1 The first embodiment of the present invention provides a video encoding method based on image similarity, which includes steps S10 to S60:

[0055] Step S10, obtaining video data to be encoded, converting the video data into YUV image data, and using the first frame of YUV image data as the first key frame of the current image group;

[0056] Step S20, determining whether the YUV image data is a new key frame by calculating the image similarity between each frame of YUV image data and the first key frame, and if so, using the new key frame as the second key frame of the next image group until each image group corresponding to the video data is obtained;

[0057] Step S30, converting the video data into RGB image data, obtaining RGB image data corresponding to the second key frame of each image group, and judging whether it is necessary to dynamically adjust the exposure parameter of the second key frame according to the image difference between the first key frame and the second key frame;

[0058] Step S40, in response to the need for dynamic adjustment, generating exposure parameters according to a preset algorithm, performing image rendering on the RGB image data corresponding to the second key frame according to the exposure parameters, and performing video encoding according to each image group and the corresponding rendered RGB image data to obtain an encoded file corresponding to each image group;

[0059] Step S50, in response to the fact that dynamic adjustment is not required, video encoding is performed according to each image group and the corresponding GRB image data to obtain an encoded file corresponding to each image group;

[0060] Step S60, encapsulating each encoding file to obtain a video encoding file corresponding to the video data to be encoded.

[0061] Before describing the video encoding method provided by the present invention, the core parameters in the existing video encoding technology are briefly described first. In video encoding technology, the video is organized into a series of GOP groups. Each GOP group starts with an I frame, followed by multiple P frames and B frames. The structure and size of the GOP group have an important impact on the compression efficiency and decoding performance of the video. In the video encoding sequence, GOP, namely Group of picture, refers to the distance between two I frames, and Reference refers to the distance between two P frames. A group of pictures is formed between two I frames, which is GOP. I frame, also known as intracode frame or key frame, is a frame completely independent of other frames. It contains complete image information and does not rely on other frames for decoding. I frame acts as a reference point in the video stream, and other frames (P frames and B frames) can be decoded based on I frame. P frame, also known as predictive coding frame, is a frame that depends on the previous I frame or P frame. P frame reduces the amount of data through prediction and motion compensation technology, and only stores the difference information between the previous reference frame. This makes the file size of P frames much smaller than that of I frames, but the decoding needs to rely on reference frames. B frames, also known as bidirectional predictive coding frames, are frames that use two reference frames before and after to predict the content of the current frame. B frames can provide higher compression efficiency than P frames because they can use information in two directions to reduce the amount of data. However, the decoding complexity of B frames is also higher because they need to access the two reference frames before and after at the same time. Therefore, the encoding frame rate of the video can be encoded in a form similar to IBBPBPBBI, with P frames filling between I frames, and B frames can be derived forward or backward from P frames or I frames; and I frames account for more than three quarters of the data volume. How to reduce the number of I frames directly determines the size of the file. Most of the current video encoding technologies use a fixed GOP length for encoding, such as arranging I frames according to the standard GOP=30, but this cannot meet the dynamic scene changes of the video. Based on this, the present invention automatically adjusts the GOP parameters by comparing the similarity of the images, and by dynamically adjusting the number of I frames, while reducing the size of the video file, the image effect of the video is guaranteed.

[0062] In this embodiment, the similarity of the image is realized based on the YUV image data of the video data. YUV is an image color encoding method, in which the Y component represents brightness, the U component represents chroma, and the V component represents saturation. Specifically, the RAW data of the video is first acquired in seconds, and each frame of RAW data within each second is converted into YUV image data. That is to say, the present invention performs image conversion on the video data within each second, and then uses the first frame of YUV image data as the first key frame of the current image group, that is, the I frame of the current GOP. Then, the image similarity between each frame of image data and the I frame is calculated to determine whether the frame of image data is the I frame of the next GOP. The specific judgment steps include:

[0063] Using an MSE algorithm, a first similarity between each component of the YUV image data and each component of the first key frame is calculated, wherein the components include a Y component, a U component, and a V component;

[0064] Using the SSIM algorithm, calculating a second similarity between the YUV image data and the first key frame;

[0065] If the first similarity of any component is greater than a first similarity threshold, or the second similarity is greater than the second similarity threshold, the YUV image data is used as a new key frame.

[0066] In this embodiment, the similarity of the image includes a first similarity and a second similarity. Specifically, the first similarity between the image data and the key frame is calculated by the MES algorithm. MES refers to the expected value between the parameter estimation value and the parameter true value. In the image processing algorithm, it is the mean of the sum of squares of the difference between the processed image pixel value and the original pixel value. The formula is as follows:

[0067]

[0068] Where M and N represent the length and width of the image, respectively, and f′(i,j) and f(i,j) represent the image to be evaluated and the original image, respectively.

[0069] Since the image data is YUV image data, in this embodiment, the MSE values ​​of the three YUV components, namely Y-MSE, U-MSE, and V-MSE, are calculated, and then the MSE values ​​of these three components are compared with the preset MSE thresholds. If they are lower than the threshold, it means that the motion difference between the current image and the key frame is not large. If they are higher than the threshold, it means that the motion difference between the current image and the key frame is large.

[0070] The second similarity in this embodiment is calculated by the SSIM algorithm, that is, the second similarity between the current image and the key frame is calculated by the SSIM algorithm. The SSIM algorithm measures the similarity between images by comparing the brightness, contrast and structural information of the two images. Its value range is between [-1, 1], 1 means completely identical, and -1 means completely different. The calculation formula of SSIM is:

[0071]

[0072] In the formula, x and y represent two images, μ represents the average brightness of the image, and σ 2 represents the contrast of the image, σ xy It represents the covariance of the two images, and C1 and C2 are constants used to avoid the situation where the denominator is zero.

[0073] By comparing the second similarity with a preset second similarity threshold, it is possible to determine whether the illumination of the image has changed. That is, the first similarity represents the motion difference between images, and the second similarity represents the illumination difference between images. Only when the motion and illumination are consistent can it be determined that the current video image is a static shot. In the case of static shooting, the I frame can be reduced, and a small number of P frames and B frames can be used to ensure the smoothness of the video, thereby reducing the size of the video file. That is to say, in the case of non-static shooting, the image data will be used as the key frame (I frame) of the next GOP.

[0074] Through the above steps, the similarity of each frame of image data is compared with the first I frame to determine which frame in the video data of this second has a relatively large difference and needs to be encoded by I frame, so all P frames and B frames are stored between the positions of the previous I frame, thus forming a GOP parameter. It can be seen that in the present invention, the GOP parameter of the video data is a dynamically adjusted parameter rather than a fixed length. This method can allocate appropriate resources for the changes in dynamic scenes, thereby ensuring the compression efficiency of the video while also improving the encoding quality of the video.

[0075] Furthermore, due to different shooting scenes, the exposure parameters may also be different. In order to improve the processing efficiency of the GPU and reduce the labor cost, the present invention provides a method for determining whether it is necessary to dynamically adjust the exposure parameters of the key frame according to the image difference on the basis of dynamically adjusting the GOP. The specific determination steps include:

[0076] Acquire a first average brightness and a first contrast of the first key frame, a second average brightness and a second contrast of the second key frame, and a covariance between the first key frame and the second key frame;

[0077] Determining whether it is necessary to dynamically adjust the Y component of the second key frame according to a comparison relationship between a brightness difference between the second average brightness and the first average brightness and a brightness threshold;

[0078] Determining whether it is necessary to dynamically adjust the U component of the second key frame according to a comparison relationship between a contrast difference between the second contrast and the first contrast and a contrast threshold;

[0079] According to the comparison relationship between the covariance and the covariance threshold, it is determined whether it is necessary to dynamically adjust the V component of the second key frame.

[0080] In the dynamic adjustment judgment of the exposure parameters in this embodiment, the relevant parameters of the image used can be the parameters used in the similarity calculation process using the above steps, that is, through the first similarity calculation step, the three YUV components of the second key frame can be extracted, and through the second similarity calculation step, the average brightness, contrast and covariance of the two key frames can be extracted. It should be noted that, in this embodiment, the I frames of the GOPs other than the first GOP are all represented by the second key frame, and the I frame of the first GOP is the first key frame.

[0081] In this embodiment, the exposure parameter refers to the adjustment parameter for the three components of YUV. Based on the above description, it can be known that for the YUV component, Y represents brightness, U represents chroma, and V represents saturation. Whether these three components need to be adjusted can be judged based on the average brightness, contrast, and covariance of the two images. Specifically, based on the threshold comparison, it is judged whether the average brightness difference between the two images is too large. If it is too large, it is necessary to adjust the brightness of the second key frame, that is, the Y component; based on the threshold comparison, it is judged whether the contrast difference between the two images is too large. If it is too large, it is necessary to adjust the chroma of the second key frame, that is, the U component; based on the threshold comparison, it is judged whether the covariance of the two images is too small. If it is too small, it is necessary to adjust the saturation of the second key frame, that is, the V component.

[0082] Through the above steps, it can be determined whether the I frame needs to dynamically adjust the exposure parameters. If no adjustment is required, the video segment can be encoded based on the GOP parameter according to the conventional encoding technology. If dynamic adjustment is required, the exposure parameters can be generated by the preset algorithm provided in this embodiment. The specific generation steps include:

[0083] Based on an image enhancement algorithm, component parameters of each component of the second key frame are calculated to obtain a component linear relationship;

[0084] In response to the need to dynamically adjust the Y component of the second key frame, generating an adjustment parameter of the Y component according to the brightness difference, and generating adjustment parameters of the U component and the V component according to the linear relationship of the components;

[0085] In response to the need to dynamically adjust the U component of the second key frame, generating an adjustment parameter of the U component according to the contrast difference, and generating adjustment parameters of the Y component and the V component according to the component linear relationship;

[0086] In response to the need to dynamically adjust the V component of the second key frame, generating an adjustment parameter of the V component according to the covariance, and generating adjustment parameters of the Y component and the U component according to the component linear relationship;

[0087] The adjustment parameters of each component are taken together as the exposure parameters.

[0088] In this embodiment, conventional parameter adjustments are mostly for a single component, but in fact the image includes three components of YUV. Adjusting only a certain component while ignoring the relationship between the components may result in poor overall exposure of the image. Based on this problem, this embodiment first analyzes the linear relationship between the components of the image through an image enhancement algorithm, and the specific steps include:

[0089] Based on the image enhancement algorithm, a reflection image model is constructed, and component parameters of each component of the second key frame are input into the reflection image model to obtain reflection image components corresponding to each component;

[0090] The greatest common divisor among the reflection image components is calculated, each reflection image component is expressed as the product of the greatest common divisor and the corresponding component coefficient, and the component linear relationship among the components is determined according to the component coefficients.

[0091] In this embodiment, the SSR image enhancement algorithm is first used to construct a reflection image model. The principle of the SSR algorithm is that a given image S(x, y) can be decomposed into two different images: a reflection image R(x, y) and an incident image L(x, y). The image S can be regarded as composed of the incident image R and the reflection image L. The incident light shines on the reflective object, and the reflected light is reflected by the reflective object and enters the human eye, which is the image seen by the human eye. The final image is expressed by the formula: S(x, y) = R(x, y) * L(x, y). Among them, x and y represent the point coordinates of the image.

[0092] Take the logarithm of the natural constant e in the above formula: Log[S(x,y)]=Log[R(x,y)]+Log[L(x,y)]; then shift it: Log[R(x,y)]=Log[S(x,y)]-Log[L(x,y)].

[0093] Since the complete value of the incident light L cannot be obtained, we can only multiply a calculated result image itself by a Gaussian kernel G(x,y) to obtain approximate data. The Gaussian kernel here can be determined according to the conventional SSR algorithm and will not be explained in detail here.

[0094] Through the above steps, we can get the expression of the reflection image model: Log[R(x,y)]=Log[S(x,y)]-Log[S(x,y)*G(x,y)]. From this formula, we can see that the reflection result of the current image is the current image minus the current image multiplied by the Gaussian kernel itself.

[0095] Then, based on the three YUV components, the corresponding reflected image components LogRY, LogRU, and LogRV are calculated respectively. Then, the greatest common divisor GCD between the three reflected image components is solved by the Euclidean algorithm, and the product of the greatest common divisor and the component coefficient is used to characterize the reflected image components. It can be obtained that: LogRY=A*GCD; LogRU=B*GCD; LogRV=C*GCD.

[0096] Through the above formula, it can be seen that the YUV components actually form a linear relationship based on A:B:C. Based on this linear relationship, when it is determined through the dynamic adjustment judgment method that a certain component needs to be adjusted, based on the linear relationship, the other two components are adaptively adjusted. For example, when the Y component needs to be adjusted due to brightness loss, the U component and the V component need to be adjusted accordingly according to the linear relationship. The exposure parameter dynamic adjustment method provided by this embodiment can ensure the quality of the image while reducing the I frame. It should be noted here that the adjustment steps for adjusting a certain component separately due to differences in brightness, contrast, etc. can refer to the conventional image exposure parameter adjustment method, and no excessive restrictions are made here.

[0097] After the exposure parameters of the key frame are determined, the RAW video data is first converted into a GRB image according to the GOP parameters and the corresponding exposure parameters, and the image is rendered based on the exposure parameters and GRB, and then conventional encoding technologies such as H264 / 265 encoding technology are used to implement video encoding.

[0098] Furthermore, in order to improve the efficiency of video encoding, in a preferred embodiment, the present invention adopts a distributed encoding method to encode video data through distributed devices. Specifically, first, the GOP parameters are sliced ​​according to the number of distributed devices, and the sliced ​​GOP and corresponding exposure parameters are sent to each distributed device. The distributed device downloads the RAW video data according to the sliced ​​GOP parameters and exposure parameters and converts it into an RGB image. Then, the image exposure rendering is performed according to the exposure parameters to obtain a new RGB image. The new RGB image is then passed to the encoding module inside the chip for H265 or H264 encoding. The first frame is I frame encoding, followed by P frame and B frame encoding. In this embodiment, the RGB image rendered by the GPU is directly encoded through the video encoding module built into the embedded chip, which consumes less energy than PC encoding. After all encoding work is completed, the locally encoded files are uploaded to the LAN server. After the server obtains all the encoded files, they are finally packaged into the final video encoding file, thereby achieving efficient parallel encoding.

[0099] The present embodiment provides a video encoding method based on image similarity. The present invention dynamically adjusts the GOP based on the similarity of the images, thereby reducing the file size while ensuring the video fluency and clarity. It determines whether the exposure parameters need to be dynamically adjusted based on the image differences, and provides a method for generating the exposure parameters, thereby further ensuring the effect of video encoding. At the same time, through the local area network mode, parallel image enhancement and parallel encoding technologies are performed, thereby effectively reducing the difficulty and complexity of large-scale video encoding, and reducing energy consumption and costs.

[0100] See also Figure 2 Based on the same inventive concept, a video encoding system based on image similarity is proposed in a second embodiment of the present invention, comprising:

[0101] The data conversion module 10 is used to obtain the video data to be encoded, convert the video data into YUV image data, and use the first frame of YUV image data as the first key frame of the current image group;

[0102] A key frame determination module 20 is used to determine whether the YUV image data is a new key frame by calculating the image similarity between each frame of YUV image data and the first key frame, and if so, use the new key frame as the second key frame of the next image group until each image group corresponding to the video data is obtained;

[0103] A dynamic adjustment determination module 30 is used to convert the video data into RGB image data, obtain RGB image data corresponding to the second key frame of each image group, and determine whether it is necessary to dynamically adjust the exposure parameters of the second key frame according to the image difference between the first key frame and the second key frame;

[0104] The video encoding module 40 is used to generate exposure parameters according to a preset algorithm in response to the need for dynamic adjustment, perform image rendering on the RGB image data corresponding to the second key frame according to the exposure parameters, and perform video encoding according to each image group and the corresponding rendered RGB image data to obtain an encoding file corresponding to each image group;

[0105] In response to the fact that dynamic adjustment is not required, video encoding is performed according to each image group and the corresponding GRB image data to obtain an encoded file corresponding to each image group;

[0106] The file encapsulation module 50 is used to encapsulate each encoding file to obtain a video encoding file corresponding to the video data to be encoded.

[0107] The technical features and technical effects of the video encoding system based on image similarity proposed in the embodiment of the present invention are the same as those of the method proposed in the embodiment of the present invention, and are not described in detail here. Each module in the above-mentioned video encoding system based on image similarity can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0108] In addition, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0109] See also Figure 3, an internal structure diagram of a computer device in an embodiment, the computer device can specifically be a terminal or a server. The computer device includes a processor, a memory, a network interface, a display and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a video encoding method based on image similarity is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0110] It can be understood by those skilled in the art that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have the same component arrangement.

[0111] In addition, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0112] In summary, the embodiments of the present invention propose a video encoding method, system, device and storage medium based on image similarity. The method obtains video data to be encoded, converts the video data into YUV image data, and uses the first frame of YUV image data as the first key frame of the current image group; determines whether the YUV image data is a new key frame by calculating the image similarity between each frame of YUV image data and the first key frame, and if so, uses the new key frame as the second key frame of the next image group until each image group corresponding to the video data is obtained; converts the video data into RGB image data, and obtains the RGB image data corresponding to the second key frame of each image group. According to the image difference between the first key frame and the second key frame, it is judged whether it is necessary to dynamically adjust the exposure parameters of the second key frame; in response to the need for dynamic adjustment, the exposure parameters are generated according to the preset algorithm, the RGB image data corresponding to the second key frame is rendered according to the exposure parameters, and video encoding is performed according to each image group and the corresponding rendered RGB image data to obtain the encoding file corresponding to each image group; in response to the need for dynamic adjustment, video encoding is performed according to each image group and the corresponding GRB image data to obtain the encoding file corresponding to each image group; each encoding file is encapsulated to obtain the video encoding file corresponding to the video data to be encoded. The present invention dynamically adjusts the GOP based on the similarity of the image, reduces the file size, and ensures the video fluency and clarity, determines whether it is necessary to dynamically adjust the exposure parameters according to the difference of the image, and provides a method for generating the exposure parameters, further ensuring the effect of video encoding, and at the same time, through the local area network mode, parallel image enhancement and parallel encoding technology are performed, effectively reducing the difficulty and complexity of large-scale video encoding, and reducing energy consumption and cost.

[0113] Each embodiment in this specification is described in a progressive manner, and the same or similar parts of each embodiment can be directly referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. It should be noted that the technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0114] The above-mentioned embodiments only express several preferred implementation modes of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in the technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be based on the protection scope of the claims.

Claims

1. A video encoding method based on image similarity, characterized in that: include: Acquire video data to be encoded, convert the video data into YUV image data, and use the first frame of YUV image data as the first key frame of the current image group; By calculating the image similarity between each frame of YUV image data and the first key frame, determining whether the YUV image data is a new key frame, and if so, taking the new key frame as the second key frame of the next image group, until each image group corresponding to the video data is obtained; Convert the video data into RGB image data to obtain RGB image data corresponding to the second key frame of each image group, and determine whether it is necessary to dynamically adjust the exposure parameter of the second key frame according to the image difference between the first key frame and the second key frame; In response to the need for dynamic adjustment, an exposure parameter is generated according to a preset algorithm, image rendering is performed on the RGB image data corresponding to the second key frame according to the exposure parameter, and video encoding is performed according to each image group and the corresponding rendered RGB image data to obtain an encoded file corresponding to each image group; In response to the fact that dynamic adjustment is not required, video encoding is performed according to each image group and the corresponding GRB image data to obtain an encoded file corresponding to each image group; Each encoding file is encapsulated to obtain a video encoding file corresponding to the video data to be encoded.

2. The video encoding method based on image similarity according to claim 1, characterized in that: The step of determining whether the YUV image data is a new key frame by calculating the image similarity between each frame of YUV image data and the first key frame comprises: Using an MSE algorithm, a first similarity between each component of the YUV image data and each component of the first key frame is calculated, wherein the components include a Y component, a U component, and a V component; Using the SSIM algorithm, calculating a second similarity between the YUV image data and the first key frame; If the first similarity of any component is greater than a first similarity threshold, or the second similarity is greater than the second similarity threshold, the YUV image data is used as a new key frame.

3. The video encoding method based on image similarity according to claim 1, characterized in that: The step of determining whether it is necessary to dynamically adjust the exposure parameters of the second key frame according to the image difference between the first key frame and the second key frame comprises: Acquire a first average brightness and a first contrast of the first key frame, a second average brightness and a second contrast of the second key frame, and a covariance between the first key frame and the second key frame; Determining whether it is necessary to dynamically adjust the Y component of the second key frame according to a comparison relationship between a brightness difference between the second average brightness and the first average brightness and a brightness threshold; Determining whether it is necessary to dynamically adjust the U component of the second key frame according to a comparison relationship between a contrast difference between the second contrast and the first contrast and a contrast threshold; According to the comparison relationship between the covariance and the covariance threshold, it is determined whether it is necessary to dynamically adjust the V component of the second key frame.

4. The video encoding method based on image similarity according to claim 3, characterized in that: The step of generating exposure parameters according to a preset algorithm comprises: Based on an image enhancement algorithm, component parameters of each component of the second key frame are calculated to obtain a component linear relationship; In response to the need to dynamically adjust the Y component of the second key frame, generating an adjustment parameter of the Y component according to the brightness difference, and generating adjustment parameters of the U component and the V component according to the linear relationship of the components; In response to the need to dynamically adjust the U component of the second key frame, generating an adjustment parameter of the U component according to the contrast difference, and generating adjustment parameters of the Y component and the V component according to the component linear relationship; In response to the need to dynamically adjust the V component of the second key frame, generating an adjustment parameter of the V component according to the covariance, and generating adjustment parameters of the Y component and the U component according to the component linear relationship; The adjustment parameters of each component are taken together as the exposure parameters.

5. The video encoding method based on image similarity according to claim 4, characterized in that: The step of calculating the component parameters of each component of the second key frame based on the image enhancement algorithm to obtain the linear relationship of the components includes: Based on the image enhancement algorithm, a reflection image model is constructed, and component parameters of each component of the second key frame are input into the reflection image model to obtain reflection image components corresponding to each component; The greatest common divisor among the reflection image components is calculated, each reflection image component is expressed as the product of the greatest common divisor and the corresponding component coefficient, and the component linear relationship among the components is determined according to the component coefficients.

6. The video encoding method based on image similarity according to claim 5, characterized in that: The reflection image model is expressed by the following formula: Log[R(x,y)]=Log[S(x,y)]-Log[S(x,y)*G(x,y)] Where R(x,y) represents the reflected image, S(x,y) represents the original image, G(x,y) represents the Gaussian kernel, and x and y represent the coordinates of the point in the image; The reflected image component is expressed by the following formula: LogRY=A*GCD LogRU=B*GCD LogRV=C*GCD Wherein, LogRY, LogRU and LogRV represent the reflected image components of Y component, U component and V component respectively, GCD represents the greatest common divisor, A, B, C represent the component coefficients of Y component, U component and V component respectively.

7. The video encoding method based on image similarity according to claim 1, characterized in that: Before the step of converting the video data into RGB image data, the method further includes: Based on the number of distributed devices, each image group of the video data is sliced, and the sliced ​​image groups and corresponding exposure parameters are sent to the corresponding distributed devices, so that the distributed devices perform video encoding on the sliced ​​image groups.

8. A video coding system based on image similarity, characterized in that: include: A data conversion module, used for acquiring video data to be encoded, converting the video data into YUV image data, and using the first frame of YUV image data as the first key frame of the current image group; a key frame determination module, configured to determine whether the YUV image data is a new key frame by calculating the image similarity between each frame of YUV image data and the first key frame, and if so, use the new key frame as the second key frame of the next image group until each image group corresponding to the video data is obtained; a dynamic adjustment determination module, configured to convert the video data into RGB image data, obtain RGB image data corresponding to the second key frame of each image group, and determine whether it is necessary to dynamically adjust the exposure parameters of the second key frame according to the image difference between the first key frame and the second key frame; a video encoding module, configured to generate exposure parameters according to a preset algorithm in response to a need for dynamic adjustment, perform image rendering on the RGB image data corresponding to the second key frame according to the exposure parameters, and perform video encoding according to each image group and the corresponding rendered RGB image data to obtain an encoding file corresponding to each image group; In response to the fact that dynamic adjustment is not required, video encoding is performed according to each image group and the corresponding GRB image data to obtain an encoded file corresponding to each image group; The file encapsulation module is used to encapsulate various coded files to obtain a video coded file corresponding to the video data to be encoded.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images