A method and apparatus for processing images in a video

Abstract

The application provides a method and device for processing images in a video, which can reduce brightness flicker of each frame image contained in the video. The method comprises the following steps: after brightness adjustment is performed on each frame image included in the video, a reference frame and a target frame to be processed are determined based on the shooting time of each frame image, the shooting time of the reference frame is earlier than the shooting time of the target frame; a first brightness of the reference frame and a second brightness of the target frame are determined; the type of the target frame is determined based on the first brightness and the second brightness; if the type of the target frame is an abnormal exposure frame, brightness adjustment processing is performed on the abnormal exposure frame, and the brightness adjustment processing comprises brightness correction processing and image enhancement processing.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for processing images in a video. Background Technology

[0002] To ensure the quality of the video footage, cameras use automatic exposure to adjust the brightness of each frame in the video when the ambient lighting changes.

[0003] When adjusting the brightness of individual frames in a video using automatic exposure, a gradual adjustment method is typically employed. The brightness of each frame or several frames is increased or decreased until one frame reaches the target brightness value; this frame can be called the target brightness frame. Furthermore, to improve the image display, after adjusting the brightness of any given frame, image enhancement processing can be performed on that frame based on the brightness information of images captured before it, after the initial brightness adjustment.

[0004] However, after the brightness of each frame in the video is adjusted using automatic exposure, multiple frames that act as a transition between the initial and target brightness frames are included. These multiple frames contain images with different brightness ranges. During the enhancement process of a particular frame, the brightness of each pixel in that frame is enhanced based on the brightness information of images captured before that frame and after brightness adjustment. In other words, different images may be enhanced using images with different brightness ranges. Therefore, after enhancing multiple frames, the brightness of these frames may fluctuate significantly, resulting in a flickering phenomenon of "bright and dark" brightness, which greatly degrades the quality of the video.

[0005] Therefore, how to reduce brightness flicker in each frame of a video while ensuring image enhancement, thereby improving the quality of the video, has become an urgent problem to be solved. Summary of the Invention

[0006] This application provides a method and apparatus for processing images in a video, which can reduce brightness flicker in each frame of the video.

[0007] Firstly, a method for processing images in a video is provided, including:

[0008] After adjusting the brightness of each frame in the video, the reference frame and the target frame to be processed are determined based on the shooting time of each frame. The shooting time of the reference frame is earlier than that of the target frame.

[0009] Determine the first brightness of the reference frame and the second brightness of the target frame;

[0010] The type of the target frame is determined based on the first brightness and the second brightness.

[0011] If the target frame is an exposure-abnormal frame, brightness adjustment processing is performed on the exposure-abnormal frame. Brightness adjustment processing includes brightness correction processing and image enhancement processing.

[0012] In one example, the type of the target frame is determined based on a first brightness and a second brightness, including:

[0013] Determine the absolute value of the difference between the first brightness and the second brightness;

[0014] If the absolute value is greater than or equal to the preset threshold, and the first brightness is greater than the second brightness, the target frame is determined to be an underexposed frame among the abnormal exposure frames;

[0015] If the absolute value is greater than or equal to the preset threshold, and the first brightness is less than the second brightness, the target frame is determined to be an overexposed frame among the abnormal exposure frames.

[0016] In one example, if the target frame is an underexposed frame, the brightness adjustment process for the abnormally exposed frame includes:

[0017] If the target frame includes RGB three-channel data, image enhancement processing is performed on the target frame based on the first brightness of the reference frame;

[0018] The brightness correction process is performed on the target frame by summing the absolute values ​​of the RGB three channels of each pixel in the target frame after image enhancement.

[0019] In one example, if the target frame is an overexposed frame, the brightness adjustment process for the abnormally exposed frame includes:

[0020] If the target frame includes RGB three-channel data, image enhancement processing is performed on the target frame based on the first brightness of the reference frame;

[0021] The brightness correction of the target frame is achieved by subtracting the absolute value of each pixel's RGB three-channel value from the image enhancement value.

[0022] In one example, if the target frame is an underexposed frame, the brightness adjustment process for the abnormally exposed frame includes:

[0023] If the target frame includes YUV three-channel data, the target frame is luminance corrected by summing the absolute values ​​of the Y channel values ​​of each pixel in the target frame.

[0024] Based on the first brightness of the reference frame, image enhancement processing is performed on the target frame after brightness correction.

[0025] In one example, if the target frame is an overexposed frame, the brightness adjustment process for the abnormally exposed frame includes:

[0026] If the target frame includes YUV three-channel data, the target frame is luminance corrected by subtracting the absolute value of the Y channel value of each pixel in the target frame.

[0027] Based on the first brightness of the reference frame, image enhancement processing is performed on the target frame after brightness correction.

[0028] In one example, the method also includes:

[0029] If the absolute value is less than the preset threshold, the target frame is determined to be a normal frame.

[0030] If the target frame is a normal frame, perform image enhancement processing on the target frame.

[0031] In one example, determining the first brightness of the reference frame includes:

[0032] Based on the row and column directions of the reference frame, the reference frame is divided into N image blocks, where N is an integer greater than 1;

[0033] The first brightness of the reference frame is determined based on the brightness of N image blocks of the reference frame and the position of each image block;

[0034] Determining the second brightness of the target frame includes:

[0035] Based on the row and column directions of the target frame, the target frame is divided into M image blocks, where M is an integer greater than 1;

[0036] The second brightness of the target frame is determined based on the brightness of M image blocks of the target frame and the position of each image block.

[0037] In one example, the first brightness of the reference frame is determined based on the brightness of N image blocks of the reference frame and the position of each image block, including:

[0038] Based on the position of each image block in the reference frame, the first weight coefficient corresponding to each image block is determined, and the sum of the first weight coefficients corresponding to N image blocks is 1.

[0039] Determine the first product of the brightness of each image block in the reference frame and the corresponding first weight coefficient. The brightness of each image block in the reference frame is the average brightness of each pixel in the image block.

[0040] Determine the first accumulated value of the first product corresponding to N image blocks in the reference frame, where the first accumulated value is the first brightness;

[0041] The second brightness of the target frame is determined based on the brightness of M image blocks and the position of each image block, including:

[0042] Based on the position of each image block in the target frame, the second weight coefficient corresponding to each image block is determined, and the sum of the second weight coefficients corresponding to M image blocks is 1.

[0043] The brightness of each image block in the target frame is determined by the second product of the corresponding second weight coefficient. The brightness of each image block in the target frame is the average brightness of each pixel in the image block.

[0044] Determine the second accumulated value of the second product corresponding to each of the M image blocks in the target frame. The second accumulated value is the second brightness.

[0045] Secondly, this application provides an image processing apparatus for a video, comprising:

[0046] The image acquisition module is used to adjust the brightness of each frame of the video and determine the reference frame and the target frame to be processed in each frame based on the shooting time of each frame. The shooting time of the reference frame is earlier than the shooting time of the target frame.

[0047] The image brightness determination module is used to determine the first brightness of the reference frame and the second brightness of the target frame;

[0048] The target frame type determination module is used to determine the type of the target frame based on the first brightness and the second brightness.

[0049] The brightness adjustment module is used to perform brightness adjustment processing on the exposure-abnormal frame if the target frame is an exposure-abnormal frame. The brightness adjustment processing includes brightness correction processing and image enhancement processing based on the first brightness and the second brightness, respectively.

[0050] In the above embodiments of this application, after adjusting the brightness of each frame in the video, in order to reduce brightness flicker caused by subsequent image enhancement of frames with different brightness ranges, this application first determines the reference frame and the target frame to be processed in each frame based on the shooting time of each frame. Then, based on the first brightness of the reference frame and the second brightness of the target frame, the type of the target frame is determined, so as to determine whether brightness correction processing is needed for the target frame according to the type of the target frame. If the type of the target frame is an exposure aberration frame, then brightness correction processing and image enhancement processing are performed on the target frame, so that the final target frame, while ensuring the image enhancement effect, avoids large brightness differences between different images and reduces brightness flicker by performing brightness correction processing.

[0051] In addition, the solution in this application only requires brightness correction processing for target frames with abnormal exposure, which can improve the efficiency of image processing.

[0052] Furthermore, due to the low computational complexity in the above embodiments, the image processing speed in the video captured by the camera in real time is relatively fast. While ensuring the image enhancement effect, it can also detect and correct brightness flicker in real time. Moreover, due to the low computational complexity, the performance requirements of the camera are low, making it applicable to various cameras and having good versatility. Attached Figure Description

[0053] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a schematic flowchart illustrating an exemplary embodiment of a video image processing method provided in this application.

[0055] Figure 2 This is a schematic diagram of image segmentation provided in an exemplary embodiment of this application;

[0056] Figure 3 This is a schematic diagram illustrating the correspondence between weight coefficients and image blocks provided in an exemplary embodiment of this application;

[0057] Figure 4 This is a schematic flowchart illustrating yet another example of a video image processing method provided in an exemplary embodiment of this application;

[0058] Figure 5 This is a schematic flowchart illustrating yet another example of a video image processing method provided in an exemplary embodiment of this application;

[0059] Figure 6 This is a schematic diagram of an image processing apparatus in a video provided in an exemplary embodiment of this application. Detailed Implementation

[0060] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0061] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0062] This application provides a method for processing images in a video, used to process each frame of images contained in the video. This method can be applied to various types of electronic devices.

[0063] For example, this method can be applied to a camera. In this case, during the video shooting process, the camera can process each frame of the video based on the method provided in this application to obtain a video with high image quality.

[0064] In another example, the electronic device may be a terminal device (such as a computer) connected to the camera, and after receiving the video captured by the camera, it processes each frame of the video based on the method provided in this application to obtain a video with high image quality.

[0065] Figure 1 This is a schematic flowchart illustrating a method for processing images in a video according to an exemplary embodiment of this application. Figure 1 As shown, the method for processing images in a video provided in this application embodiment includes the following steps:

[0066] S110, after adjusting the brightness of each frame image included in the video, based on the shooting time of each frame image, determines the reference frame and the target frame to be processed in each frame image, wherein the shooting time of the reference frame is earlier than the shooting time of the target frame.

[0067] The brightness adjustment of the image can be performed by means of automatic exposure or other methods. After the brightness adjustment, each frame of the image may exhibit brightness flickering. Therefore, the brightness flickering phenomenon can be reduced based on the method provided in the embodiments of this application.

[0068] In one example, based on the capture time of each frame, the reference frame and the target frame to be processed in each frame are determined, including:

[0069] First, determine the number of frames Q between the reference frame and the target frame to be processed, where Q is an integer greater than 0.

[0070] Then, the target frame to be processed is obtained from each frame of images that have undergone brightness adjustment. Typically, the target frame to be processed can be the image with the earliest shooting time among the unprocessed images. That is, in this embodiment, each frame of images in the video is processed one by one based on the shooting time of each frame. If the electronic device performing the method provided in this embodiment is a camera, then the target frame can be the image captured by the camera at the current moment.

[0071] Finally, based on the frame number Q and the target frame to be processed, reference frames are obtained from each frame image whose shooting time is earlier than the shooting time of the target frame.

[0072] In this application, Q is typically a preset fixed value. Of course, the value of Q can be adjusted during the execution of the method of this application embodiment. Generally, the smaller the value of Q, the closer the shooting time of the reference frame and the target frame is, the higher the accuracy of determining the type of the target frame in subsequent steps, and correspondingly, the better the processing effect of the target frame. Furthermore, since the shooting time of the reference frame and the target frame is closer, it is easier to determine the reference frame as quickly as possible.

[0073] For example, the value of Q is 1, meaning the reference frame is the frame preceding the target frame.

[0074] S120, determine the first brightness of the reference frame and the second brightness of the target frame.

[0075] In this embodiment of the application, the brightness of an image frame is determined based on the brightness of each pixel in the image. Therefore, the first brightness of the reference frame and the second brightness of the target frame can be determined based on the brightness of each pixel in the reference frame and the target frame, respectively.

[0076] Furthermore, in this application, the first brightness of the reference frame refers to the brightness of the reference frame after brightness adjustment, and the second brightness of the target frame refers to the brightness of the target frame after brightness adjustment. If the reference frame, after brightness adjustment, is further subjected to brightness adjustment processing according to the method provided in the embodiments of this application, then the first brightness of the reference frame is the brightness of the reference frame after brightness adjustment and before brightness adjustment processing.

[0077] In one example, determining the first brightness of the reference frame includes:

[0078] First, the reference frame is divided into N image blocks according to the row and column directions, where N is an integer greater than 1. Then, the first brightness of the reference frame is determined based on the brightness of the N image blocks and the position of each image block.

[0079] In one example, determining the second brightness of the target frame includes:

[0080] First, the target frame is divided into M image blocks according to the row and column directions, where M is an integer greater than 1. Then, the second brightness of the target frame is determined based on the brightness of the M image blocks and the position of each image block.

[0081] In the example above, the brightness of each image block is the average brightness of the pixels contained in that image block.

[0082] This application takes into account the difference in light transmittance between areas near the center of the lens and areas far from the center, resulting in different brightness levels in the image's center and corners. Therefore, in the example above, the overall brightness of the reference frame can be accurately determined based on the brightness of N image blocks and the position of each block. Similarly, the overall brightness of the reference frame can be accurately determined based on the brightness of M image blocks and the position of each block in the target frame.

[0083] As a further example, the number of N image patches in the reference frame in both the row direction and the column direction is even, and the number of M image patches in the target frame in both the row direction and the column direction is even, and N = M. Of course, in other examples, N and M can also be different positive integers, and this application does not limit this.

[0084] This application takes into account that the number of pixels in the row direction and the number of pixels in the column direction of the current image are both even. Therefore, the number of blocks in the row direction and the number of blocks in the column direction of the reference frame and the target frame are both even. The probability that even numbers can be divided by each other is much greater than the probability that even numbers can be divided by each other. This can make each image block contain the same number of pixels, thereby improving the rate of dividing the reference frame and the target frame into corresponding image blocks and further improving the efficiency of image processing in this application.

[0085] For example Figure 2 As shown, the reference frame and the target frame are divided into 8*16 image blocks respectively. Considering that the information of the image changes greatly in the row direction, this application can, for example, divide the reference frame and the target frame into twice the number of blocks in the row direction as the number of blocks in the column direction.

[0086] In one example, the first brightness of the reference frame is determined based on the brightness of N image blocks of the reference frame and the position of each image block, including:

[0087] First, based on the position of each image block in the reference frame, determine the first weight coefficient corresponding to each image block. The sum of the first weight coefficients corresponding to N image blocks is 1. Next, determine the first product of the brightness of each image block in the reference frame and the corresponding first weight coefficient. The brightness of each image block in the reference frame is the average brightness of each pixel within the image block. Then, determine the first accumulated value of the first product corresponding to each of the N image blocks in the reference frame, as shown in the following formula (1). The first accumulated value is the first brightness.

[0088] L a1 =L 11 *w 11 +L 12 *w 12 +…+L 1N *w 1N Formula (1)

[0089] Among them, L a1 L represents the first accumulated value, which is the first brightness of the reference frame. 11 L 12 ...L 1N This represents the brightness of each image block in the reference frame; correspondingly, w 11 w 12 ...w 1N This represents the first weighting coefficient corresponding to each image block in the reference frame.

[0090] like Figure 3 As shown, taking a reference frame divided into 128 blocks as an example, each image block in the reference frame is numbered 1, 2, or 3. Image blocks numbered 1 are closer to the center of the reference frame than image blocks with other numbers. Image blocks numbered 2 are distributed around image blocks numbered 1, and are closer to the center of the reference frame than image blocks numbered 3. Image blocks numbered 3 are distributed around image blocks numbered 2. The first weighting coefficient corresponding to image blocks numbered 1 is 1 / 65, the first weighting coefficient corresponding to image blocks numbered 2 is 1 / 130, and the first weighting coefficient corresponding to image blocks numbered 3 is 1 / 260.

[0091] In one example, the second brightness of the target frame is determined based on the brightness of M image blocks of the target frame and the position of each image block, including:

[0092] First, based on the position of each image block in the target frame, determine the second weight coefficient corresponding to each image block. The sum of the second weight coefficients corresponding to M image blocks is 1. Next, determine the second product of the brightness of each image block in the target frame and the corresponding second weight coefficient. The brightness of each image block in the target frame is the average brightness of each pixel within the image block. Then, determine the second accumulated value of the second product corresponding to the M image blocks in the target frame, as shown in the following formula (2). The second accumulated value is the second brightness.

[0093] L a2 =L 21 *w 21 +L 22 *w 22 +…+L 2M *w 2M Formula (2)

[0094] Among them, L a2 L represents the second accumulated value, i.e., the second brightness of the target frame. 11 L 12 ...L 1N This represents the brightness of each image patch in the target frame; correspondingly, w 21 w 22 ...w 2M This represents the second weighting coefficient corresponding to each image block in the target frame.

[0095] For example, taking the division of the target frame into 128 blocks as an example, the second weighting coefficients corresponding to each image block of the target frame are as described above. Figure 3 The explanation will not be repeated here.

[0096] In the example above, the weight coefficients corresponding to the image patches are determined based on their positions, such that the magnitude of the weight coefficients is related to the position of the corresponding image patch. Furthermore, since the brightness of the area near the center of the image differs from that of the areas near the four corners, the proportion of brightness of image patches at different positions in the overall image brightness varies. Therefore, multiplying the brightness of each image patch by its corresponding weight coefficient and then summing the results allows for a more accurate determination of the overall image brightness compared to directly averaging the brightness of all image patches.

[0097] For example, before determining the first brightness of the reference frame, if the reference frame includes RGB channel data, the data storage format of the reference frame is converted according to the following formula (3) to make the reference frame include YUV channel data. Then, the Y channel data of the reference frame is extracted to determine a new reference frame containing only Y channel data. Then, the first brightness is determined according to the new reference frame. The method for determining the first brightness is the same as the example above and will not be repeated here.

[0098] Y = 0.299*R + 0.587*G + 0.114*B (Formula 3)

[0099] For example, before determining the second brightness of the target frame, if the target frame includes RGB channel data, the data storage format of the target frame is converted according to formula (3) so that the target frame includes YUV channel data. Then, the Y channel data of the target frame is extracted to determine a new target frame containing only Y channel data. Then, the second brightness is determined based on the new target frame. The method for determining the second brightness is the same as the example above and will not be repeated here.

[0100] In the example above, calculating brightness based on the image's Y-channel data significantly reduces the amount of data compared to calculating brightness based on RGB-channel data, thus improving computational speed. Furthermore, since the image's Y-channel data represents pixel brightness, while the U-channel and V-channel data represent pixel chromaticity, calculating the overall brightness of the image based on the Y-channel data is more accurate. On the other hand, converting image data from different color spaces to the YUV domain and then calculating the overall brightness of the image based on the Y-channel data reduces system complexity.

[0101] S130, determine the type of the target frame based on the first brightness and the second brightness.

[0102] In this embodiment, the image type can be divided into normal frames and exposure-abnormal frames. Exposure-abnormal frames can include overexposed frames and underexposed frames. Overexposed frames are images that are overexposed due to factors such as strong ambient light, while underexposed frames are images that are underexposed due to factors such as insufficient ambient light.

[0103] S140, if the target frame is an exposure-abnormal frame, perform brightness adjustment processing on the exposure-abnormal frame. The brightness adjustment processing includes brightness correction processing and image enhancement processing.

[0104] An exposure-abnormal frame is an image with a sudden change in brightness, which may cause brightness flickering. Therefore, this application embodiment performs brightness adjustment processing on exposure-abnormal frames. Through brightness correction processing, the brightness of the exposure-abnormal frame can be corrected. Through image enhancement processing, image quality can be improved, ensuring the effectiveness of image enhancement processing on the target frame.

[0105] In one feasible design, the image enhancement processing applied in this application embodiment is an algorithm based on a mapping enhancement curve or a mapping enhancement table. When performing image enhancement processing on a certain frame of image, the mapping enhancement curve or mapping enhancement table is determined based on the brightness of historical images after brightness adjustment. The historical images are images taken before this frame of image. The mapping enhancement curve or mapping enhancement table is used to map the original brightness of the frame of image to the enhanced brightness indicated by the mapping enhancement curve or mapping enhancement table. This image enhancement processing may employ, but is not limited to, enhancement algorithms based on min-max normalization, histogram equalization, color constancy perception (Retinex theory), and neural networks, etc., and this application does not limit the specific algorithms used.

[0106] The video image processing method provided in this application embodiment can determine the type of the target frame based on the target frame and a reference frame whose shooting time is earlier than the target frame. If the type of the target frame is an exposure error frame, then brightness correction processing and image enhancement processing are performed on the exposure error frame. Through brightness correction processing, large brightness differences between different images can be avoided. Thus, while ensuring the effect of image enhancement processing of each frame, the brightness of the exposure error frame is corrected, the phenomenon of brightness flicker is reduced, and the picture quality of the video is improved.

[0107] Furthermore, the method provided in this application embodiment has low computational complexity and a simpler processing procedure, thus processing images quickly. When processing each frame of a video using this method, the processed image can be obtained quickly, with low image processing latency, high real-time performance, and low requirements for camera performance. Therefore, it is applicable to various types of cameras and has good versatility.

[0108] In step S120 of the above embodiment, the type of the target frame is determined based on the first brightness and the second brightness. In one example, this operation may include the following steps:

[0109] First, determine the absolute value of the difference between the first brightness and the second brightness. Then, if the absolute value is greater than or equal to a preset threshold and the first brightness is greater than the second brightness, the target frame is determined to be an underexposed frame among the exposure abnormal frames; if the absolute value is greater than or equal to the preset threshold and the first brightness is less than the second brightness, the target frame is determined to be an overexposed frame among the exposure abnormal frames.

[0110] The preset threshold can be set as needed, and this application does not impose any restrictions on it.

[0111] For example, the preset threshold is L T The absolute value D = |L a1 -L a2 |, if D≥L T And L a1 >La2 The target frame is the underexposed frame among the frames with abnormal exposure; if D≥L T And L a1 <L a2 The target frame is an overexposed frame among the frames with abnormal exposure.

[0112] In the above example, by measuring the absolute value of the difference between the brightness of the reference frame and the brightness of the target frame, and by measuring the relationship between the brightness of the reference frame and the brightness of the target frame, it is possible to accurately determine whether the target frame is an exposure-abnormal frame, and if the target frame is an exposure-abnormal frame, to accurately determine whether the target frame is an underexposed frame or an overexposed frame within the exposure-abnormal frames.

[0113] After determining that the target frame is an exposure-abnormal frame, the brightness of the target frame can be adjusted in different ways depending on whether the target frame is underexposed or overexposed.

[0114] In one example, if the target frame is an underexposed frame, the brightness adjustment process for the abnormally exposed frame includes:

[0115] If the target frame includes RGB three-channel data, firstly, image enhancement processing is performed on the target frame based on the first brightness of the reference frame. Then, brightness correction processing is performed on the target frame by summing the absolute values ​​of the RGB three-channel values ​​of each pixel in the image-enhanced target frame.

[0116] In other words, if the target frame is represented by RGB three-channel data, the target frame can be first enhanced and then its brightness corrected to avoid changes in the color tone of the target frame.

[0117] For example, image enhancement processing of the target frame includes:

[0118] Based on image enhancement algorithms, a mapping enhancement curve or mapping enhancement table is obtained according to the first brightness of the reference frame. The mapping enhancement curve or mapping enhancement table is used to map the second brightness of the target frame to the enhanced brightness. The image enhancement processing includes, but is not limited to, enhancement algorithms based on min-max normalization, histogram equalization, color constancy perception (Retinex theory), and neural networks.

[0119] In the example above, since the target frame is underexposed compared to the reference frame, during the brightness correction process, the absolute values ​​of the RGB three channels of each pixel in the target frame after image enhancement are summed to increase the brightness of the target frame and avoid brightness flickering caused by image enhancement after the target frame has undergone automatic exposure brightness adjustment.

[0120] To illustrate the advantages of this application in reducing brightness flicker, a specific example is disclosed below.

[0121] In this example, if after brightness adjustment, the brightness values ​​of each pixel in the first frame are between 20 and 160, the brightness values ​​of each pixel in the preceding frame are between 20 and 160, the brightness values ​​of each pixel in the second frame are between 0 and 100, and the brightness values ​​of each pixel in the following frame (i.e., the third frame) are also between 0 and 100, then the existing technology performs image enhancement processing on the frames with different brightness ranges after brightness adjustment using the following steps:

[0122] (1) The first mapping enhancement curve determined based on the brightness information of the previous frame indicates that the brightness value 20 is mapped to the brightness value 0, and the brightness value 160 is mapped to the brightness value 255. According to the first mapping enhancement curve, after image enhancement of the first frame, the brightness value range of the first frame image is adjusted to 0-255.

[0123] (2) The second mapping enhancement curve, determined based on the original brightness information of the first frame without image enhancement, is used to map the brightness value 20 to 0 and the brightness value 160 to 255. However, according to the second mapping enhancement curve, when performing image enhancement on the second frame, the brightness value 0 of the second frame is mapped to the brightness value 0, and the brightness value 100 can only be mapped to the brightness value 180. Therefore, the brightness value range of the second frame after image enhancement is 0-180.

[0124] After the second frame image is enhanced, the maximum value of the brightness range of the second frame is less than the maximum value of the brightness range of the first frame, so the second frame is darker than the first frame.

[0125] (3) The third mapping enhancement curve, determined based on the original brightness information of the second frame without image enhancement, is used to map the brightness value 0 to brightness value 0 and the brightness value 100 to brightness value 255. When performing image enhancement on the third frame image based on the third mapping enhancement curve, the brightness value 0 of the third frame image is mapped to brightness value 0 and the brightness value 100 is mapped to brightness value 255.

[0126] Therefore, the brightness value of the third frame after image enhancement ranges from 0 to 255, making it brighter than the second frame. This results in a flickering effect between the first, second, and third frames. In other words, after image enhancement processing of a brightness-adjusted image using existing technology, flickering will occur in each frame.

[0127] To address the aforementioned issues, the first frame is used as the reference frame, and the second frame as the target frame. The method described in the example above is applied to process the target frame. Taking the R-channel brightness of the reference and target frames as an example, the first brightness corresponding to the R-channel brightness of the reference frame is 160, and the second brightness corresponding to the R-channel brightness of the target frame is 100. After image enhancement processing, the second brightness of the target frame is mapped to 180. The absolute value of the difference between the second and first brightness is 60. This absolute value is greater than a preset threshold, and since the first brightness is greater than the second brightness, the target frame can be determined to be an underexposed frame among the exposure-abnormal frames, requiring brightness adjustment processing.

[0128] In this brightness adjustment process, image enhancement processing is first performed on the target frame, which includes RGB three-channel data. Then, brightness correction processing is performed on the target frame after image enhancement. After brightness correction of the second brightness of 180 after image enhancement, the second brightness is adjusted to 240, which is close to the brightness of 255 of the reference frame after image enhancement. Compared with the above problem, where the target frame with a maximum brightness of 180 differs greatly from the reference frame with a maximum brightness of 255, causing brightness flicker, the above embodiment of this application can reduce the occurrence of brightness flicker.

[0129] Furthermore, this application illustrates, through another example, that if the target frame is represented using RGB three-channel data, performing image enhancement processing on the target frame first, followed by brightness correction processing, can avoid changes in the target frame's hue. In this example, the brightness of the reference frame and the target frame are the same as in the example above.

[0130] In this example, the average values ​​of the RGB channels of the target frame after brightness adjustment are 10, 0, and 0, respectively, and the hue of the target frame after brightness adjustment is black. If brightness correction is performed on the target frame first, followed by image enhancement, the average values ​​of the RGB channels of the target frame after brightness correction are 20, 10, and 10, respectively, which means that the data of each channel is accumulated by 10. Taking the mapping enhancement curve as an example, which maps 0 to 0, 10 to 20, and 20 to 200, the average values ​​of the RGB channels of the target frame after brightness correction and image enhancement are 200, 100, and 100, respectively, and the hue is pink, which is significantly different from the original hue, meaning that the hue of the target frame has been significantly changed.

[0131] Conversely, after image enhancement of the target frame, the average values ​​of its RGB channels are 20, 0, and 0, respectively. After brightness correction, the average values ​​of its RGB channels are 30, 10, and 10, respectively, resulting in a hue close to black. It can be seen that if the target frame includes RGB data, performing image enhancement before brightness correction can avoid hue changes.

[0132] Furthermore, in the method provided in this application embodiment, if the sum of the absolute values ​​of the first channel values ​​of a pixel in the target frame after image enhancement processing is greater than 2^x - 1, then the first channel value of the pixel after brightness correction processing is determined to be 2^x - 1. The first channel is an R channel, a G channel, or a B channel. Here, x represents the number of bits used to store the image data, for example, x equals 8. In this example, by limiting the maximum value of the first channel value, abnormal storage of the first channel value due to data overflow can be prevented.

[0133] In one example, if the target frame is an overexposed frame, the brightness adjustment process for the abnormally exposed frame includes:

[0134] If the target frame includes RGB three-channel data, firstly, image enhancement processing is performed on the target frame based on the first brightness of the reference frame. Then, brightness correction processing is performed on the target frame by subtracting the absolute value of each pixel's RGB three-channel value from the image enhancement processing.

[0135] In other words, if the target frame is represented by RGB three-channel data, the target frame can be first enhanced and then its brightness corrected to avoid changes in the color tone of the target frame.

[0136] In the example above, since the target frame is an overexposed frame compared to the reference frame, during the brightness correction process, the absolute values ​​of the RGB three channels of each pixel in the target frame after image enhancement are subtracted respectively to reduce the brightness of the target frame and avoid brightness flickering of the target frame after automatic exposure due to image enhancement processing.

[0137] The following is a specific example illustrating how the absolute values ​​of the RGB three channels of each pixel in the target frame after image enhancement are subtracted to reduce the brightness of the target frame and avoid the brightness flickering effect that occurs after automatic exposure and image enhancement processing.

[0138] Taking the brightness of the R channel of the reference frame and the target frame as an example, the first brightness of the R channel of the reference frame is 100, which is mapped to 180 after image enhancement processing. The second brightness of the R channel of the target frame is 160. After image enhancement processing, the second brightness of the target frame is mapped to 255. It can be seen that the brightness of 255 after mapping the R channel of the reference frame is significantly different from the brightness of 180 after mapping the R channel of the target frame, resulting in a brightness flickering phenomenon.

[0139] To address the aforementioned issues, the method described in the example above is used to process the target frame. Since the absolute value of the difference between the second brightness and the first brightness is 60, which is greater than a preset threshold, and the second brightness being greater than the first brightness indicates that the target frame is an overexposed frame among the abnormal exposure frames, brightness adjustment processing of the target frame is required.

[0140] In this brightness adjustment process, image enhancement processing is first performed on the target frame, which includes RGB three-channel data. Then, brightness correction processing is performed on the enhanced target frame. After brightness correction of the target frame's R-channel brightness of 255 after image enhancement, the target frame's R-channel brightness is adjusted to 195, which is close to the reference frame's R-channel brightness of 180 after image enhancement. Compared to the aforementioned problem where a large difference between the target frame's maximum brightness of 255 and the reference frame's maximum brightness of 180 causes brightness flickering, the above embodiment of this application can reduce the occurrence of brightness flickering.

[0141] Furthermore, in the method provided in this application embodiment, if the absolute value of the first channel value of a pixel in the target frame after image enhancement processing is less than 0, then the first channel value of that pixel after brightness correction processing is determined to be 0. The first channel is an R channel, a G channel, or a B channel. In this example, by limiting the minimum value of the first channel value, it is possible to prevent abnormal storage of the first channel value due to data overflow.

[0142] In one example, if the target frame is an underexposed frame, the brightness adjustment process for the abnormally exposed frame includes:

[0143] If the target frame includes YUV three-channel data, the target frame is first luminance corrected by summing the absolute values ​​of the Y channel values ​​of each pixel. Then, based on the first luminance of the reference frame, the luminance-corrected target frame is subjected to image enhancement processing.

[0144] In other words, if the target frame is represented by YUV three-channel data, the target frame can be first luminance corrected and then image enhanced to ensure the effect of reducing luminance flicker.

[0145] In the above example, if the target frame includes YUV three-channel data and the target frame is an underexposed frame, the absolute value of the Y channel values ​​of each pixel in the target frame is summed to increase the brightness of the target frame and reduce the brightness flicker caused by the image enhancement.

[0146] The following is a specific example illustrating how the absolute values ​​of the Y channel of each pixel in the target frame are summed to increase the brightness of the target frame and reduce the brightness flickering effect caused by image enhancement.

[0147] In this example, after brightness adjustment, in different frames with varying brightness levels corresponding to the Y channel, the first brightness level of the reference frame's Y channel is 20, and the second brightness level of the target frame's Y channel is 10. A mapping enhancement curve obtained from the original brightness information of the reference frame is used to map the brightness value 10 to 100 and the brightness value 20 to 200. After image mapping enhancement processing, the first brightness level of the reference frame's Y channel is mapped to 200. If image mapping enhancement processing is directly applied to the target frame, the second brightness level of the target frame's Y channel, 10, is mapped to 100, which differs from the reference frame's brightness level of 200 after image mapping enhancement processing by 100, resulting in brightness flickering.

[0148] Using the method of the above embodiments of this application, since the absolute value of the difference between the second brightness and the first brightness is 10, which is greater than a preset threshold, and the first brightness is greater than the second brightness, it can be determined that the target frame is an underexposed frame in the exposure abnormal frame, and the brightness of the target frame needs to be adjusted.

[0149] In the brightness adjustment process, the target frame is first subjected to brightness correction processing, and then the brightness-corrected target frame is subjected to image enhancement processing. Specifically, after brightness correction, the second brightness corresponding to the Y channel of the target frame is adjusted to 20. After image enhancement processing of the brightness-corrected target frame, the second brightness mapping corresponding to the Y channel is 200, which is the same as the brightness of 200 corresponding to the Y channel of the reference frame after image mapping enhancement processing. Therefore, the above embodiments of this application can reduce the occurrence of brightness flicker.

[0150] Furthermore, this application illustrates, through another example, that if the target frame is represented using YUV three-channel data, the target frame is first subjected to brightness correction processing, and then image enhancement processing is performed on it to ensure the effect of reducing brightness flicker. In this example, the brightness of the reference frame and the target frame are the same as in the example above.

[0151] In this example, if the target frame is first image-enhanced and then brightness-corrected, the second brightness corresponding to the Y channel of the target frame is mapped to 100 after image enhancement, and then becomes 110 after brightness correction. This differs from the brightness of the reference frame's Y channel after image enhancement by 90. It can be seen that compared to the target frame without brightness correction (Y channel brightness 100 after image enhancement), which differs from the reference frame's Y channel brightness of 200 after image enhancement by 100, performing image enhancement on the target frame first and then brightness correction results in a poorer reduction in brightness flicker.

[0152] Conversely, by first performing brightness correction on the target frame, the second brightness of the Y channel of the target frame after brightness correction is 20. Then, after image enhancement processing, the brightness of the Y channel of the target frame is 200. At this point, the difference between the target frame and the reference frame after image enhancement is 0. It can be seen that for target frames containing YUV three-channel data, performing brightness correction before image enhancement effectively reduces brightness flicker.

[0153] In one example, if the target frame is an overexposed frame, the brightness adjustment process for the abnormally exposed frame includes:

[0154] If the target frame includes YUV three-channel data, the target frame is first luminance corrected by subtracting the absolute value of the Y channel value of each pixel. Then, image enhancement processing is performed on the luminance-corrected target frame.

[0155] In other words, if the target frame is represented by YUV three-channel data, the target frame can be first luminance corrected and then image enhanced to ensure the effect of reducing luminance flicker.

[0156] In the above example, if the target frame includes YUV three-channel data and the target frame is an overexposed frame, the absolute value of the Y channel value of each pixel in the target frame is subtracted to reduce the brightness of the target frame and reduce brightness flicker caused by image enhancement.

[0157] The following is a specific example illustrating how the absolute value of the Y channel value of each pixel in the target frame is subtracted to reduce the brightness of the target frame and reduce the brightness flickering effect caused by image enhancement.

[0158] In this example, after brightness adjustment, in different frames with varying brightness levels corresponding to the Y channel, the first brightness level of the reference frame's Y channel is 10, and the second brightness level of the target frame's Y channel is 20. A mapping enhancement curve obtained from the original brightness information of the reference frame is used to map the brightness value 10 to 100 and the brightness value 20 to 200. After image mapping enhancement processing, the first brightness level of the reference frame's Y channel is mapped to 100. If image mapping enhancement processing is directly applied to the target frame, the second brightness level of the target frame's Y channel, 20, is mapped to 200, which differs from the reference frame's brightness level of 100 after image mapping enhancement processing, resulting in brightness flickering.

[0159] Using the method of the above embodiments of this application, since the absolute value of the difference between the second brightness and the first brightness is 10, which is greater than a preset threshold, and the second brightness is greater than the first brightness, it can be determined that the target frame is an overexposed frame in the exposure abnormal frame, and the brightness of the target frame needs to be adjusted.

[0160] In the brightness adjustment process, the target frame is first subjected to brightness correction processing, and then the brightness-corrected target frame is subjected to image enhancement processing. Specifically, after brightness correction, the second brightness corresponding to the Y channel of the target frame is adjusted to 10. After image enhancement processing of the brightness-corrected target frame, the second brightness corresponding to the Y channel is mapped to 100, which is the same as the Y channel brightness of 100 in the reference frame after image mapping enhancement processing. Therefore, the above embodiments of this application can reduce the occurrence of brightness flicker.

[0161] In one example, the method also includes:

[0162] If the absolute value is less than a preset threshold, the target frame is determined to be a normal frame. If the target frame is a normal frame, image enhancement processing is performed on the target frame.

[0163] For example, the preset threshold is L T The absolute value D = |L a1 -L a2 |, if D < L T The target frame is a normal frame.

[0164] If the absolute value is less than a preset threshold, the brightness of the target frame is close to that of the reference frame, and image enhancement processing of the target frame will not cause brightness flicker. By distinguishing the type of target frame, brightness correction processing can be performed on target frames with abnormal exposure, while normal frames are not subjected to brightness correction processing, thereby improving the efficiency of image processing.

[0165] In the above embodiments of this application, after adjusting the brightness of each frame in the video, in order to reduce brightness flicker caused by subsequent image enhancement of frames with different brightness ranges, this application first determines the reference frame and the target frame to be processed in each frame based on the shooting time of each frame. The type of the target frame is determined according to the first brightness of the reference frame and the second brightness of the target frame, so as to determine whether brightness correction processing is needed for the target frame based on its type. That is, brightness correction processing is only required for target frames with abnormal exposure, which can improve the image processing efficiency. If the target frame is an abnormal exposure frame, brightness correction processing and image enhancement processing are performed on the target frame, so that the final target frame, while ensuring the image enhancement effect, reduces brightness flicker by performing brightness correction processing. Because the computational complexity of the above embodiments is low, it can detect and correct brightness flicker in real time while ensuring the image enhancement effect of video images captured by the camera, and it is applicable to various cameras, exhibiting good versatility.

[0166] Because calculating brightness based on the Y-channel data of an image significantly reduces the amount of data compared to calculating brightness based on RGB-channel data, the processing speed is improved. Therefore, this application also provides an embodiment for performing brightness correction processing on a target frame based on the Y-channel data of a reference frame and the Y-channel data of the target frame, such as... Figure 4 As shown:

[0167] First, the reference frame brightness L is determined based on the Y channel data of the reference frame. a1 Based on the Y channel data of the target frame, determine the brightness L of the target frame. a2 .

[0168] Then, based on the Y-channel data of the reference frame, determine the mapping enhancement curve or mapping enhancement table.

[0169] Next, based on the reference frame brightness L a1 and target frame brightness L a2 To determine whether the target frame is an exposure-abnormal frame.

[0170] Finally, if the target frame is an exposure-abnormal frame, the brightness of the exposure-abnormal frame is adjusted according to the mapping enhancement curve or the mapping enhancement table. The brightness adjustment process includes brightness correction processing and image enhancement processing.

[0171] For example, the reference frame brightness L is determined based on the Y channel data of the reference frame. a1 The process of mapping enhancement curves is as follows Figure 5 As shown:

[0172] First, based on the Y-channel data of the reference frame, a mapping enhancement curve or mapping enhancement table is determined using an image enhancement algorithm, so that the target frame can be image enhanced subsequently based on the mapping enhancement curve or mapping enhancement table.

[0173] Then, determine the average Y-channel luminance of each of the N image blocks in the reference frame, including L... 11、 L 12 ...L 1N Finally, obtain the weights corresponding to the N image patches, including w. 12 …..w 1N .

[0174] Finally, based on the average Y-channel luminance of each of the N image blocks and their corresponding weights, the Y-channel luminance L of the reference frame is calculated using formula (1). a1 .

[0175] Based on the Y channel data of the target frame, determine the brightness L of the target frame. a2 See the process. Figure 5 This will not be elaborated upon here.

[0176] Based on the above-described method for processing images in a video, this application also provides an apparatus for processing images in a video, such as... Figure 6 include:

[0177] The image acquisition module is used to adjust the brightness of each frame of the video and determine the reference frame and the target frame to be processed in each frame based on the shooting time of each frame. The shooting time of the reference frame is earlier than the shooting time of the target frame.

[0178] The image brightness determination module is used to determine the first brightness of the reference frame and the second brightness of the target frame;

[0179] The target frame type determination module is used to determine the type of the target frame based on the first brightness and the second brightness.

[0180] The brightness adjustment module is used to perform brightness adjustment processing on the exposure-abnormal frame if the target frame is an exposure-abnormal frame. The brightness adjustment processing includes brightness correction processing and image enhancement processing.

[0181] In one example, the target frame type determination module can determine the type of the target frame in the following way:

[0182] Determine the absolute value of the difference between the first brightness and the second brightness;

[0183] If the absolute value is greater than or equal to the preset threshold, and the first brightness is greater than the second brightness, the target frame is determined to be an underexposed frame among the abnormal exposure frames;

[0184] If the absolute value is greater than or equal to the preset threshold, and the first brightness is less than the second brightness, the target frame is determined to be an overexposed frame among the abnormal exposure frames.

[0185] In one example, if the target frame is an underexposed frame, the brightness adjustment module can adjust the brightness of the frame with abnormal exposure in the following way:

[0186] If the target frame includes RGB three-channel data, image enhancement processing is performed on the target frame based on the first brightness of the reference frame;

[0187] The brightness correction process is performed on the target frame by summing the absolute values ​​of the RGB three channels of each pixel in the target frame after image enhancement.

[0188] In one example, if the target frame is an overexposed frame, the brightness adjustment module can adjust the brightness of the abnormally exposed frame in the following way:

[0189] If the target frame includes RGB three-channel data, image enhancement processing is performed on the target frame based on the first brightness of the reference frame;

[0190] The brightness correction of the target frame is achieved by subtracting the absolute value of each pixel's RGB three-channel value from the image enhancement value.

[0191] In one example, if the target frame is an underexposed frame, the brightness adjustment module can adjust the brightness of the frame with abnormal exposure in the following way:

[0192] If the target frame includes YUV three-channel data, the target frame is luminance corrected by summing the absolute values ​​of the Y channel values ​​of each pixel in the target frame.

[0193] Based on the first brightness of the reference frame, image enhancement processing is performed on the target frame after brightness correction.

[0194] In one example, if the target frame is an overexposed frame, the brightness adjustment module can adjust the brightness of the abnormally exposed frame in the following way:

[0195] If the target frame includes YUV three-channel data, the target frame is luminance corrected by subtracting the absolute value of the Y channel value of each pixel in the target frame.

[0196] Based on the first brightness of the reference frame, image enhancement processing is performed on the target frame after brightness correction.

[0197] In one example, the brightness adjustment module is also used to determine the type of the target frame as a normal frame if the absolute value is less than a preset threshold; if the type of the target frame is a normal frame, the target frame is subjected to image enhancement processing.

[0198] In one example, the image brightness determination module can determine the first brightness of the reference frame in the following way:

[0199] Based on the row and column directions of the reference frame, the reference frame is divided into N image blocks, where N is an integer greater than 1;

[0200] The first brightness of the reference frame is determined based on the brightness of N image blocks of the reference frame and the position of each image block;

[0201] The image brightness determination module can determine the second brightness of the target frame in the following ways:

[0202] Based on the row and column directions of the target frame, the target frame is divided into M image blocks, where M is an integer greater than 1;

[0203] The second brightness of the target frame is determined based on the brightness of M image blocks of the target frame and the position of each image block.

[0204] In one example, the image brightness determination module can determine the first brightness of the reference frame based on the brightness of N image blocks and the position of each image block in the reference frame as follows:

[0205] Based on the position of each image block in the reference frame, the first weight coefficient corresponding to each image block is determined, and the sum of the first weight coefficients corresponding to N image blocks is 1.

[0206] Determine the first product of the brightness of each image block in the reference frame and the corresponding first weight coefficient. The brightness of each image block in the reference frame is the average brightness of each pixel in the image block.

[0207] Determine the first accumulated value of the first product corresponding to N image blocks in the reference frame, where the first accumulated value is the first brightness;

[0208] The image brightness determination module can determine the second brightness of the target frame based on the brightness of M image blocks and the position of each image block in the target frame in the following way:

[0209] Based on the position of each image block in the target frame, the second weight coefficient corresponding to each image block is determined, and the sum of the second weight coefficients corresponding to M image blocks is 1.

[0210] The brightness of each image block in the target frame is determined by the second product of the corresponding second weight coefficient. The brightness of each image block in the target frame is the average brightness of each pixel in the image block.

[0211] Determine the second accumulated value of the second product corresponding to each of the M image blocks in the target frame. The second accumulated value is the second brightness.

[0212] Other implementation methods and effects of this device are described in the image processing method in the video above, and will not be repeated here.

[0213] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0214] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0215] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0216] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0217] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0218] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for processing images in a video, characterized in that, include: After adjusting the brightness of each frame in the video, a reference frame and a target frame to be processed are determined based on the shooting time of each frame. The shooting time of the reference frame is earlier than the shooting time of the target frame. Determine the first brightness of the reference frame and the second brightness of the target frame; The type of the target frame is determined based on the first brightness and the second brightness; If the target frame is an exposure-abnormal frame, the exposure-abnormal frame is subjected to brightness adjustment processing, which includes brightness correction processing and image enhancement processing. The step of determining the type of the target frame based on the first brightness and the second brightness includes: Determine the absolute value of the difference between the first brightness and the second brightness; If the absolute value is greater than or equal to a preset threshold, and the first brightness is greater than the second brightness, the target frame is determined to be an underexposed frame among the exposure abnormal frames; If the absolute value is greater than or equal to a preset threshold, and the first brightness is less than the second brightness, the target frame is determined to be an overexposed frame among the exposure abnormal frames; If the target frame is an underexposed frame, the brightness adjustment process for the abnormally exposed frame includes: If the target frame includes RGB three-channel data, the image enhancement processing is performed on the target frame based on the first brightness of the reference frame; The brightness correction process is performed on the target frame by summing the absolute values ​​of the RGB three-channel values ​​of each pixel in the target frame after image enhancement processing. If the target frame is an underexposed frame, the brightness adjustment process for the abnormally exposed frame includes: If the target frame includes YUV three-channel data, the brightness correction process is performed on the target frame by summing the absolute values ​​of the Y channel values ​​of each pixel in the target frame. Based on the first brightness of the reference frame, the image enhancement processing is performed on the target frame after the brightness correction processing.

2. The method according to claim 1, characterized in that, If the target frame is an overexposed frame, the brightness adjustment process for the abnormally exposed frame includes: If the target frame includes RGB three-channel data, the image enhancement processing is performed on the target frame based on the first brightness of the reference frame; The brightness correction process is performed on the target frame by subtracting the absolute value from the RGB three-channel values ​​of each pixel in the target frame after image enhancement processing.

3. The method according to claim 1, characterized in that, If the target frame is an overexposed frame, the brightness adjustment process for the abnormally exposed frame includes: If the target frame includes YUV three-channel data, the brightness correction process is performed on the target frame by subtracting the absolute value from the Y channel value of each pixel of the target frame. Based on the first brightness of the reference frame, the image enhancement processing is performed on the target frame after the brightness correction processing.

4. The method according to claim 1, characterized in that, The method further includes: If the absolute value is less than the preset threshold, the target frame is determined to be a normal frame. If the target frame is a normal frame, perform image enhancement processing on the target frame.

5. The method according to any one of claims 1-4, characterized in that, Determining the first brightness of the reference frame includes: Based on the row and column directions of the reference frame, the reference frame is divided into N image blocks, where N is an integer greater than 1; The first brightness of the reference frame is determined based on the brightness of the N image blocks of the reference frame and the position of each image block; Determining the second brightness of the target frame includes: Based on the row and column directions of the target frame, the target frame is divided into M image blocks, where M is an integer greater than 1; The second brightness of the target frame is determined based on the brightness of the M image blocks of the target frame and the position of each image block.

6. The method according to claim 5, characterized in that, Determining the first brightness of the reference frame based on the brightness of the N image blocks of the reference frame and the position of each image block includes: Based on the position of each image block in the reference frame, a first weight coefficient corresponding to each image block is determined, and the sum of the first weight coefficients corresponding to the N image blocks is 1. The brightness of each image block in the reference frame is determined by a first product of the brightness of the corresponding first weight coefficient, wherein the brightness of each image block in the reference frame is the average brightness of each pixel in the image block; Determine the first accumulated value of the first product corresponding to the N image blocks in the reference frame, wherein the first accumulated value is the first brightness; Determining the second brightness of the target frame based on the brightness of the M image blocks and the position of each image block includes: Based on the position of each image block in the target frame, a second weight coefficient corresponding to each image block is determined, and the sum of the second weight coefficients corresponding to the M image blocks is 1. The brightness of each image block in the target frame is determined by a second product of the brightness and the corresponding second weighting coefficient, wherein the brightness of each image block in the target frame is the average brightness of each pixel within the image block; Determine the second accumulated value of the second product corresponding to the M image blocks in the target frame, where the second accumulated value is the second brightness.

7. An apparatus for processing images in a video, characterized in that, include: The image acquisition module is used to, after adjusting the brightness of each frame image included in the video, determine a reference frame and a target frame to be processed in each frame image based on the shooting time of each frame image, wherein the shooting time of the reference frame is earlier than the shooting time of the target frame. An image brightness determination module is used to determine the first brightness of the reference frame and the second brightness of the target frame; A target frame type determination module is used to determine the type of the target frame based on the first brightness and the second brightness; A brightness adjustment module is used to perform brightness adjustment processing on the exposure-abnormal frame if the target frame is an exposure-abnormal frame. The brightness adjustment processing includes brightness correction processing and image enhancement processing based on the first brightness and the second brightness, respectively. The step of determining the type of the target frame based on the first brightness and the second brightness includes: Determine the absolute value of the difference between the first brightness and the second brightness; If the absolute value is greater than or equal to a preset threshold, and the first brightness is greater than the second brightness, the target frame is determined to be an underexposed frame among the exposure abnormal frames; If the absolute value is greater than or equal to a preset threshold, and the first brightness is less than the second brightness, the target frame is determined to be an overexposed frame among the exposure abnormal frames; If the target frame is an underexposed frame, the brightness adjustment process for the abnormally exposed frame includes: If the target frame includes RGB three-channel data, the image enhancement processing is performed on the target frame based on the first brightness of the reference frame; The brightness correction process is performed on the target frame by summing the absolute values ​​of the RGB three-channel values ​​of each pixel in the target frame after image enhancement processing. If the target frame is an underexposed frame, the brightness adjustment process for the abnormally exposed frame includes: If the target frame includes YUV three-channel data, the brightness correction process is performed on the target frame by summing the absolute values ​​of the Y channel values ​​of each pixel in the target frame. Based on the first brightness of the reference frame, the image enhancement processing is performed on the target frame after the brightness correction processing.

Images