Video frame brightness interference determination method and related device

Abstract

The invention discloses a video frame brightness interference determination method and a related device, and the method comprises the steps: obtaining the statistical brightness of each first block in a current actual frame and the statistical brightness of each second block in a previous reference frame, enabling the first blocks to be in one-to-one correspondence with the second blocks, and enabling the first blocks to be in one-to-one correspondence with the second blocks; each first block and each second block are obtained by respectively dividing the current actual frame and the previous reference frame according to the same division mode; respectively acquiring a statistical brightness difference between the statistical brightness of each first block and the statistical brightness of the corresponding second block; based on the statistical brightness difference corresponding to each first block, the interference degree of the moving light source on each first block is determined, and the interference degree is in positive correlation with the statistical brightness difference. According to the scheme, the interference degree of the moving light source on the current actual frame can be determined.

Description

Technical Field

[0001] The present application relates to the technical field of video processing, and in particular, to a method for determining video frame brightness interference, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of technology, the application of video frame analysis technology is becoming more and more extensive. For example, video frame analysis technology can be applied to object detection, object recognition, etc.

[0003] The quality of a video frame is one of the main factors affecting the video frame analysis effect. The better the quality, the better the analysis effect.

[0004] In order to improve the quality of video frames, it is usually necessary to preprocess the video frames after they are captured by a camera device before using them for video frame analysis. The preprocessing techniques include Auto White Balance (AWB), Auto Exposure (AE), and ambient brightness estimation, etc. And these preprocessing techniques often need to consider the degree of interference of the video frame brightness. Therefore, there is an urgent need for a method for determining video frame brightness interference to determine the degree of interference of the video frame brightness. Summary of the Invention

[0005] The present application provides a method for determining video frame brightness interference, a device for determining video frame brightness interference, an electronic device, and a computer-readable storage medium, which can determine the degree of interference of the video frame brightness.

[0006] The present application provides a method for determining video frame brightness interference, including: obtaining the statistical brightness of each first block in the current actual frame and the statistical brightness of each second block in the previous reference frame, where the first blocks and the second blocks correspond one by one, and each first block and each second block are obtained by dividing the current actual frame and the previous reference frame respectively according to the same division method; respectively obtaining the statistical brightness difference between the statistical brightness of each first block and the statistical brightness of the corresponding second block; and determining the degree of interference of each first block by a moving light source based on the statistical brightness difference corresponding to each first block, where the degree of interference is positively correlated with the statistical brightness difference.

[0007] The present application provides a device for determining the brightness interference of video frames, including: a brightness acquisition module, a brightness difference acquisition module, and a determination module. The brightness acquisition module is configured to acquire the statistical brightness of each first sub-block in the current actual frame and the statistical brightness of each second sub-block in the previous reference frame. The first sub-blocks and the second sub-blocks are in one-to-one correspondence, and each first sub-block and each second sub-block are obtained by dividing the current actual frame and the previous reference frame respectively according to the same division method. The brightness difference acquisition module is configured to respectively acquire the statistical brightness difference between the statistical brightness of each first sub-block and the corresponding second sub-block. The determination module is configured to determine the interference degree of each first sub-block by a moving light source based on the statistical brightness difference corresponding to each first sub-block, where the interference degree is positively correlated with the statistical brightness difference.

[0008] The present application provides an electronic device, including a memory and a processor. The processor is configured to execute program instructions stored in the memory to implement the above method for determining the brightness interference of video frames.

[0009] The present application provides a computer-readable storage medium, on which program instructions are stored. When the program instructions are executed by a processor, the above method for determining the brightness interference of video frames is implemented.

[0010] In the above solution, the statistical brightness of each first sub-block in the current actual frame and the statistical brightness of each second sub-block in the previous reference frame are acquired, and the first sub-blocks and the second sub-blocks are in one-to-one correspondence; the statistical brightness difference between the statistical brightness of each first sub-block and the corresponding second sub-block is respectively acquired; based on the statistical brightness difference corresponding to each first sub-block, the interference degree of each first sub-block by a moving light source is determined, and the interference degree is positively correlated with the statistical brightness difference. Thus, on the one hand, through this method for determining brightness interference, the interference degree of the current actual frame by a moving light source can be determined, and this method for determining brightness interference is simple, easy to understand, and easy to implement. On the other hand, using the interference degrees of each first sub-block by a moving light source to jointly represent the interference degree of the current actual frame by a moving light source takes into account the differences in the interference degrees of different first sub-blocks in the current actual frame by a moving light source, and expresses the interference degree of the current actual frame by a moving light source more accurately. On the third hand, the unit for determining brightness interference is the first sub-block, which can improve the efficiency of determining brightness interference compared with the method where the unit for determining brightness interference is a pixel.

[0011] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings herein are incorporated into the specification and form a part of this specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to explain the technical solutions of the present application.

[0013] Figure 1It is a schematic flowchart of an embodiment of the method for determining video frame brightness interference provided by this application;

[0014] Figure 2 It is a schematic flowchart of an embodiment of the method for determining video frame brightness interference provided by this application;

[0015] Figure 3 It is a schematic flowchart of an embodiment of the method for determining video frame brightness interference provided by this application;

[0016] Figure 4 It is a schematic flowchart of an embodiment of the method for determining video frame brightness interference provided by this application;

[0017] Figure 5 It is a schematic flowchart of a specific example of the method for determining video frame brightness interference provided by this application;

[0018] Figure 6 It is a schematic flowchart of a specific example of the method for determining video frame brightness interference provided by this application;

[0019] Figure 7 It is a schematic flowchart of a specific example of the device for determining video frame brightness interference provided by this application;

[0020] Figure 8 It is a schematic structural diagram of an embodiment of the electronic device of this application;

[0021] Figure 9 It is a schematic structural diagram of an embodiment of the computer-readable storage medium of this application. Specific Embodiments

[0022] The solutions of the embodiments of this application will be described in detail below with reference to the accompanying drawings of the specification.

[0023] In the following description, specific details such as specific system architectures, interfaces, and technologies are presented for the purpose of illustration rather than limitation, in order to thoroughly understand this application.

[0024] The term "and / or" in this document is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after. In addition, "multiple" in this document means two or more than two. In addition, the term "at least one" in this document represents any one of multiple or any combination of at least two of multiple. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set composed of A, B, and C.

[0025] The imaging device mentioned in this application can be an electronic device with video frame acquisition function such as an ordinary camera or an action camera.

[0026] The environment where the imaging device of this application is located can be outdoors (such as traffic roads), indoors, etc.

[0027] The light sources that may exist in the environment where the imaging device is located in this application can be divided into stationary light sources and moving light sources. A stationary light source refers to a light source with a relatively fixed position. A moving light source refers to a light source whose position changes over time. For example, the sun, moon, street lights, and indoor lighting in the outdoor environment are stationary light sources, while car lights, road surface reflections on rainy days, and road surface reflections on snowy days are moving light sources.

[0028] Through long-term research by the inventors of this application, it is found that the video frame brightness is relative to the stationary light source, and the moving light source in the environment where the imaging device is located will interfere with the brightness of the collected video frames, resulting in abnormal increase or decrease in the brightness of the video frames collected by the imaging device.

[0029] In order to determine the degree of interference of the moving light source on the brightness of the video frames collected by the imaging device, a method for determining video frame brightness interference is provided.

[0030] Figure 1 It is a schematic flowchart of an embodiment of the method for determining video frame brightness interference provided by this application. As Figure 1 shown, in this embodiment, the method for determining video frame brightness interference may include the following steps:

[0031] S11: Obtain the statistical brightness of each first sub-block in the current actual frame and the statistical brightness of each second sub-block in the previous reference frame.

[0032] The first sub-blocks and the second sub-blocks correspond one by one, and each first sub-block and each second sub-block are obtained by dividing the current actual frame and the previous reference frame respectively according to the same division method.

[0033] The execution subject of this embodiment can be any electronic device with the ability to determine video frame brightness interference, such as a computer, a mobile phone, or an imaging device.

[0034] The current actual frame is obtained by the imaging device. The current actual frame can be collected in real time or non-real time.

[0035] The previous reference frame can be a reference to an actual historical frame, or a result of motion light source filtering of the reference actual historical frame, or a fusion result of multiple reference actual historical frames. The reference actual historical frame can be the previous actual frame of the current actual frame in the video, or an actual historical frame before the previous actual frame. The video is obtained by a camera device collecting in its environment. The previous actual frame refers to an adjacent actual frame in the video whose acquisition time is earlier than that of the current actual frame. When the reference actual historical frame is the previous actual frame, the frame interval determined by the luminance interference is 0. When the reference actual historical frame is an actual historical frame before the previous actual frame, the frame interval determined by the luminance interference is greater than or equal to 1, which can improve the efficiency of luminance interference in the video. The frame interval can be set according to the actual requirements in the application scenario.

[0036] Each first block and each second block are obtained by dividing the current actual frame and the previous reference frame respectively according to the same division method, which means that the number of pixels, width, and height included in the first block and the corresponding second block are the same, and the area of the first block in the current actual frame is the same as the area of the corresponding second block in the previous reference frame.

[0037] Different first blocks are in different regions of the current actual frame, and the regions where adjacent first blocks are located may not overlap or may partially overlap. Similarly, different second blocks are in different regions of the current reference frame, and the regions where adjacent second blocks are located may not overlap or may partially overlap.

[0038] The statistical luminance of the first block / second block is the statistical result of the luminance of each pixel in the first block / second block, which can represent the luminance of the first block / second block. The luminance statistical result can be the minimum luminance value, the maximum luminance value, the third central tendency value, etc. The third central tendency value can be the mean, the mode, the median, etc. In some embodiments, the luminance statistical result can use the 3A statistical information in the ISP chip.

[0039] In some embodiments, the statistical luminance of the first block / second block is the third central tendency value of each pixel. S11 includes: for each first block / second block, based on the luminance of each pixel in the corresponding block, obtaining the third central tendency value of the pixel luminance of the block; using the third central tendency value corresponding to the block as the statistical luminance of the block.

[0040] S12: Respectively obtain the statistical luminance difference between the statistical luminance of each first block and the statistical luminance of the corresponding second block.

[0041] The statistical luminance difference between the first block and the corresponding second block can represent the luminance change of the first block relative to the second block.

[0042] For a first block and a corresponding second block, the statistical luminance of the first block can be subtracted from the statistical luminance of the second block to obtain the statistical luminance difference between the first block and the corresponding second block.

[0043] S13: Based on the statistical luminance differences corresponding to each first block, determine the degree of interference of each first block by the moving light source.

[0044] Among them, the degree of interference is positively correlated with the statistical luminance difference.

[0045] The larger the statistical luminance difference, the greater the luminance change of the first block relative to the second block, and the higher the degree of interference by the moving light source. On the contrary, the smaller the statistical luminance difference, the greater the luminance change of the first block relative to the second block, and the lower the degree of interference by the moving light source.

[0046] Through the implementation of this embodiment, the present application obtains the statistical luminance of each first block in the current actual frame and the statistical luminance of each second block in the previous reference frame, and the first block and the second block correspond one by one; respectively obtain the statistical luminance difference between the statistical luminance of each first block and the statistical luminance of the corresponding second block; based on the statistical luminance differences corresponding to each first block, determine the degree of interference of each first block by the moving light source, and the degree of interference is positively correlated with the statistical luminance difference. Thus, on the one hand, through this luminance interference determination method, the degree of interference of the current actual frame by the moving light source can be determined, and this luminance interference determination method is simple, easy to understand, and easy to implement. On the other hand, using the degree of interference of each first block by the moving light source to jointly represent the degree of interference of the current actual frame by the moving light source takes into account the differences in the degree of interference of different first blocks in the current actual frame by the moving light source, and expresses the degree of interference of the current actual frame by the moving light source more accurately. On the other hand, the unit of luminance interference determination is the first block, which can improve the efficiency of luminance interference determination compared with the method where the unit of luminance interference determination is a pixel.

[0047] In some embodiments, in S13, the mean value of the statistical luminance differences corresponding to each first block can be obtained, and based on the magnitude relationship between the statistical luminance difference corresponding to each first block and the mean value, determine the degree of interference of each block by the moving light source. For example, if the statistical luminance difference corresponding to the first block is greater than the mean value, it represents a higher degree of interference by the moving light source. On the contrary, it represents a lower degree of interference by the moving light source.

[0048] In some embodiments, in S13, a conversion formula between the statistical luminance difference and the interference degree can be preset. Through this conversion formula, the statistical luminance difference can be converted to obtain the interference degree, and the obtained interference degree belongs to a preset numerical range. A value within this numerical range is selected as the interference threshold. When the interference degree obtained through the conversion formula is greater than the interference threshold, it indicates that the first block is interfered by a moving light source. When the interference degree obtained through the conversion formula is not greater than the interference threshold, it indicates that the first block is not interfered by a moving light source. For example, the numerical range is [-1, 1], the interference threshold is 0, an interference degree greater than 0 indicates being interfered by a moving light source, and an interference degree less than or equal to 0 indicates not being interfered by a moving light source.

[0049] In some embodiments, in S13, the statistical luminance difference can be divided into several luminance difference ranges, and the interference degree can be divided into several degree levels. Different luminance difference ranges correspond to different degree levels. The number of luminance difference ranges and the number of interference degree levels can be 1, 2, 3, 4, etc., and can be flexibly set according to the actual requirements of the application scenario. The following takes the case where the number of luminance difference ranges and the number of interference degree levels is 3 as an example for illustration.

[0050] Figure 2 It is a schematic flowchart of an embodiment of the method for determining video frame luminance interference provided by the present application. In this embodiment, the number of luminance difference ranges and the number of interference degree levels is 3. As Figure 2 shown, in this embodiment, S13 may include the following steps:

[0051] S131: Obtain a first reference luminance difference and a second reference luminance difference.

[0052] The second reference luminance difference is greater than the first reference luminance difference.

[0053] The first reference luminance difference and the second reference luminance difference are the discrimination thresholds for 3 luminance difference ranges. That is, the statistical luminance difference is divided into 3 luminance difference ranges by the first reference luminance difference and the second reference luminance difference.

[0054] Both the first reference luminance difference and the second reference luminance difference are fixed absolute thresholds, or both are dynamic relative thresholds, or one is an absolute threshold and the other is a relative threshold. The absolute threshold is within the luminance range supported by the imaging device. The relative threshold is dynamically adjusted according to the first central tendency value of the statistical luminance difference corresponding to each first block. The larger the first central tendency value, the larger the relative threshold. The first central tendency value is similar to the aforementioned third central tendency value and will not be elaborated here.

[0055] It can be understood that when the first reference luminance difference is an absolute threshold and the second reference luminance difference is a relative threshold, the absolute threshold and the relative threshold can complement each other, improving the accuracy and adaptability of the luminance difference range division. Specifically, with the first reference luminance difference fixed, it can adapt to the situation where the statistical luminance differences of different first sub-blocks in the same actual frame vary greatly (for example, the difference between the maximum statistical luminance difference and the minimum statistical luminance difference is greater than the difference threshold), thus avoiding the problem of inaccurate luminance difference range division caused by an overly large first reference luminance difference, and further avoiding the problem of easy errors in the subsequent determination of the low and medium interference levels. With the second reference luminance difference fixed, it can adapt to the situation where the first central tendency value of the statistical luminance differences corresponding to the first sub-blocks varies greatly between different actual frames, avoiding the problem of inaccurate luminance difference range division caused by an overly large second reference luminance difference when the first central tendency value is large, and further avoiding the problem of easy errors in the subsequent determination of the medium and high interference levels.

[0056] In some embodiments, the first reference luminance difference is less than the first central tendency value, and the second reference luminance difference is greater than the first central tendency value.

[0057] In some embodiments, the first reference threshold is the product of the maximum luminance supported by the imaging device and a preset luminance ratio, and the second reference threshold is greater than the first central tendency value. In this case, S131 includes: multiplying the maximum luminance supported by the imaging device by the preset luminance ratio to obtain the first reference luminance difference; and obtaining the first central tendency value of the statistical luminance differences corresponding to each first sub-block, and determining the second reference luminance difference based on the first central tendency value. Among them, the second reference luminance difference is greater than the first central tendency value. The maximum luminance supported by the imaging device is 2 n , where n represents the number of luminance bits. For example, when the number of luminance bits is 10, the maximum luminance supported by the imaging device is 2 10 = 1024. The preset luminance ratio can be any ratio between 1% and 2%, or it can be 3% etc.

[0058] S132: Based on the first reference luminance difference and the second reference luminance difference, determine the first luminance difference range, the second luminance difference range, and the third luminance difference range.

[0059] Among them, the first luminance difference range is less than the first reference luminance difference, the second luminance difference range is greater than or equal to the first reference luminance difference and less than the second reference luminance difference, and the third luminance difference range is greater than or equal to the second reference luminance difference.

[0060] S133: For each first sub-block, determine the luminance difference range to which the statistical luminance difference corresponding to the first sub-block belongs.

[0061] In response to the statistical luminance difference corresponding to the first block belonging to the first luminance difference range, execute S134; in response to the statistical luminance difference corresponding to the first block belonging to the second luminance difference range, execute S135; in response to the statistical luminance difference corresponding to the first block belonging to the third luminance difference range, execute S136.

[0062] S134: Determine that the interference degree is low.

[0063] Low degree means no interference or almost no moving light source interference.

[0064] S135: Determine that the interference degree is medium.

[0065] S136: Determine that the interference degree is high.

[0066] It can be understood that when the statistical luminance difference belongs to the first luminance difference range, it can be regarded that the luminance change is mainly caused by a stationary light source, and the luminance change caused by the moving light source interference is small or even there is no moving light source interference. Therefore, the interference degree is determined to be low. When the statistical luminance difference belongs to the second luminance difference range, it can be regarded that the luminance change is caused by both a stationary light source and a moving light source. Therefore, the interference degree is determined to be medium. When the statistical luminance difference belongs to the third luminance difference range, it can be regarded that the luminance change is mainly caused by a moving light source. Therefore, the interference degree is determined to be high.

[0067] Through the implementation of this embodiment, the luminance difference is divided into 3 luminance difference ranges based on the first reference luminance difference and the second reference luminance difference. Different luminance difference ranges correspond to different interference degrees. Thus, based on the luminance difference range to which the statistical luminance difference corresponding to the first block belongs, the interference degree of the first block by the moving light source can be determined.

[0068] The video frame luminance interference determination method provided in any of the above embodiments can be but is not limited to being applied in scenarios such as Auto White Balance (AWB), Auto Exposure (AE), and ambient luminance estimation. The following takes the ambient luminance estimation application scenario as an example for detailed description.

[0069] In the ambient luminance estimation application scenario, the ambient luminance can be determined based on the luminance of the video frames captured by the imaging device. However, if there are moving light sources in the environment where the imaging device is located, under the interference of the moving light sources, the luminance of the video frames captured by the imaging device abnormally increases or decreases, and cannot accurately reflect the ambient luminance where the imaging device is located, resulting in inaccurate ambient luminance determined based on the luminance of the video frames captured by the imaging device.

[0070] Therefore, on the basis of determining the interference degree of the current actual frame by the moving light source through the above embodiment, the current actual frame can be further filtered for the moving light source based on the interference degree, and then the current ambient luminance can be determined.

[0071] Figure 3 This is a schematic flowchart of an embodiment of the method for determining video frame brightness interference provided by the present application. This embodiment is a further extension of the above embodiment to determine the current ambient brightness. As Figure 3 shown, in the application scenario of ambient brightness estimation, after S13, the following steps may be included:

[0072] S14: Based on the interference degree of each first block by the moving light source, filter the moving light source from the current actual frame to obtain the current reference frame.

[0073] The method of filtering the moving light source may be to apply the statistical brightness difference to the current actual frame, or to apply the statistical brightness difference to the previous reference frame.

[0074] In some embodiments, apply the statistical brightness difference to the previous reference frame to obtain the current reference frame. S14 includes S141 - S142 (not shown in the figure). S141: Obtain the fusion ratio of the statistical brightness difference corresponding to each first block. Among them, the fusion ratio is negatively correlated with the interference degree. S142: According to the corresponding fusion ratio, respectively fuse the statistical brightness difference corresponding to each first block to the previous reference frame to obtain the current reference frame.

[0075] It can be understood that in S141 - S142, the higher the interference degree of the first block by the moving light source, the higher the proportion of the change in the statistical brightness of the first block relative to the statistical brightness of the second block affected by the moving light source. The lower the fusion ratio is set, the higher the filtering intensity of the moving light source when fusing to the previous reference frame, so as to realize the filtering of the moving light source adapted to the interference degree of the first block.

[0076] In S141, the interference degree can be divided into multiple degree levels, and the number of degree levels can be 1, 2, 3, 4, etc. Different degree levels correspond to different fusion ratios. The higher the degree level, the higher the interference degree, and the lower the fusion ratio. In some embodiments, the interference degree is divided into 3 degree levels, namely low degree, medium degree, and high degree. The fusion ratios corresponding to low degree, medium degree, and high degree are the first ratio, the second ratio, and the third ratio respectively. Among them, the first ratio is higher than the second ratio, and the second ratio is higher than the third ratio.

[0077] In this case, S141 includes: for each first block, determine whether the interference degree of the first block is low degree, medium degree, or high degree; in response to the interference degree being low degree, determine the fusion ratio of the statistical brightness difference corresponding to the first block as the first ratio; in response to the interference degree being medium degree, determine the fusion ratio of the statistical brightness difference corresponding to the first block as the second ratio; in response to the interference degree being high degree, determine the fusion ratio of the statistical brightness difference corresponding to the first block as the third ratio.

[0078] In some embodiments, S142 includes: according to the corresponding fusion ratio, respectively fusing the statistical luminance difference corresponding to each first sub-block with the luminance of the corresponding pixel in the previous reference frame to obtain the current reference frame. In some embodiments, S142 includes: according to the corresponding fusion ratio, respectively fusing the statistical luminance of each first sub-block with the statistical luminance of the corresponding second sub-block to obtain a third sub-block, so as to form the current reference frame.

[0079] In some embodiments, the statistical luminance difference is applied to the current actual frame to obtain the current reference frame. S14 includes S143 - S144 (not shown in the figure). S143: Based on the degree of interference of each first sub-block by the moving light source, determine the reduction ratio of each first sub-block. Wherein, the degree of interference is positively correlated with the reduction ratio. S144: Reduce the luminance of the pixels in each first sub-block or the statistical luminance of each first sub-block according to the corresponding reduction ratio to obtain a fourth sub-block, so as to form the current reference frame.

[0080] It can be understood that in S143 - S144, the higher the degree of interference of the first sub-block by the moving light source, the higher the proportion of the change in the statistical luminance of the first sub-block relative to the statistical luminance of the second sub-block affected by the moving light source. The larger the reduction ratio is set, the higher the filtering intensity of the moving light source for the first sub-block, so as to realize the filtering of the moving light source adapted to the degree of interference of the first sub-block.

[0081] In addition, in this embodiment, the previous reference frame is obtained by processing based on the reference historical actual frame. The processing method of obtaining the previous reference frame based on the reference historical actual frame is the same as the processing method of obtaining the current reference frame based on the current actual frame. The reference historical actual frame is the previous actual frame of the current actual frame in the video or the historical actual frame before the previous actual frame. The smaller the frame interval between the reference historical actual frame and the previous reference frame, the faster the moving light source filtering speed.

[0082] S15: Determine the current ambient luminance based on the luminance of each pixel in the current reference frame.

[0083] Generally speaking, the current ambient luminance is the visible light luminance.

[0084] In some embodiments, the current actual frame is of the visible light type, and the obtained current reference frame is also of the visible light type. S15 includes: obtaining the second central tendency value of the pixel luminance of the current reference frame based on the luminance of each pixel in the current reference frame; determining the current ambient luminance based on the second central tendency value, the current exposure time, and the current gain. The second central tendency value is similar to the aforementioned third central tendency value and will not be elaborated here.

[0085] In some embodiments, the calculation formula for the current ambient luminance is:

[0086] currLum = visibleEv / (currGain * currShut);

[0087] Among them, currLum represents the current ambient brightness, visibleEv represents the second central tendency value, currShut represents the current exposure time, and currGain represents the current gain.

[0088] In some embodiments, the calculation formula for the current ambient brightness is:

[0089] currLum = f(visibleEv / (currGain * currShut));

[0090] Among them, f(.) represents a linear stretching transformation or a log transformation. Through the linear stretching transformation or the log transformation, the discrimination degree of currLum under different visibleEvs can be increased.

[0091] In some embodiments, the current actual frame is of the mixed light type of visible light - infrared light, and the obtained current reference frame is also of the mixed light type of visible light - infrared light. The obtaining method of the second central tendency value in S15 is different from the obtaining method of the second central tendency value under the visible light type. In some embodiments, the obtaining method of the second central tendency value under the mixed light type is as follows:

[0092] visibleEv = mean(X, Y) / xPos;

[0093] Among them, X represents the Euclidean distance or Manhattan distance between the current reference frame under mixed light illumination in the R / G and B / G color spaces and the current reference frame under only infrared light illumination.

[0094] Y represents the G value of the current reference frame in the RGB space under mixed light illumination; mean(X * Y) represents the average value of X * Y of each remaining sub - block after removing the sub - blocks with brightness values greater than the set threshold in the current reference frame under mixed light illumination.

[0095] xPos represents the maximum value of the Euclidean distance or Manhattan distance between the current reference frame under only visible light illumination in the R / G and B / G color spaces and the current reference frame under only infrared light illumination.

[0096] Through the implementation of this embodiment, based on the interference degree of each first block in the current actual frame by the moving light source, each first block is filtered for the moving light source to obtain the current reference frame; based on the brightness of each pixel in the current reference frame, the current ambient brightness is determined. Since the current reference frame is the result of filtering the moving light source of the current actual frame, compared with the current actual frame, the current ambient brightness determined based on the brightness of each pixel in the current reference frame is more accurate, thereby improving the accuracy and stability of the ambient brightness estimation. In addition, filtering each first block based on the interference degree of each first block in the current actual frame by the moving light source takes into account the differences in the interference degrees of each first block by the moving light source, making the filtering of each first block adapt to the interference intensity, and improving the effectiveness and adaptability of the moving light source filtering.

[0097] Further, the current ambient brightness obtained by the method for determining the video frame brightness interference provided by the above embodiment can be applied to scenarios such as the mode switching of a camera device and an intelligent lighting system. The following is a detailed description taking the application scenario of the mode switching of a camera device as an example.

[0098] In the application scenario of the mode switching of a camera device, the modes of the camera device at least include a daytime mode and a nighttime mode. Generally speaking, the daytime mode is a color mode, and the nighttime mode is a black-and-white mode. During a high ambient brightness period, it is expected that the camera device is in the daytime mode, and during a low ambient brightness period, it is expected that the camera device is in the nighttime mode to improve the quality of the video frames collected by the camera device. Therefore, it is necessary to adaptively switch the mode of the camera device according to the change of the ambient brightness where the camera device is located. When the ambient brightness changes from a high ambient brightness to a low ambient brightness, it is expected that the mode switching of the camera device is from the daytime mode to the nighttime mode. When the ambient brightness changes from a low ambient brightness to a high ambient brightness, it is expected that the mode switching of the camera device is from the nighttime mode to the daytime mode. The situation of high ambient brightness can be sufficient illumination, such as outdoors with sufficient natural light during the day or indoors with sufficient lighting. The situation of low ambient brightness can be scarce or even no illumination, such as at night. The high and low of the ambient brightness can be distinguished by an ambient brightness threshold.

[0099] The ambient brightness where the camera device is located is determined based on the brightness of the video frames collected by the camera device. Under normal circumstances, the light source in the environment where the camera device is located is a stationary light source. However, if there is a moving light source in the environment where the camera device is located, the moving light source will interfere with the brightness of the video frames collected by the camera device, resulting in the fact that the brightness of the video frames cannot accurately reflect the ambient brightness where the camera device is located. Therefore, the ambient brightness determined based on the brightness of the video frames collected by the camera device is not accurate enough, resulting in abnormal back-and-forth switching of the mode of the camera device, affecting the stability and accuracy of the mode switching of the camera device.

[0100] Therefore, after obtaining the current ambient brightness through the above embodiment, it can be determined whether it is necessary to switch the mode of the camera device based on the current ambient brightness.

[0101] Figure 4 This is a schematic flowchart of an embodiment of the method for determining video frame brightness interference provided by the present application. This embodiment is a further extension of the above embodiment to determine whether to switch the mode of the imaging device. As Figure 4 shown, in the application scenario of ambient brightness estimation, after S15, the following steps may be included:

[0102] S16: Based on the current ambient brightness, determine whether the current actual frame meets the switching condition.

[0103] The switching condition includes that the current mode of the imaging device is the night mode and the current ambient brightness is greater than the first ambient brightness threshold, or the current mode of the imaging device is the day mode and the current ambient brightness is less than the second ambient brightness threshold. The first ambient brightness threshold is the ambient brightness threshold for switching from the night mode to the day mode, and the second ambient brightness threshold is the ambient brightness threshold for switching from the day mode to the night mode.

[0104] Generally, the second ambient brightness threshold is less than the first ambient brightness threshold to avoid the imaging device mode from switching back and forth due to the ambient brightness threshold.

[0105] In response to the current actual frame meeting the switching condition, execute S17. In response to the current actual frame not meeting the switching condition, execute S18.

[0106] S17: Determine that it is necessary to switch the mode of the imaging device.

[0107] In some embodiments, directly determine that it is necessary to switch the mode of the imaging device.

[0108] In some embodiments, it may further be determined whether the number of consecutive video frames currently meeting the switching condition reaches the frame number threshold; in response to reaching the frame number threshold, determine that it is necessary to switch the mode of the imaging device; in response to not reaching the frame number threshold, continue to perform similar processing based on the next video frame newly acquired by the imaging device until the frame number threshold is obtained.

[0109] S18: Determine that it is not necessary to switch the mode of the imaging device.

[0110] In the related art, in order to avoid the abnormal back-and-forth switching of the imaging device mode, the solution is to increase the ambient brightness threshold or lengthen the judgment interval. However, these two methods sacrifice the accuracy of mode switching and can only be used as temporary avoidance measures, and do not fundamentally solve the problem of abnormal back-and-forth switching of the imaging device mode.

[0111] Through the implementation of this embodiment, on the basis of determining the current ambient brightness, it is further determined whether to switch the mode of the imaging device based on the current ambient brightness. Since the determined current ambient brightness is more accurate, the problem of abnormal back-and-forth switching of the imaging device mode caused by moving light sources is fundamentally avoided, and the accuracy, stability, and adaptability of the mode switching of the imaging device are improved. Moreover, this method does not depend on the type of acquisition sensor included in the imaging device and can be applied to imaging devices including any type of acquisition sensor.

[0112] To facilitate the understanding of this application, the method for determining video frame brightness interference provided by this application is described below through a specific example.

[0113] Figure 6 It is a schematic flowchart of a specific example of the method for determining video frame brightness interference provided by this application. As Figure 6 shown, the method for determining video frame brightness interference includes:

[0114] 1. Obtain the statistical brightness of each first block in the current actual frame H3AStataC and the statistical brightness of each second block in the previous reference frame H3AStataX.

[0115] 2. Obtain the statistical brightness difference currDiffX between the statistical brightness of each first block and the statistical brightness of the corresponding second block.

[0116] 3. Based on the statistical brightness difference currDiffX corresponding to each first block, determine the interference degree corresponding to each first block.

[0117] (1) Obtain the first reference brightness difference evThr1 = 16 based on the maximum brightness supported by the imaging device. Obtain evThr2 = ABS(avgDiffX)*1.2 based on the mean value of the statistical brightness differences of each second block. Where avgDiffX represents the mean value of the statistical brightness differences of each second block, and ABS(.) represents taking the absolute value.

[0118] (2) Determine the first brightness difference range (evThr1), the second brightness difference range (≥evThr1 and <evThr2), and the third brightness difference range (≥evThr2), and the corresponding interference degrees are low degree, medium degree, and high degree respectively.

[0119] 4. Based on the interference degree of each first block by the moving light source, filter the current actual frame H3AStataC to obtain the current reference frame H3AStataY.

[0120] (1) Obtain the fusion ratio of the statistical brightness difference currDiffX corresponding to each first block.

[0121] The interference levels are low, medium, and high, and the corresponding fusion ratios are the first fusion ratio evRatio1 = 1 / 2, the second fusion ratio evRatio2 = 1 / 64, and the third fusion ratio evRatio3 = 1 / 128, respectively.

[0122] (2)Fuse the statistical luminance difference currDiffX corresponding to each first block with the previous reference frame H3AStataY to obtain the current reference frame H3AStataY = H3AStataX + (currDiffX * evRatio).

[0123] 5. Determine the current ambient luminance currLum based on the luminance of each pixel in the current reference frame H3AStataY.

[0124] (1) Based on the luminance of each pixel in the current reference frame H3AStataY, determine the luminance mean visibleEv = average(H3AStataY) of each pixel in the current reference frame H3AStataY. Here, average(.) represents calculating the mean.

[0125] (2) Based on the luminance mean visibleEv of each pixel in the current reference frame H3AStataY, the current exposure time currShut, and the current gain currGain, determine the current ambient luminance currLum = visibleEv / (currGain * currShut).

[0126] 6. Determine whether to switch the mode of the imaging device based on the current ambient luminance visibleEv.

[0127] (1) When the current mode currDNmode of the imaging device is the night mode, determine whether currLum > N2DLumThr is satisfied, where N2DLumThr is the first ambient luminance threshold. In response to satisfying currLum > N2DLumThr and the cumulative number of consecutive frames satisfying currLum > N2DLumThr being greater than the frame threshold, determine that a switch is required; otherwise, determine that a switch is not required.

[0128] (2) When the current mode currDNmode of the imaging device is the day mode, determine whether currLum < D2NlumThr is satisfied, where D2NlumThr is the second ambient luminance threshold. In response to satisfying currLum < D2NlumThr and the cumulative number of consecutive frames satisfying currLum < D2NlumThr being greater than the frame threshold, determine that a switch is required; otherwise, determine that a switch is not required.

[0129] In case of a need for switching, switch the current mode of the imaging device, clear the accumulated frame numbers, update the current actual frame with the next actual frame, and update the previous reference frame with the current reference frame, and repeat the above steps. In case of no need for switching, update the current actual frame with the next actual frame, and update the previous reference frame with the current reference frame, and repeat the above steps.

[0130] The method for determining the video frame brightness interference provided by the present application was verified based on the video frames collected by the camera arranged along the traffic road for 5 consecutive minutes. The verification results are shown in Figure 5 . Figure 5 The horizontal axis represents the video frames collected by the imaging device, and the vertical axis represents visibleEv. The blue line represents visibleEv before filtering the moving light source, and the orange line represents visibleEv after filtering the moving light source. With continuous vehicle passing for 5 minutes, the average value of visibleEv after filtering the moving light source decreases by 30%, the standard deviation decreases by 40%, and the peak value decreases by 50%. Thereby, the accuracy and stability of the environmental brightness estimation can be improved.

[0131] Figure 7 is a schematic flowchart of a specific example of the video frame brightness interference determination device provided by the present application. As Figure 7 shown, the video frame brightness interference determination device 20 includes that the video frame brightness interference determination method includes: a brightness acquisition module 21, a brightness difference acquisition module 22, and a determination module 23.

[0132] Among them, the brightness acquisition module 21 is configured to acquire the statistical brightness of each first sub-block in the current actual frame and the statistical brightness of each second sub-block in the previous reference frame, where the first sub-blocks and the second sub-blocks are in one-to-one correspondence, and each first sub-block and each second sub-block are obtained by dividing the current actual frame and the previous reference frame respectively according to the same division method.

[0133] The brightness difference acquisition module 22 is configured to respectively acquire the statistical brightness differences between the statistical brightness of each first sub-block and the statistical brightness of the corresponding second sub-block.

[0134] The determination module 23 is configured to determine the interference degree of each first sub-block by the moving light source based on the statistical brightness differences corresponding to each first sub-block, where the interference degree is positively correlated with the statistical brightness difference.

[0135] For other detailed descriptions of the video frame brightness interference determination device, please refer to the previous embodiments and will not be elaborated here.

[0136] Figure 8 is a schematic structural diagram of an embodiment of the electronic device of the present application. As Figure 8As shown, the electronic device 30 includes a memory 31 and a processor 32. The processor 32 is configured to execute program instructions stored in the memory 31 to implement the steps in any of the above method embodiments. In a specific implementation scenario, the electronic device 30 may include, but is not limited to, a microcomputer, a server. In addition, the electronic device 30 may also include carrier devices such as a laptop computer, a tablet computer, etc., which are not limited herein.

[0137] Specifically, the processor 32 is configured to control itself and the memory 31 to implement the steps in any of the above method embodiments. The processor 32 may also be referred to as a CPU (Central Processing Unit). The processor 32 may be an integrated circuit chip with signal processing capabilities. The processor 32 may also be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Additionally, the processor 32 may be implemented jointly by integrated circuit chips.

[0138] Please refer to Figure 9 , Figure 9 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 40 stores program instructions 41 thereon. When the program instructions 41 are executed by a processor, the steps in any of the above method embodiments are implemented.

[0139] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure may be used to execute the methods described in the above method embodiments. The specific implementation may refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0140] The descriptions of the above embodiments tend to emphasize the differences between the embodiments. The same or similar parts can be referred to each other. For the sake of brevity, they will not be repeated herein.

[0141] In several embodiments provided in the present application, it should be understood that the disclosed methods and apparatuses can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. In another image position, the couplings or direct couplings or communication connections shown or discussed among each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0142] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

Claims

1. A method for determining brightness interference of a video frame, characterized in that: include: Obtaining statistical brightness of each first block in the current actual frame and statistical brightness of each second block in the previous reference frame, wherein the first blocks correspond to the second blocks one by one, and the first blocks and the second blocks are obtained by dividing the current actual frame and the previous reference frame respectively in the same division manner; Respectively obtaining statistical brightness differences between the statistical brightness of each first sub-block and the statistical brightness of the corresponding second sub-block; Based on the statistical brightness difference corresponding to each of the first blocks, the interference degree of each of the first blocks caused by the moving light source is determined, wherein the interference degree is positively correlated with the statistical brightness difference.

2. The method according to claim 1, characterized in that The determining, based on the statistical brightness difference corresponding to each of the first blocks, the degree to which each of the first blocks is disturbed by the moving light source includes: Acquire a first reference brightness difference and a second reference brightness difference, wherein the second reference brightness difference is greater than the first reference brightness difference; Based on the first reference brightness difference and the second reference brightness difference, determining a first brightness difference range, a second brightness difference range, and a third brightness difference range, wherein the first brightness difference range is smaller than the first reference brightness difference, the second brightness difference range is greater than or equal to the first reference brightness difference and smaller than the second reference brightness difference, and the third brightness difference range is greater than or equal to the second reference brightness difference; For each of the first blocks, in response to the statistical brightness difference corresponding to the first block belonging to the first brightness difference range, determining that the interference level is low; In response to the statistical brightness difference corresponding to the first block belonging to the second brightness difference range, determining that the interference level is medium; In response to the statistical brightness difference corresponding to the first block belonging to the third brightness difference range, it is determined that the interference level is high.

3. The method according to claim 2, characterized in that The current actual frame is acquired by a camera device, and the acquiring of the first reference brightness difference and the second reference brightness difference includes: Multiplying the maximum brightness supported by the camera device by a preset brightness ratio to obtain the first reference brightness difference; and A first central tendency value of the statistical brightness difference corresponding to each of the first blocks is obtained, and the second reference brightness difference is determined based on the first central tendency value, wherein the second reference brightness difference is greater than the first central tendency value.

4. The method according to claim 1, characterized in that: After determining the interference degree of each of the first blocks by the moving light source based on the statistical brightness difference corresponding to each of the first blocks, the method further includes: Based on the interference degree of each of the first blocks by the moving light source, filtering the current actual frame with the moving light source to obtain a current reference frame; The current environment brightness is determined based on the brightness of each pixel in the current reference frame.

5. The method according to claim 4, characterized in that The step of filtering the current actual frame with moving light sources based on the interference degree of each of the first blocks with moving light sources to obtain a current reference frame includes: Obtaining a fusion ratio of statistical brightness differences corresponding to each of the first blocks, wherein the fusion ratio is negatively correlated with the interference degree; According to the corresponding fusion ratio, the statistical brightness differences corresponding to the first blocks are respectively fused into the previous reference frame to obtain a current reference frame.

6. The method according to claim 5, characterized in that Obtaining the fusion ratio of the statistical brightness differences corresponding to each of the first blocks includes: For each of the first blocks, in response to the interference level being low, determining a fusion ratio of statistical brightness differences corresponding to the first blocks as a first ratio; In response to the interference degree being a medium degree, determining a fusion ratio of the statistical brightness difference corresponding to the first block to be a second ratio; In response to the interference degree being high, determining that the fusion ratio of the statistical brightness difference corresponding to the first block is a third ratio, wherein the first ratio is higher than the second ratio, and the second ratio is higher than the third ratio; And / or, according to the corresponding fusion ratio, respectively fusing the statistical brightness differences corresponding to each of the first blocks into the previous reference frame to obtain the current reference frame, including: According to the corresponding fusion ratio, the statistical brightness of each of the first blocks is fused with the statistical brightness of the corresponding second block to obtain a third block to form the current reference frame.

7. The method according to claim 4, characterized in that The determining the current environment brightness based on the brightness of each pixel in the current reference frame includes: Based on the brightness of each pixel in the current reference frame, obtaining a second central tendency value of the pixel brightness of the current reference frame; The current environment brightness is determined based on the second central tendency value, the current exposure time and the current gain.

8. The method according to claim 4, characterized in that The previous reference frame is obtained based on a reference historical actual frame. The processing method for obtaining the previous reference frame based on the reference historical actual frame is the same as the processing method for obtaining the current reference frame based on the current actual frame. The reference historical actual frame is the previous actual frame of the current actual frame in the video, or the historical actual frame before the previous actual frame.

9. The method according to claim 4, characterized in that The current actual frame is acquired by a camera device. After determining the current environment brightness based on the brightness of each pixel in the current reference frame, the method further includes: Based on the current ambient brightness, determining whether the current actual frame satisfies a switching condition, the switching condition comprising that the current mode of the camera device is the night mode and the current ambient brightness is greater than a first ambient brightness threshold, or the current mode of the camera device is the day mode and the current ambient brightness is less than a second ambient brightness threshold, the first ambient brightness threshold is the ambient brightness threshold for switching from the night mode to the day mode, and the second ambient brightness threshold is the ambient brightness threshold for switching from the day mode to the night mode; In response to the current actual frame satisfying the switching condition, it is determined that the mode of the camera device needs to be switched.

10. The method according to claim 9, characterized in that In response to the current actual frame satisfying the switching condition, determining that the mode of the camera device needs to be switched includes: In response to the current actual frame satisfying the switching condition, determining whether the number of continuous video frames that currently satisfy the switching condition reaches a frame number threshold; In response to reaching the frame number threshold, it is determined that the mode of the camera device needs to be switched.

11. The method according to claim 1, characterized in that The current actual frame is of visible light type; and / or The obtaining of the statistical brightness of each first block in the current actual frame and the statistical brightness of each second block in the previous reference frame includes: For each of the first blocks / the second blocks, based on the brightness of each pixel in the corresponding block, obtaining a third central tendency value of the pixel brightness of the block; The third central tendency value corresponding to the block is used as the statistical brightness of the block.

12. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 11.

13. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images