Picture flicker frequency detection method and apparatus, picture flicker elimination method and apparatus, and electronic device

Abstract

The present application provides a picture flicker frequency detection method and apparatus, a picture flicker elimination method and apparatus, and an electronic device. The picture flicker frequency detection method comprises: determining coordinate information of a sub-region, wherein the sub-region is a region within an acquired image containing an electronic screen; determining a brightness information difference of the sub-region between a current frame image and a previous frame image on the basis of the coordinate information, wherein the brightness information difference is a difference between target brightness information corresponding to the current frame image and target brightness information corresponding to the previous frame image, and the target brightness information is brightness information caused only by the electronic screen; and determining the pulse width modulation frequency of the electronic screen on the basis of the brightness information difference. Thus, when an image contains an electronic screen, the pulse width modulation frequency of the electronic screen can be determined.

Description

Methods for detecting screen flicker frequency, methods for eliminating screen flicker, devices and electronic equipment

[0001] This application claims priority to Chinese Patent Application No. 202411815210.6, filed on December 10, 2024, entitled “Method for detecting screen flicker frequency, method for eliminating screen flicker, apparatus and electronic device”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of signal processing technology, and in particular to a method for detecting screen flicker frequency, a method for eliminating screen flicker, an apparatus, and an electronic device. Background Technology

[0003] Flicker, also known as banding or flicker, refers to the phenomenon in video recording where the exposure time of the line exposure sensor in the camera equipment is not an integer multiple of the energy cycle of the light source, resulting in alternating bright and dark stripes in the captured image, or screen flickering.

[0004] The brightness of AC-powered lighting equipment changes with the alternating current. While the human eye may not perceive this flicker, a camera can capture it under certain conditions, resulting in noticeable banding in the captured image. Similarly, when using a line exposure sensor to photograph an area containing the screen of an electronic device such as a mobile phone or television, the screen also uses PWM (Pulse Width Modulation) dimming. If the exposure time of the line exposure sensor is not an integer multiple of the screen's PWM cycle, banding will also appear in the photographed screen area.

[0005] In the field of XR (Extended Reality), when a user wears an XR device, if the user's gaze is fixed on the electronic screen and there are obvious stripes in the area where the screen is located, the user experience will be poor. Therefore, it is necessary to eliminate the stripes in the area where the electronic screen is located. Thus, how to detect the screen flicker frequency in a sub-region is a technical problem that needs to be solved. Summary of the Invention

[0006] This application provides a method for detecting screen flicker frequency, a method for eliminating screen flicker, an apparatus, and an electronic device, used to accurately determine the screen flicker frequency of a sub-region corresponding to an electronic screen when the captured image contains an electronic screen.

[0007] Firstly, this application provides a method for detecting screen flicker frequency, including:

[0008] Determine the coordinate information of the sub-region; the sub-region is the area where the electronic screen is located in the acquired image.

[0009] Based on the coordinate information, determine the difference in brightness information of a sub-region between the current frame image and the previous frame image; the brightness information difference is the difference between the target brightness information corresponding to the current frame image and the target brightness information corresponding to the previous frame image; the target brightness information is the brightness information caused only by the electronic screen;

[0010] The pulse width modulation frequency of the electronic screen is determined based on the difference in brightness information.

[0011] Optionally, the brightness information difference includes row-wise brightness information difference and column-wise brightness information difference; determining the pulse width modulation frequency of the electronic screen based on the brightness information difference includes:

[0012] The row pulse width modulation frequency is determined based on the row brightness information difference;

[0013] The column pulse width modulation frequency is determined based on the column brightness information difference;

[0014] The row-wise pulse width modulation frequency and the column-wise pulse width modulation frequency are combined to determine the pulse width modulation frequency of the electronic screen.

[0015] Optionally, the non-sub-regions in the row containing the sub-region in the image are defined as the first region; determining the difference in brightness information between the sub-regions in the current frame image and the previous frame image includes:

[0016] For the current frame image, determine the row-direction brightness information of the sub-region, and determine the row-direction brightness information of the first region; the row-direction brightness information is related to the brightness value of each pixel in a row;

[0017] For any row containing a sub-region, calculate the difference between the row-direction brightness information of that row in the sub-region and the row-direction brightness information of that row in the first region, in order to determine the first row-direction brightness information of the sub-region in the current frame image;

[0018] The difference between the first row brightness information of the sub-region in the current frame image and the first row brightness information of the sub-region in the previous frame image is determined as the row brightness information difference.

[0019] Optionally, determine the difference in brightness information of a sub-region between the current frame image and the previous frame image, including:

[0020] For the current frame image, determine the column-oriented brightness information of the sub-region; the column-oriented brightness information is related to the brightness value of each pixel in a column;

[0021] The difference between the column-directed brightness information of a sub-region in the current frame image and the column-directed brightness information of a sub-region in the previous frame image is determined as the column-directed brightness information difference.

[0022] Optionally, regions in the image other than the sub-region are considered non-sub-regions; the method also includes:

[0023] For the current frame image, determine the row-wise brightness information of non-sub-regions;

[0024] The global frequency is determined based on the difference between the row brightness information of non-sub-regions in the current frame image and the row brightness information of non-sub-regions in the previous frame image; the global frequency is the frequency of the light sources present in the image.

[0025] Optionally, determine the coordinate information of the sub-region, including:

[0026] Semantic segmentation is used to determine the coordinate information of the area where the electronic screen is located, in order to determine the coordinate information of the sub-region.

[0027] Secondly, this application provides a method for eliminating screen flickering, including:

[0028] Determine the limiting frequency of the camera equipment; the limiting frequency is the minimum frequency value that the camera equipment supports for flicker removal; the camera equipment is used to capture images, including those from electronic screens;

[0029] When both the pulse width modulation frequency and the global frequency of the electronic screen exist, the target exposure time is determined based on the pulse width modulation frequency, the global frequency, and the limiting frequency of the electronic screen.

[0030] Adjust the actual exposure time of the camera equipment to the target exposure time.

[0031] Optionally, the target exposure time is determined based on the pulse width modulation frequency, global frequency, and limiting frequency of the electronic screen, including:

[0032] Determine the greatest common factor of the pulse width modulation frequency and the global frequency of the electronic screen;

[0033] When the greatest common factor is greater than or equal to the limiting frequency, the target exposure time is determined based on the greatest common factor and the limiting frequency.

[0034] Optionally, the target exposure time can be determined based on the greatest common factor and the limiting frequency, including:

[0035] Determine the minimum exposure time based on the greatest common factor;

[0036] The maximum exposure time supported by the camera equipment is determined based on the limiting frequency.

[0037] The target exposure time is determined based on the minimum and maximum exposure times; the target exposure time is an integer multiple of the minimum exposure time and is less than or equal to the maximum exposure time.

[0038] Optionally, the method also includes:

[0039] When there is no greatest common factor, or when the greatest common factor is less than the limiting frequency, the target exposure time is determined based on the pulse width modulation frequency, global frequency, first frequency type and limiting frequency of the electronic screen; the first frequency type is the frequency type that is preferentially eliminated; the second frequency type is the frequency type other than the first frequency type among the pulse width modulation frequency type and global frequency type of the electronic screen.

[0040] The target exposure time is used to eliminate flicker corresponding to the first frequency type and suppress flicker corresponding to the second frequency type.

[0041] Optionally, the target exposure time is determined based on the pulse width modulation frequency, global frequency, first frequency type, and limiting frequency of the electronic screen, including:

[0042] When the frequency corresponding to the first frequency type is greater than or equal to the limiting frequency, multiple first exposure times are determined according to the frequency of the first frequency type; the first exposure time is less than the maximum exposure time supported by the camera device determined by the limiting frequency.

[0043] A second exposure time is determined. For any first exposure time, a third exposure time is determined based on the second exposure time, and the difference between the second exposure time and the first exposure time is less than a preset value. The second exposure time is the minimum exposure time that can eliminate the flicker corresponding to the second frequency type. The third exposure time is an integer multiple of the second exposure time.

[0044] The target exposure time is determined from multiple first exposure times; the target exposure time is the first exposure time with the smallest difference from the corresponding third exposure time.

[0045] Optionally, the method also includes:

[0046] When only the pulse width modulation frequency of the electronic screen exists, and when the pulse width modulation frequency of the electronic screen is greater than or equal to the threshold frequency, the target exposure time is determined based on the pulse width modulation frequency of the electronic screen; and / or,

[0047] When only the global frequency exists, and the global frequency is greater than or equal to the limiting frequency, the target exposure time is determined based on the global frequency.

[0048] Optionally, there are multiple target exposure times. The actual exposure time of the camera device is adjusted to the target exposure time, including:

[0049] Determine the maximum allowed exposure time based on the current frame rate;

[0050] If there exists a target exposure time that is less than or equal to the current maximum allowed exposure time, then the actual exposure time is adjusted to the target exposure time; and / or,

[0051] If all target exposure times are greater than the current maximum allowed exposure time, adjust the frame rate and adjust the actual exposure time to any of the target exposure times.

[0052] Thirdly, this application provides a screen flicker frequency detection device, the device comprising:

[0053] The first determining module is used to determine the coordinate information of a sub-region; the sub-region is the area where the electronic screen is located in the acquired image.

[0054] The processing module is used to determine the difference in brightness information of a sub-region between the current frame image and the previous frame image based on the coordinate information; the brightness information difference is the difference between the target brightness information corresponding to the current frame image and the target brightness information corresponding to the previous frame image; the target brightness information is the brightness information caused only by the electronic screen;

[0055] The second determining module is used to determine the pulse width modulation frequency of the electronic screen based on the brightness information difference.

[0056] Fourthly, this application provides a screen flicker elimination device, comprising:

[0057] The third determining module is used to determine the limiting frequency of the camera device; the limiting frequency is the minimum frequency value that the camera device supports for flicker removal; the camera device is used to acquire images including those from electronic screens;

[0058] The fourth determining module is used to determine the target exposure time based on the pulse width modulation frequency, global frequency, and limit frequency of the electronic screen when both the pulse width modulation frequency and the global frequency of the electronic screen exist.

[0059] The adjustment module is used to adjust the actual exposure time of the camera device to the target exposure time.

[0060] Fifthly, this application provides an electronic device, comprising: at least one processor and a memory;

[0061] The memory stores the instructions that the computer executes;

[0062] At least one processor executes computer execution instructions stored in memory, causing at least one processor to perform the method as described in either the first or second aspect.

[0063] Sixthly, this application provides a wearable device, including a camera device and a processing unit;

[0064] Camera equipment is used to capture images;

[0065] The processing unit is configured to perform the method as described in any of the first aspects; or, the processing unit is configured to perform the method as described in any of the second aspects; or, the processing unit is configured to perform the method as described in either the first or the second aspects.

[0066] In a seventh aspect, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method of either the first or second aspect.

[0067] Eighthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method of either the first or second aspect.

[0068] This application provides a method, apparatus, and electronic device for detecting screen flicker frequency, eliminating screen flicker, and a device for doing so. The method involves determining the coordinate information of a sub-region, where the sub-region is the area containing the electronic screen in the acquired image. Based on the coordinate information, the method determines the difference in brightness information between the current frame image and the previous frame image. This brightness difference is the difference between the target brightness information corresponding to the current frame image and the target brightness information corresponding to the previous frame image. The target brightness information is the brightness information caused solely by the electronic screen. The method determines the pulse width modulation frequency of the electronic screen based on the brightness difference, thus determining the pulse width modulation frequency of the electronic screen when it exists in the image. Attached Figure Description

[0069] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0070] Figure 1 is a schematic diagram of flicker caused by an electronic screen according to an embodiment of this application;

[0071] Figure 2 is a schematic diagram illustrating one of the causes of banding according to an embodiment of this application;

[0072] Figure 3 is a schematic diagram of one cause of banding provided in an embodiment of this application;

[0073] Figure 4 is a flowchart illustrating a screen flicker frequency detection method provided in an embodiment of this application;

[0074] Figure 5 is a schematic diagram of a sub-region and a first region provided in an embodiment of this application;

[0075] Figure 6 is a schematic diagram of an image to be detected according to an embodiment of this application;

[0076] Figure 7 is a schematic diagram of a determined global frequency in Figure 6 provided by an embodiment of this application;

[0077] Figure 8 is a schematic diagram of a determined pulse width modulation frequency of the electronic screen in Figure 6 provided in an embodiment of this application;

[0078] Figure 9 is a schematic diagram of an image to be detected according to an embodiment of this application;

[0079] Figure 10 is a schematic diagram of a determined row pulse width modulation frequency of the electronic screen in Figure 9 provided by an embodiment of this application;

[0080] Figure 11 is a schematic diagram of a determined column pulse width modulation frequency of the electronic screen in Figure 9 provided by an embodiment of this application;

[0081] Figure 12 is a flowchart illustrating a screen flicker elimination method provided in an embodiment of this application;

[0082] Figure 13 is a schematic diagram of a screen flicker elimination process provided in an embodiment of this application;

[0083] Figure 14 is a schematic diagram of a screen flicker frequency detection device provided in an embodiment of this application;

[0084] Figure 15 is a schematic diagram of a screen flicker elimination device provided in an embodiment of this application;

[0085] Figure 16 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0086] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0087] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0088] When using a line exposure sensor to photograph an OLED (Organic Light-Emitting Diode) screen (electronic screens such as mobile phones and televisions), banding (flickering) will appear in the photographed screen area. Figure 1 is a schematic diagram of flickering caused by an electronic screen according to an embodiment of this application. As shown in Figure 1, when the line exposure time is not an integer multiple of the electronic screen's PWM cycle, the line integral brightness of the line exposure sensor in the screen area will be inconsistent, thus exhibiting a banding phenomenon. The cause of banding is briefly explained below.

[0089] Figure 2 is a schematic diagram illustrating one cause of banding according to an embodiment of this application. As shown in Figure 2, the PWM period of the electronic screen is 10ms. When the exposure time of the row exposure sensor is 10ms, if the Mth row and the Nth row start exposure at times tm and tn respectively, the area of ​​the shaded region in the figure represents the brightness of that row. The integral area of ​​the Mth row is the same as that of the Nth row because the integration time is exactly an integer multiple of the PWM period of the electronic screen. Therefore, the brightness sensed by different rows is the same, and no banding phenomenon occurs.

[0090] Figure 3 is a schematic diagram illustrating the cause of banding according to an embodiment of this application. As shown in Figure 3, the PWM period of the electronic screen is 10ms. When the exposure time of the row exposure sensor is 8ms, the shaded areas in the figure represent the brightness of the Mth row and the Nth row, respectively. The integral area of ​​the Mth row is greater than that of the Nth row because the integral area of ​​the Mth row crosses a peak, while the integral area of ​​the Nth row crosses a trough. Therefore, there will be a difference in brightness between the Mth and Nth rows, resulting in a ripple effect in the current frame image, as shown in Figure 1(a).

[0091] Similarly, the brightness of AC-powered lighting equipment will also change with the alternation of current. When the line exposure sensor takes a picture containing the above-mentioned lighting equipment or takes a picture of a scene exposed to the above-mentioned lighting equipment, obvious stripes will appear in the picture globally. This banding phenomenon is caused by the global lamp frequency (also known as the global frequency).

[0092] As shown in Figure 1(a), the line sampling direction of the line exposure sensor is from top to bottom. Depending on the placement of the phone, the refresh PWM direction of the phone screen is also from top to bottom, which can produce horizontal stripes.

[0093] The banding phenomenon caused by the electronic screen is based on the same principle as that generated by the global LED frequency. However, because the refresh PWM of the electronic screen is directional, when the horizontal sampling direction of the horizontal exposure sensor is inconsistent with the pulse width modulation direction (PWM direction) of the electronic screen, the banding in the captured screen area will appear as diagonal stripes, as shown in Figure 1(b). Specifically, as shown in Figure 1(b), the horizontal sampling direction of the horizontal exposure sensor is from top to bottom, while the refresh PWM direction of the phone screen is from left to right depending on the phone's placement. Therefore, the superposition of the two can produce diagonal stripes.

[0094] In addition, the phone can also be placed at an angle, in which case the refresh PWM direction of the phone screen will be the angled direction, and the superposition of the two can also produce diagonal stripes.

[0095] Head-mounted display devices, such as MR (Mixed Reality) devices, feature a video perspective mode. Video perspective refers to capturing a real-time view of the surrounding environment through cameras on the device and displaying it on a screen, giving the impression that the human eye can directly see the real world through the head-mounted display. In the XR field, a possible scenario is that when a user wears an XR device to view an electronic screen, if the screen has obvious stripes, i.e., image flicker, the user cannot accurately view the content displayed on the screen. Therefore, it is necessary to eliminate the stripes in the area where the electronic screen is located. Eliminating the stripes in the area where the electronic screen is located, i.e., eliminating image flicker, requires first detecting the pulse width modulation frequency of the area where the electronic screen is located.

[0096] The screen flicker frequency detection method proposed in this application can divide the image into sub-regions and non-sub-regions. The sub-region is the area where the electronic screen is located. For the sub-region, the difference in brightness information caused only by the electronic screen between the current frame image and the previous frame image can be calculated. Then, the calculated difference is processed by DFT (Discrete Fourier Transform) or FFT (Fast Fourier Transform) to obtain the screen flicker frequency caused by the electronic screen, that is, the pulse width modulation frequency, so as to detect the pulse width modulation frequency of the electronic screen.

[0097] Figure 4 is a flowchart illustrating a screen flicker frequency detection method provided in an embodiment of this application. The method includes steps S401 to S403:

[0098] Step S401: Determine the coordinate information of the sub-region; the sub-region is the area where the electronic screen is located in the acquired image.

[0099] After acquiring an image using a line exposure sensor, the coordinate information of a sub-region within the image can be determined first. Optionally, the sub-region is the area where the electronic screen in the image is located. As shown in Figure 1, the area where the mobile phone is located in the image is the sub-region. Furthermore, the electronic screen is not limited to the screen of a mobile phone; it can also be the screen of other electronic devices, such as televisions, tablets, and educational devices.

[0100] When determining the coordinates of a sub-region, it can be based on user input, such as the user clicking on the area where an electronic screen is located in the image, thus setting that area as the sub-region. Alternatively, the image can be input into a neural network model, which has the ability to identify electronic screens in the image, thereby setting the area where the electronic screen is located as the sub-region.

[0101] Once a sub-region is determined, its coordinates in the image can be obtained, such as the sub-region being located in the area consisting of rows 5 to 15 and columns 10 to 20.

[0102] Step S402: Based on the coordinate information, determine the difference in brightness information of the sub-regions in the current frame image and the previous frame image; the difference in brightness information is the difference between the target brightness information corresponding to the current frame image and the target brightness information corresponding to the previous frame image; the target brightness information is the brightness information caused only by the electronic screen.

[0103] After determining the coordinates of the sub-region, the brightness information difference of the sub-region can be calculated. The brightness information difference of the sub-region refers to the difference between the target brightness information of the sub-region of the current frame image and the target brightness information of the sub-region of the previous frame image.

[0104] Target brightness information refers to the brightness information caused solely by the electronic screen. Since the scene in which the image is captured may include lighting equipment, the operation of the lighting equipment may also affect the brightness information of the sub-region. Here, the target brightness information is the value after eliminating the influence of the lighting equipment on the brightness information of the sub-region, so that the pulse width modulation frequency of the electronic screen can be determined based on the difference in brightness information of the sub-region.

[0105] Optionally, the brightness information can be determined by statistically analyzing the G channel values ​​or Y values ​​of the RAW image. Optionally, for any row of the current frame image, the values ​​of each G channel in that row are added together to obtain the sum, and the sum is used as the brightness information for that row; or, the sum can be averaged to obtain the brightness information for that row.

[0106] Optionally, the preceding frame image here can refer to the frame image before the current frame image, or it can refer to multiple frames preceding the current frame image. When the preceding frame image is multiple frames preceding the current frame image, the corresponding target brightness information is the average target brightness information of the multiple frames. When the preceding frame image is the frame image before the current frame image, the corresponding target brightness information is the target brightness information of the previous frame image.

[0107] By calculating the brightness information difference, the background signal of the current frame image can be eliminated, thereby obtaining the pulse width modulation frequency of the electronic screen. For example, if the preceding frame image is the frame image before the current frame image, then the image information of the nth frame image can be considered as Pn = On + Fn. If the preceding frame image is the mth frame image, then the image information of the mth frame image is Pm = Om + Fm, where O represents the background signal and F represents the brightness information caused only by the electronic screen. Since the time between frames is short, the background signals of the two frames are considered to be completely identical, so Pn - Pm = Fn - Fm. That is, the brightness information difference between the two frames is the light intensity difference caused by the light energy received from the electronic screen. By performing frequency domain analysis on the brightness information difference, the pulse width modulation frequency of the electronic screen can be obtained.

[0108] Step S403: Determine the pulse width modulation frequency of the electronic screen based on the brightness information difference.

[0109] After determining the brightness information difference, DFT or FFT processing is performed on the brightness information difference to obtain the pulse width modulation frequency of the electronic screen.

[0110] This application provides a method for detecting screen flicker frequency. The method involves determining the coordinate information of a sub-region, where the sub-region is the area containing the electronic screen in the acquired image. Based on the coordinate information, the method determines the difference in brightness information between the sub-region in the current frame image and the previous frame image. The brightness difference is the difference between the target brightness information corresponding to the current frame image and the target brightness information corresponding to the previous frame image. The target brightness information is the brightness information caused solely by the electronic screen. Based on the brightness difference, the method determines the pulse width modulation frequency of the electronic screen, thus determining the pulse width modulation frequency of the electronic screen when it is present in the image.

[0111] Optionally, the brightness information difference includes row-wise brightness information difference and column-wise brightness information difference; determining the pulse width modulation frequency of the electronic screen based on the brightness information difference includes:

[0112] The row pulse width modulation frequency is determined based on the row brightness information difference;

[0113] The column pulse width modulation frequency is determined based on the column brightness information difference;

[0114] The row-wise pulse width modulation frequency and the column-wise pulse width modulation frequency are combined to determine the pulse width modulation frequency of the electronic screen.

[0115] Depending on how the electronic device is positioned relative to the image, the pulse width modulation frequency may only have a row-direction pulse width modulation frequency, or it may only have a column-direction pulse width modulation frequency, or it may have both row-direction and column-direction pulse width modulation frequencies simultaneously. Thus, the row-direction and column-direction pulse width modulation frequencies are combined to obtain the pulse width modulation frequency of the electronic screen.

[0116] Optionally, when the electronic device is placed as shown in Figure 1(a), the electronic screen has a row-direction pulse width modulation frequency, oriented vertically downwards. When the electronic device is placed as shown in Figure 1(b), the electronic screen has a column-direction pulse width modulation frequency, oriented horizontally to the right. When the electronic device is placed in other ways, the electronic screen has both row-direction and column-direction pulse width modulation frequencies.

[0117] The row-direction pulse width modulation frequency can be determined based on the row-direction brightness information difference. Similarly, the column-direction pulse width modulation frequency can be determined based on the column-direction brightness information difference.

[0118] Based on the determined row-direction pulse width modulation frequency and column-direction pulse width modulation frequency, the pulse width modulation frequency of the synthesized electronic screen can be obtained.

[0119] By calculating the row-direction pulse width modulation frequency and the column-direction pulse width modulation frequency separately, the pulse width modulation frequency of the electronic screen can be accurately determined.

[0120] Optionally, the non-sub-regions in the row containing the sub-region in the image are defined as the first region; determining the difference in brightness information between the sub-regions in the current frame image and the previous frame image includes:

[0121] For the current frame image, determine the row-direction brightness information of the sub-region, and determine the row-direction brightness information of the first region; the row-direction brightness information is related to the brightness value of each pixel in a row;

[0122] For any row containing a sub-region, calculate the difference between the row-direction brightness information of that row in the sub-region and the row-direction brightness information of that row in the first region, in order to determine the first row-direction brightness information of the sub-region in the current frame image;

[0123] The difference between the first row brightness information of the sub-region in the current frame image and the first row brightness information of the sub-region in the previous frame image is determined as the row brightness information difference.

[0124] When determining the difference in row luminance information between the current frame image and the previous frame image regarding a sub-region, the first row luminance information of the sub-region in the current frame image and the first row luminance information of the sub-region in the previous frame image can be determined separately, and the difference between the two can be obtained to obtain the row luminance information difference.

[0125] Because the sub-region is affected by two frequencies, namely the lighting equipment and the electronic screen, if the two brightness information differs greatly, the pulse width modulation frequency of the electronic screen cannot be accurately obtained without decoupling the two brightness information.

[0126] Figure 5 is a schematic diagram of a sub-region and a first region provided in an embodiment of this application. As shown in Figure 5, the area where the electronic screen is located is the sub-region, which has a width of sub-w and a height of sub-h. The size and position of the sub-region can be adjusted according to the actual scene. The non-sub-regions in the rows where the sub-region is located in the image are the first region. Therefore, the row directional brightness information of each row in the sub-region can be calculated, and the row directional brightness information of each row in the first region can also be calculated.

[0127] Optionally, the row brightness information for each row in the sub-region can be: the average brightness value of each pixel in that row, or the sum of the brightness values ​​of each pixel in that row.

[0128] Since the row-direction brightness information of each row in the first region may differ, for any row in the sub-region, the difference between the row-direction brightness information of that row and the row-direction brightness information of that row in the first region can be calculated to obtain the first row-direction brightness information of the sub-region. The row-direction brightness information of that row in the first region is the brightness information caused by the lighting equipment, and this brightness information also exists in the sub-region. Therefore, by calculating the difference, the row-direction brightness information caused solely by the electronic screen, i.e., the first row-direction brightness information, can be obtained.

[0129] Once the first row luminance information of a sub-region in the current frame image is determined, the difference between the first row luminance information of the sub-region in the current frame image and the first row luminance information of the sub-region in the previous frame image can be calculated to obtain the row luminance information difference.

[0130] For example, as shown in Figure 5, the G channel signal or Y value signal of each row in the first region is statistically analyzed to obtain the row luminance information. At this time, the statistical value can be considered as On + F1n, where O is the image background signal and F1n is the global banding signal caused by the lighting device. The G channel signal or Y value signal of each row in the sub-region is statistically analyzed to obtain the row luminance information. At this time, the statistical value can be considered as On + F1n + F2n, where F2n is the banding signal caused by the electronic screen. The difference signal is obtained by subtracting the global corresponding row statistical diff_global (the difference in the row luminance information of the two frames corresponding to the sub-region) from the diff_sub (the difference in row luminance information of the two frames corresponding to the global) within the row range of the sub-region. The difference signal is the banding signal caused by the electronic screen in the sub-region, which is F2n. The sub-region row banding frequency sub_freq_h can be obtained by performing FFT on the difference signal.

[0131] By calculating the difference between the row directional brightness information of each row in the sub-region and the corresponding row directional brightness information in the first region, the row directional brightness information caused solely by the electronic screen can be accurately obtained, thereby improving the accuracy of the difference in row directional brightness information of the determined sub-region.

[0132] Optionally, determine the difference in brightness information of a sub-region between the current frame image and the previous frame image, including:

[0133] For the current frame image, determine the column-oriented brightness information of the sub-region; the column-oriented brightness information is related to the brightness value of each pixel in a column;

[0134] The difference between the column-directed brightness information of a sub-region in the current frame image and the column-directed brightness information of a sub-region in the previous frame image is determined as the column-directed brightness information difference.

[0135] Similarly, when determining the column-oriented brightness information of a sub-region, the column-oriented brightness information can be calculated separately for each column in the sub-region. Optionally, for any column in the sub-region, the average brightness value of each pixel in that column can be calculated, and the brightness value can be a statistical representation of the G channel or a statistical representation of the Y value.

[0136] After calculating the column-direction brightness information of a sub-region in the current frame image, the difference between the column-direction brightness information of the sub-region in the current frame image and the column-direction brightness information of the sub-region in the previous frame image can be calculated to determine the difference in column-direction brightness information of the sub-region.

[0137] Since the column-direction brightness information of each column in the first region is the same, when calculating the difference in column-direction brightness information of the sub-regions, it is not necessary to subtract the column-direction brightness information of the sub-regions from the column-direction brightness information of the first region.

[0138] The above method allows for a simple and convenient calculation of the column-direction brightness information difference in a sub-region, and the calculated brightness information difference can accurately reflect the column-direction pulse width modulation frequency.

[0139] In addition to detecting the pulse width modulation frequency of an electronic screen, this application can also detect the frequency of the light source corresponding to the non-sub-region.

[0140] Optionally, regions in the image other than the sub-region are considered non-sub-regions; the method also includes:

[0141] For the current frame image, determine the row-wise brightness information of non-sub-regions;

[0142] The global frequency is determined based on the difference between the row brightness information of non-sub-regions in the current frame image and the row brightness information of non-sub-regions in the previous frame image; the global frequency is the frequency of the light sources present in the image.

[0143] As shown in Figure 5, the area outside the sub-region is considered a non-sub-region, which includes the first region. For the non-sub-regions, the row-wise brightness information of each row in the current frame image can be calculated. Optionally, for rows without sub-regions, the average brightness value of each pixel in that row is directly calculated; for rows with sub-regions, only the average brightness value of each pixel belonging to the first region in that row can be calculated to avoid the brightness values ​​of pixels belonging to the sub-region in that row affecting the calculated frequency of the light source.

[0144] Similarly, after determining the row luminance information of the non-sub-regions in the current frame image, the difference between the row luminance information of the non-sub-regions in the current frame image and the row luminance information of the non-sub-regions in the previous frame image can be calculated, and the global frequency, that is, the frequency of the light source, can be determined based on the calculated difference.

[0145] By only statistically analyzing the row luminance information of non-sub-regions, the frequency of the light source can be accurately detected.

[0146] Optionally, determine the coordinate information of the sub-region, including:

[0147] Semantic segmentation is used to determine the coordinate information of the area where the electronic screen is located, in order to determine the coordinate information of the sub-region.

[0148] When determining the coordinate information of a sub-region, semantic segmentation can be used to determine the area where the electronic screen is located and the coordinate information of the area where the electronic screen is located, so as to obtain the coordinate information of the sub-region.

[0149] Semantic segmentation can assign a semantic category to each pixel in an image. For example, when an electronic screen (electronic device) is obtained through semantic segmentation, the area where the electronic screen is located can be determined as a sub-region, thereby obtaining the coordinate information of the sub-region.

[0150] By determining the coordinate information of sub-regions through semantic segmentation, the coordinate information of sub-regions can be accurately obtained without user intervention, and the determined sub-regions can be dynamically changed.

[0151] Figure 6 is a schematic diagram of an image to be detected according to an embodiment of this application. Figure 7 is a schematic diagram of a determined global frequency in Figure 6 according to an embodiment of this application. Figure 8 is a schematic diagram of a determined pulse width modulation frequency of the electronic screen in Figure 6 according to an embodiment of this application. By using a signal decoupling method to separate the row luminance information corresponding to the lighting device from the row luminance information corresponding to the sub-region, the pulse width modulation frequency of the electronic screen and the frequency of the light source in the sub-region can be calculated separately using FFT. Combining Figures 6, 7, and 8, it can be seen that Figure 6 is an image obtained at a global frequency of 100 Hz and a row directional pulse width modulation frequency of 960 Hz in the sub-region. As shown in Figures 7 and 8, the method of this application can detect the global frequency and the row directional pulse width modulation frequency of the sub-region separately.

[0152] Furthermore, the stripes formed within the sub-region may be of three types, such as horizontal stripes, vertical stripes, or diagonal stripes, and this application can detect the corresponding pulse width modulation frequency respectively.

[0153] Figure 9 is a schematic diagram of an image to be detected according to an embodiment of this application. Figure 10 is a schematic diagram of a determined row pulse width modulation frequency of the electronic screen in Figure 9 according to an embodiment of this application. Figure 11 is a schematic diagram of a determined column pulse width modulation frequency of the electronic screen in Figure 9 according to an embodiment of this application. Combining Figures 10, 11, and 12, it can be seen that Figure 9 is an image obtained when the row pulse width modulation frequency of the sub-region is 320 Hz and the column pulse width modulation frequency is 960 Hz. As shown in Figures 10 and 11, the method of this application can detect the row and column pulse width modulation frequencies of the sub-region respectively.

[0154] After determining the pulse width modulation frequency of the sub-region and the global frequency, flicker can be eliminated.

[0155] Figure 12 is a flowchart illustrating a screen flicker elimination method provided in an embodiment of this application. As shown in Figure 12, the method includes steps S1201 to S1203:

[0156] Step S1201: Determine the limiting frequency of the camera device; the limiting frequency is the minimum frequency value that the camera device supports for flicker removal; the camera device is used to acquire images including those from electronic screens.

[0157] When eliminating flicker, the limiting frequency of the camera device, which can be the camera unit in a head-mounted display, can be determined first. The limiting frequency is the minimum frequency value supported by the camera device, and it is the minimum frequency value that the camera device can remove. This limiting frequency is related to the hardware capabilities of the camera device (min fps, minimum frame rate).

[0158] Among them, the camera equipment is used to capture images. If the captured image includes an electronic screen, the captured image may flicker due to the presence of the electronic screen, which is the stripes shown in Figure 1.

[0159] Step S1202: When both the pulse width modulation frequency and the global frequency of the electronic screen exist, determine the target exposure time based on the pulse width modulation frequency, global frequency, and limit frequency of the electronic screen.

[0160] When two frequencies coexist—that is, the pulse-width modulation frequency and the global frequency of the electronic screen—image flicker will occur during image capture, exhibiting flicker in sub-regions and flicker in non-sub-regions. To eliminate both types of flicker as much as possible, the target exposure time can be determined based on the electronic screen's pulse-width modulation frequency, global frequency, and limiting frequency, thereby eliminating both types of flicker simultaneously, or eliminating only one type of flicker. Optionally, the global frequency can be the frequency of light sources present in the scene being captured. Furthermore, the global frequency can also be other frequencies that can cause flicker throughout the image.

[0161] When only one frequency exists, the target exposure time can be determined based on that single frequency in order to eliminate the flicker caused by that frequency.

[0162] Step S1203: Adjust the actual exposure time of the camera equipment to the target exposure time.

[0163] Once the target exposure time is determined, the actual exposure time of the camera can be adjusted so that the camera can obtain an image with flicker eliminated when capturing the image.

[0164] For example, when the global frequency is 100 Hz and the pulse width modulation frequency of the electronic screen is 1000 Hz, the target exposure time can be set to 10 ms, or an integer multiple of 10 ms, to eliminate both types of flicker at the same time, so that there are no two types of flicker in the image captured by the camera device.

[0165] For example, when the global frequency is 100 Hz and the pulse width modulation frequency of the electronic screen is 960 Hz, the target exposure time can be set to 10 ms to eliminate the flicker corresponding to the light source. Then, the image captured by the camera will not have the flicker caused by the light source, but it will have the flicker caused by the electronic screen.

[0166] Optionally, the pulse width modulation frequency of the electronic screen may not be limited to the implementation methods of the foregoing embodiments.

[0167] This application provides a method for eliminating screen flicker by determining the limiting frequency of the camera device; the limiting frequency is the minimum frequency value supported by the camera device for flicker removal; the camera device is used to capture images including an electronic screen; when both the pulse width modulation frequency and the global frequency of the electronic screen exist, the target exposure time is determined according to the pulse width modulation frequency, the global frequency and the limiting frequency of the electronic screen, and the actual exposure time of the camera device is adjusted to the target exposure time, so as to minimize flicker when capturing images.

[0168] Figure 13 is a schematic diagram of a screen flicker elimination process provided in an embodiment of this application. As shown in Figure 13, when both the global frequency and the pulse modulation frequency of the electronic screen exist simultaneously, the greatest common factor of the two frequencies can be calculated. It is then determined whether the greatest common factor is greater than the limiting frequency. If it is greater than (or equal to) the limiting frequency, the target exposure time can be determined. After determining the target exposure time, it can be determined whether the current frame rate supports the target exposure time. If it does, the actual exposure time is adjusted to the target exposure time. If it does not support it, the frame rate is adjusted, and the actual exposure time is adjusted to the target exposure time. Furthermore, when the greatest common factor is less than the limiting frequency, the target exposure time can be determined to eliminate flicker corresponding to the frequency type of interest to the user, while simultaneously suppressing flicker corresponding to another frequency type. The above process will be described in detail below.

[0169] Optionally, the target exposure time is determined based on the pulse width modulation frequency, global frequency, and limiting frequency of the electronic screen, including:

[0170] Determine the greatest common factor of the pulse width modulation frequency and the global frequency of the electronic screen;

[0171] When the greatest common factor is greater than or equal to the limiting frequency, the target exposure time is determined based on the greatest common factor and the limiting frequency.

[0172] When both the pulse width modulation frequency and the global frequency of the electronic screen exist simultaneously, their greatest common factor can be calculated, and the target exposure time can be determined based on this greatest common factor.

[0173] For example, when the global frequency is 100 and the pulse width modulation frequency of the electronic screen is 1000 Hz, the greatest common factor is 100. Therefore, the period corresponding to the global frequency is 10 ms and the period corresponding to the pulse width modulation frequency of the electronic screen is 1 ms. Since there will be no flicker when the exposure time is an integer multiple of the period, when the exposure time is set to an integer multiple of the period corresponding to the global frequency, the exposure time is simultaneously an integer multiple of the periods corresponding to both frequencies, thereby eliminating both types of flicker at the same time.

[0174] Optionally, the target exposure time can be determined based on the greatest common factor and the limiting frequency, including:

[0175] Determine the minimum exposure time based on the greatest common factor;

[0176] The maximum exposure time supported by the camera equipment is determined based on the limiting frequency.

[0177] The target exposure time is determined based on the minimum and maximum exposure times; the target exposure time is an integer multiple of the minimum exposure time and is less than or equal to the maximum exposure time.

[0178] The minimum exposure time corresponding to the greatest common factor refers to the minimum exposure time to simultaneously eliminate both types of flicker.

[0179] After determining the greatest common factor (GCF), we can also determine the relationship between the GCF and the limiting frequency. The limiting frequency corresponds to the maximum exposure time supported by the camera equipment. When the GCF is greater than the limiting frequency, the exposure time corresponding to the GCF is less than the maximum exposure time. Therefore, the target exposure time can be determined based on the GCF.

[0180] For example, when the greatest common factor (GCF) is 100 and the limiting frequency is 30, the maximum exposure time is 33.3ms. Exposure times calculated based on the GCF can be 10ms, 20ms, 30ms, etc., thus yielding the target exposure time. However, when the GCF is 20, the calculated exposure times can be 50ms, 100ms, 150ms, etc., all of which are greater than the maximum exposure time, making it impossible to obtain the target exposure time achievable by the camera.

[0181] By calculating the greatest common factor of the two frequencies, the minimum exposure time that can simultaneously eliminate both types of flicker can be calculated. At the same time, based on the comparison between the greatest common factor and the limiting frequency, it can be determined whether the minimum exposure time that can eliminate both types of flicker is an exposure time that the camera equipment can support, thereby improving the feasibility of the determined target exposure time.

[0182] Optionally, the method also includes:

[0183] When there is no greatest common factor, or when the greatest common factor is less than the limiting frequency, the target exposure time is determined based on the pulse width modulation frequency, global frequency, first frequency type and limiting frequency of the electronic screen; the first frequency type is the frequency type that is preferentially eliminated; the second frequency type is the frequency type other than the first frequency type among the pulse width modulation frequency type and global frequency type of the electronic screen.

[0184] The target exposure time is used to eliminate flicker corresponding to the first frequency type and suppress flicker corresponding to the second frequency type.

[0185] If there is no greatest common factor or the greatest common factor is less than the limiting frequency, it means that it is impossible to eliminate both types of flicker at the same time.

[0186] When it is impossible to eliminate both types of flickering simultaneously, one type of flickering can be eliminated first according to the user's needs. Optionally, the frequency type to be eliminated can be preset, or a prompt message can be sent to the user to obtain the frequency type to be eliminated first when it is impossible to eliminate both types of flickering simultaneously.

[0187] Once the frequency type to be prioritized for elimination, i.e., the first frequency type, is determined, the target exposure time can be determined based on the electronic screen's pulse width modulation frequency, global frequency, first frequency type, and limiting frequency. Optionally, when eliminating flicker corresponding to the first frequency type, to improve the user experience, flicker corresponding to the second frequency type can be suppressed as much as possible.

[0188] For example, when the two frequencies are 100 Hz and 960 Hz, the greatest common factor is 10. When the limiting frequency is 30 Hz, the greatest common factor is less than 30, so it is impossible to eliminate both frequencies simultaneously. In this case, if the frequency of the first frequency type is determined to be 100 Hz, multiple exposure times can be determined based on 100 Hz, such as 10 ms, 20 ms, 30 ms, etc. The flicker caused by the second frequency type differs at each of these exposure times. Therefore, a target exposure time can be determined from multiple exposure times to eliminate the flicker corresponding to the second frequency type as much as possible.

[0189] Optionally, when there is no greatest common factor, or when the greatest common factor is less than the limiting frequency, a target exposure time can be determined to suppress both types of flicker simultaneously. In this case, although neither of the two types of flicker can be eliminated, both types of flicker can be effectively suppressed.

[0190] By setting the first frequency type to be eliminated first, the target exposure time for eliminating the flicker corresponding to the first frequency type and suppressing the flicker corresponding to the second frequency type can be obtained, thereby reducing the intensity of the flicker corresponding to the second frequency type in the captured image.

[0191] Optionally, the target exposure time is determined based on the pulse width modulation frequency, global frequency, first frequency type, and limiting frequency of the electronic screen, including:

[0192] When the frequency corresponding to the first frequency type is greater than or equal to the limiting frequency, multiple first exposure times are determined according to the frequency of the first frequency type; the first exposure time is less than the maximum exposure time supported by the camera device determined by the limiting frequency.

[0193] A second exposure time is determined. For any first exposure time, a third exposure time is determined based on the second exposure time, and the difference between the second exposure time and the first exposure time is less than a preset value. The second exposure time is the minimum exposure time that can eliminate the flicker corresponding to the second frequency type. The third exposure time is an integer multiple of the second exposure time.

[0194] The target exposure time is determined from multiple first exposure times; the target exposure time is the first exposure time with the smallest difference from the corresponding third exposure time.

[0195] When it is not possible to eliminate both types of flicker simultaneously, the selected exposure time can be set to be as close as possible to an integer multiple of the exposure time corresponding to the other frequency.

[0196] Optionally, multiple first exposure times can be calculated based on the frequency corresponding to the first frequency type. For example, when the frequency corresponding to the first frequency type is 100 Hz and the limiting frequency is 40 Hz, the maximum exposure time is 25 ms. The determined multiple first exposure times can be 10 ms or 20 ms, where the first exposure time is less than the maximum exposure time.

[0197] Once the first exposure time is determined, a target exposure time can be selected from among them. Specifically, a second exposure time can be determined first, which is the minimum exposure time required to eliminate the flicker corresponding to the second frequency type. The third exposure time is an integer multiple of the second exposure time, meaning the third exposure time can also eliminate the flicker corresponding to the second frequency type. When the frequency corresponding to the second frequency type is 960 Hz, the second exposure time is 1.042 ms, then the third exposure time could be 1.042 ms, 2.084 ms, 3.126 ms, and so on.

[0198] After determining the third exposure time, a target exposure time can be selected from the first exposure time. For example, for the first exposure time, calculate the difference between it and the corresponding third exposure time, and determine the first exposure time with the smallest difference as the target exposure time. When the first exposure time is 10ms, and the third exposure time is 10.42ms (10 times the second exposure time), the difference between the first and third exposure times is 0.42; when the third exposure time is 9.378ms (9 times the second exposure time), the difference is 0.622. When the first exposure time is 20ms, and the third exposure time is 19.798ms (19 times the second exposure time), the difference is 0.202; when the third exposure time is 20.84ms (20 times the second exposure time), the difference is 0.84. Therefore, when the first exposure time is 20ms, flicker corresponding to the first frequency type can be eliminated, and flicker corresponding to the second frequency type can be suppressed to the greatest extent possible.

[0199] By using the method described above for calculating the target exposure time, it is possible to eliminate one type of flicker while suppressing another type of flicker as much as possible, thereby reducing the intensity of flicker in the image.

[0200] Optionally, the method also includes:

[0201] When only the pulse width modulation frequency of the electronic screen exists, and when the pulse width modulation frequency of the electronic screen is greater than or equal to the threshold frequency, the target exposure time is determined based on the pulse width modulation frequency of the electronic screen; and / or,

[0202] When only the global frequency exists, and the global frequency is greater than or equal to the limiting frequency, the target exposure time is determined based on the global frequency.

[0203] When the image contains only flicker corresponding to the pulse width modulation frequency of the electronic screen, or only flicker corresponding to the global frequency, if the existing frequency is greater than the limit frequency, which is the minimum frequency value supported by the camera device, the target exposure time can be determined based on the frequency to eliminate the flicker corresponding to that frequency.

[0204] For example, when the limit frequency is 30, if only the global frequency exists and the global frequency is 100, which is greater than the limit frequency, the target exposure time can be set to 10ms, or an integer multiple of 10ms, to eliminate the flicker corresponding to the global frequency.

[0205] For example, when the limiting frequency is 30, if only the pulse width modulation frequency of the electronic screen exists, and the pulse width modulation frequency is 960, which is greater than the limiting frequency, the target exposure time can be set to 1.042ms, or an integer multiple of 1.042ms, to eliminate the flicker corresponding to the pulse width modulation frequency of the electronic screen.

[0206] When there is only one frequency that causes flicker, this frequency is compared with the limiting frequency, and the target exposure time is determined based on this frequency in order to eliminate the flicker corresponding to that frequency.

[0207] Optionally, there are multiple target exposure times. The actual exposure time of the camera device is adjusted to the target exposure time, including:

[0208] Determine the maximum allowed exposure time based on the current frame rate;

[0209] If there exists a target exposure time that is less than or equal to the current maximum allowed exposure time, then the actual exposure time is adjusted to the target exposure time; and / or,

[0210] If all target exposure times are greater than the current maximum allowed exposure time, adjust the frame rate and adjust the actual exposure time to any of the target exposure times.

[0211] The camera also has a current frame rate, which affects the maximum allowed exposure time. If there are multiple calculated target exposure times, and one of them is shorter than the maximum allowed exposure time, the actual exposure time can be adjusted to that target exposure time.

[0212] For example, if the maximum allowed exposure time is 20ms and the calculated target exposure time is 10ms or 20ms, then 10ms or 20ms can be adjusted to the actual exposure time.

[0213] Conversely, if all calculated target exposure times are greater than the current maximum allowed exposure time, the frame rate can be adjusted to adjust the current maximum allowed exposure time, so that at least one of the calculated target exposure times is less than or equal to the adjusted current maximum allowed exposure time.

[0214] For example, if the current maximum allowed exposure time is 10ms, and the calculated target exposure time is 15ms or 30ms, the current frame rate can be adjusted to ensure that the current maximum allowed exposure time is at least 15ms.

[0215] The actual exposure time is adjusted by determining the maximum allowed exposure time corresponding to the current frame rate.

[0216] Figure 14 is a schematic diagram of a screen flicker frequency detection device provided in an embodiment of this application. The device 140 includes:

[0217] The first determining module 1401 is configured to determine the coordinate information of a sub-region; the sub-region is the area where the electronic screen is located in the acquired image.

[0218] Processing module 1402 is configured to determine the difference in brightness information of a sub-region between the current frame image and the previous frame image based on coordinate information; the difference in brightness information is the difference between the target brightness information corresponding to the current frame image and the target brightness information corresponding to the previous frame image; the target brightness information is the brightness information caused only by the electronic screen;

[0219] The second determining module 1403 is configured to determine the pulse width modulation frequency of the electronic screen based on the brightness information difference.

[0220] Optionally, the brightness information difference includes row-wise brightness information difference and column-wise brightness information difference; when the second determining module 1403 determines the pulse width modulation frequency of the electronic screen based on the brightness information difference, it is specifically configured as follows:

[0221] The row pulse width modulation frequency is determined based on the row brightness information difference;

[0222] The column pulse width modulation frequency is determined based on the column brightness information difference;

[0223] The row-wise pulse width modulation frequency and the column-wise pulse width modulation frequency are combined to determine the pulse width modulation frequency of the electronic screen.

[0224] Optionally, the non-sub-region in the row containing the sub-region in the image is the first region; when determining the difference in brightness information between the sub-regions in the current frame image and the previous frame image, the processing module 1402 is specifically configured as follows:

[0225] For the current frame image, determine the row-direction brightness information of the sub-region, and determine the row-direction brightness information of the first region; the row-direction brightness information is related to the brightness value of each pixel in a row;

[0226] For any row containing a sub-region, calculate the difference between the row-direction brightness information of that row in the sub-region and the row-direction brightness information of that row in the first region, in order to determine the first row-direction brightness information of the sub-region in the current frame image;

[0227] The difference between the first row brightness information of the sub-region in the current frame image and the first row brightness information of the sub-region in the previous frame image is determined as the row brightness information difference.

[0228] Optionally, when the processing module 1402 determines the difference in brightness information of a sub-region between the current frame image and the previous frame image, it is specifically configured as follows:

[0229] For the current frame image, determine the column-oriented brightness information of the sub-region; the column-oriented brightness information is related to the brightness value of each pixel in a column;

[0230] The difference between the column-directed brightness information of a sub-region in the current frame image and the column-directed brightness information of a sub-region in the previous frame image is determined as the column-directed brightness information difference.

[0231] Optionally, regions in the image other than sub-regions are considered non-sub-regions; the device further includes a global frequency determination module, configured to:

[0232] For the current frame image, determine the row-wise brightness information of non-sub-regions;

[0233] The global frequency is determined based on the difference between the row brightness information of non-sub-regions in the current frame image and the row brightness information of non-sub-regions in the previous frame image; the global frequency is the frequency of the light sources present in the image.

[0234] Optionally, the first determining module 1401 is specifically configured as follows:

[0235] Semantic segmentation is used to determine the coordinate information of the area where the electronic screen is located, in order to determine the coordinate information of the sub-region.

[0236] The screen flicker frequency detection device 140 provided in this application embodiment can realize the screen flicker frequency detection method shown in FIG4 above. Its implementation principle and technical effect are similar, and will not be described again here.

[0237] Figure 15 is a schematic diagram of a screen flicker elimination device provided in an embodiment of this application. The device 150 includes:

[0238] The third determining module 1501 is configured to determine the limiting frequency of the camera device; the limiting frequency is the minimum frequency value that the camera device supports for flicker removal; the camera device is used to acquire images including those from an electronic screen;

[0239] The fourth determining module 1502 is configured to determine the target exposure time based on the pulse width modulation frequency, global frequency and limit frequency of the electronic screen when both the pulse width modulation frequency and the global frequency of the electronic screen exist.

[0240] The adjustment module 1503 is configured to adjust the actual exposure time of the camera device to the target exposure time.

[0241] Optionally, when determining the target exposure time based on the pulse width modulation frequency, global frequency, and limiting frequency of the electronic screen, the fourth determining module 1502 is specifically configured as follows:

[0242] Determine the greatest common factor of the pulse width modulation frequency and the global frequency of the electronic screen;

[0243] When the greatest common factor is greater than or equal to the limiting frequency, the target exposure time is determined based on the greatest common factor and the limiting frequency.

[0244] Optionally, the fourth determining module 1502 is specifically configured to determine the target exposure time based on the greatest common factor and the limiting frequency as follows:

[0245] Determine the minimum exposure time based on the greatest common factor;

[0246] The maximum exposure time supported by the camera equipment is determined based on the limiting frequency.

[0247] The target exposure time is determined based on the minimum and maximum exposure times; the target exposure time is an integer multiple of the minimum exposure time and is less than or equal to the maximum exposure time.

[0248] Optionally, the device further includes: a first processing module configured to:

[0249] When there is no greatest common factor, or when the greatest common factor is less than the limiting frequency, the target exposure time is determined based on the pulse width modulation frequency, global frequency, first frequency type and limiting frequency of the electronic screen; the first frequency type is the frequency type that is preferentially eliminated; the second frequency type is the frequency type other than the first frequency type among the pulse width modulation frequency type and global frequency type of the electronic screen.

[0250] The target exposure time is used to eliminate flicker corresponding to the first frequency type and suppress flicker corresponding to the second frequency type.

[0251] Optionally, when determining the target exposure time based on the pulse width modulation frequency, global frequency, first frequency type, and limiting frequency of the electronic screen, the first processing module is specifically configured as follows:

[0252] When the frequency corresponding to the first frequency type is greater than or equal to the limiting frequency, multiple first exposure times are determined according to the frequency of the first frequency type; the first exposure time is less than the maximum exposure time supported by the camera device determined by the limiting frequency.

[0253] A second exposure time is determined. For any first exposure time, a third exposure time is determined based on the second exposure time, and the difference between the second exposure time and the first exposure time is less than a preset value. The second exposure time is the minimum exposure time that can eliminate the flicker corresponding to the second frequency type. The third exposure time is an integer multiple of the second exposure time.

[0254] The target exposure time is determined from multiple first exposure times; the target exposure time is the first exposure time with the smallest difference from the corresponding third exposure time.

[0255] Optionally, the device further includes: a second processing module configured to:

[0256] When only the pulse width modulation frequency of the electronic screen exists, and when the pulse width modulation frequency of the electronic screen is greater than or equal to the threshold frequency, the target exposure time is determined based on the pulse width modulation frequency of the electronic screen; and / or,

[0257] When only the global frequency exists, and the global frequency is greater than or equal to the limiting frequency, the target exposure time is determined based on the global frequency.

[0258] Optionally, there are multiple target exposure times, and the adjustment module 1503 is specifically configured as follows:

[0259] Determine the maximum allowed exposure time based on the current frame rate;

[0260] If there exists a target exposure time that is less than or equal to the current maximum allowed exposure time, then the actual exposure time is adjusted to the target exposure time; and / or,

[0261] If all target exposure times are greater than the current maximum allowed exposure time, adjust the frame rate and adjust the actual exposure time to any of the target exposure times.

[0262] The screen flicker elimination device 150 provided in this application embodiment can realize the screen flicker elimination method shown in FIG12 above. Its implementation principle and technical effect are similar, and will not be described again here.

[0263] Figure 16 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. As shown in Figure 16, the electronic device provided in this embodiment includes at least one processor 1601 and a memory 1602. The processor 1601 and the memory 1602 are connected via a bus 1603.

[0264] In a specific implementation, at least one processor 1601 executes computer execution instructions stored in memory 1602, causing at least one processor 1601 to execute the method in the above method embodiment.

[0265] The specific implementation process of processor 1601 can be found in the above method embodiments, and its implementation principle and technical effect are similar, so it will not be repeated here.

[0266] Optionally, the electronic device can be a head-mounted display device.

[0267] In the embodiment shown in Figure 16 above, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0268] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage.

[0269] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0270] This application also provides a wearable device, including a camera device and a processing unit; the camera device is used to capture images; the processing unit is used to execute any one of the aforementioned image flicker frequency detection method and image flicker elimination method; or, the processing unit is used to execute any one of the image flicker frequency detection methods; or, the processing unit is used to execute any one of the image flicker elimination methods.

[0271] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in the above-described method embodiments.

[0272] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the above method embodiments.

[0273] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0274] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0275] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0276] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0277] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.

[0278] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for detecting screen flicker frequency, characterized in that, include: Determine the coordinate information of the sub-region; the sub-region is the area where the electronic screen is located in the acquired image. Based on the coordinate information, determine the difference in brightness information between the sub-regions in the current frame image and the previous frame image; The brightness information difference is the difference between the target brightness information corresponding to the current frame image and the target brightness information corresponding to the previous frame image; The target brightness information is the brightness information caused solely by the electronic screen; The pulse width modulation frequency of the electronic screen is determined based on the brightness information difference.

2. The method according to claim 1, characterized in that, The brightness information difference includes row-wise brightness information difference and column-wise brightness information difference; Determining the pulse width modulation frequency of the electronic screen based on the brightness information difference includes: The row pulse width modulation frequency is determined based on the row brightness information difference; The column pulse width modulation frequency is determined based on the column brightness information difference; The row-direction pulse width modulation frequency and the column-direction pulse width modulation frequency are combined to determine the pulse width modulation frequency of the electronic screen.

3. The method according to claim 2, characterized in that, The non-sub-regions in the row containing the sub-region in the image are the first region; determining the difference in brightness information between the sub-regions in the current frame image and the previous frame image includes: For the current frame image, determine the row luminance information of the sub-region, and determine the row luminance information of the first region; the row luminance information is related to the luminance value of each pixel in a row; for any row in which the sub-region is located, calculate the difference between the row luminance information of the row in the sub-region and the row luminance information of the row in the first region, so as to determine the first row luminance information of the sub-region in the current frame image; The difference between the first row brightness information of the sub-region in the current frame image and the first row brightness information of the sub-region in the previous frame image is determined as the row brightness information difference.

4. The method according to claim 2, characterized in that, Determining the difference in brightness information between the sub-regions in the current frame image and the previous frame image includes: For the current frame image, determine the column-oriented brightness information of the sub-region; the column-oriented brightness information is related to the brightness value of each pixel in a column; The difference between the column-directed brightness information of the sub-region in the current frame image and the column-directed brightness information of the sub-region in the previous frame image is determined as the column-directed brightness information difference.

5. The method according to any one of claims 1-4, characterized in that, The regions in the image other than the sub-region are non-sub-regions; the method further includes: For the current frame image, determine the row-direction brightness information of the non-sub-region; The global frequency is determined based on the difference between the row brightness information of the non-sub-region in the current frame image and the row brightness information of the non-sub-region in the previous frame image; the global frequency is the frequency of the light source present in the image.

6. The method according to any one of claims 1-5, characterized in that, Determine the coordinate information of the sub-region, including: The coordinate information corresponding to the area where the electronic screen is located is determined by semantic segmentation, so as to determine the coordinate information of the sub-region.

7. A method for eliminating screen flicker, characterized in that, include: Determine the limiting frequency of the camera equipment; The limiting frequency is the minimum frequency value supported by the camera device for flicker removal; the camera device is used to capture images including those from an electronic screen; When both the pulse width modulation frequency and the global frequency of the electronic screen exist, the target exposure time is determined based on the pulse width modulation frequency of the electronic screen, the global frequency, and the limiting frequency. Adjust the actual exposure time of the camera device to the target exposure time.

8. The method according to claim 7, characterized in that, Determining the target exposure time based on the pulse width modulation frequency of the electronic screen, the global frequency, and the limiting frequency includes: Determine the greatest common factor of the pulse width modulation frequency of the electronic screen and the global frequency; When the greatest common factor is greater than or equal to the limiting frequency, the target exposure time is determined based on the greatest common factor and the limiting frequency.

9. The method according to claim 8, characterized in that, Determining the target exposure time based on the greatest common factor and the limiting frequency includes: The minimum exposure time is determined based on the greatest common factor. The maximum exposure time supported by the camera device is determined based on the specified limiting frequency. The target exposure time is determined based on the minimum exposure time and the maximum exposure time; the target exposure time is an integer multiple of the minimum exposure time and is less than or equal to the maximum exposure time.

10. The method according to claim 8, characterized in that, The method further includes: When the greatest common factor does not exist, or when the greatest common factor is less than the limiting frequency, the target exposure time is determined based on the pulse width modulation frequency of the electronic screen, the global frequency, the first frequency type, and the limiting frequency; the first frequency type is the frequency type that is preferentially eliminated; the second frequency type is the frequency type other than the first frequency type among the pulse width modulation frequency type of the electronic screen and the global frequency type. The target exposure time is used to eliminate flicker corresponding to the first frequency type and suppress flicker corresponding to the second frequency type.

11. The method according to claim 10, characterized in that, The target exposure time is determined based on the pulse width modulation frequency of the electronic screen, the global frequency, the first frequency type, and the limiting frequency, including: When the frequency corresponding to the first frequency type is greater than or equal to the limiting frequency, a plurality of first exposure times are determined according to the frequency of the first frequency type; the first exposure time is less than the maximum exposure time supported by the camera device as determined by the limiting frequency; A second exposure time is determined. For any first exposure time, a third exposure time is determined based on the second exposure time, and the difference between the second exposure time and the first exposure time is less than a preset value. The second exposure time is the minimum exposure time that can eliminate the flicker corresponding to the second frequency type. The third exposure time is an integer multiple of the second exposure time. The target exposure time is determined from a plurality of first exposure times; the target exposure time is the first exposure time with the smallest difference from the corresponding third exposure time.

12. The method according to any one of claims 7-11, characterized in that, The method further includes: When only the pulse width modulation frequency of the electronic screen exists, and when the pulse width modulation frequency of the electronic screen is greater than or equal to the threshold frequency, the target exposure time is determined based on the pulse width modulation frequency of the electronic screen; and / or, When only the global frequency exists, and when the global frequency is greater than or equal to the limiting frequency, the target exposure time is determined based on the global frequency.

13. The method according to any one of claims 7-12, characterized in that, The target exposure time is multiple, and adjusting the actual exposure time of the camera device to the target exposure time includes: Determine the maximum allowed exposure time based on the current frame rate; If any of the target exposure times exists that is less than or equal to the currently allowed maximum exposure time, then the actual exposure time is adjusted to the target exposure time; and / or, If all the target exposure times are greater than the current maximum allowed exposure time, then the frame rate is adjusted and the actual exposure time is adjusted to any of the target exposure times.

14. A device for detecting screen flicker frequency, characterized in that, The device includes: The first determining module is configured to determine the coordinate information of a sub-region; the sub-region is the area where the electronic screen is located in the acquired image. The processing module is configured to determine the difference in brightness information between the sub-regions in the current frame image and the previous frame image based on the coordinate information; the difference in brightness information is the difference between the target brightness information corresponding to the current frame image and the target brightness information corresponding to the previous frame image; the target brightness information is the brightness information caused solely by the electronic screen. The second determining module is configured to determine the pulse width modulation frequency of the electronic screen based on the brightness information difference.

15. A screen flicker elimination device, characterized in that, include: The third determining module is configured to determine the limiting frequency of the camera device; The limiting frequency is the minimum frequency value supported by the camera device for flicker removal; the camera device is used to capture images including those from an electronic screen; The fourth determining module is configured to determine the target exposure time based on the pulse width modulation frequency of the electronic screen, the global frequency, and the limiting frequency when both the pulse width modulation frequency and the global frequency of the electronic screen exist. The adjustment module is configured to adjust the actual exposure time of the camera device to the target exposure time.

16. An electronic device, characterized in that, include: At least one processor and memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method as described in any one of claims 1 to 13.

17. A wearable device, characterized in that, Includes camera equipment and processing unit; The camera device is used to capture images; The processing unit is configured to perform the method as described in any one of claims 1 to 6; or, the processing unit is configured to perform the method as described in any one of claims 7 to 13; or, the processing unit is configured to perform the method as described in any one of claims 1 to 13.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1 to 13.

19. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 13.

Images