Method and apparatus for compensating backlight by using tolerance information

Abstract

A method and apparatus for compensating backlight using automatic tolerance information is disclosed. A backlight compensation device includes: at least one memory; and one or more processors wherein, by executing the instructions, the at least one processor is configured to: receive, by the in-vehicle system, an image from the camera, estimate a location of a horizon line in the image using intrinsic and extrinsic parameters of the camera, set a region of interest (ROI) based on the location of the horizon line, divide the ROI into a plurality of blocks, and transmit the blocks to the in-vehicle system. And measuring an average luminance value of each of the plurality of blocks, grouping the blocks into a plurality of block groups based on the average luminance value of each block, determining a backlight condition of the ROI according to a predetermined average luminance value of each block in the plurality of block groups, and adjusting the plurality of block luminance values according to the backlight condition of the ROI.

Description

[0001] Cross-reference of related applications

[0002] This application claims priority to Korean Patent Application No. 10-2024-0055686, filed with the Korean Intellectual Property Office on April 25, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to a method and apparatus for compensating backlight using automated tolerance information. Background Technology

[0004] The following description provides background information relevant to this embodiment only and does not constitute prior art.

[0005] With advancements in autonomous driving technology, various camera-based image recognition systems are being used in vehicles to identify their surroundings. However, backlighting can occur due to the presence of various light sources around a vehicle. Images captured under backlighting conditions are difficult to interpret because the brightness of certain areas (such as white holes) is distorted. In particular, the recognition performance of vehicles equipped with wide-angle cameras deteriorates because these cameras are frequently exposed to various backlighting conditions. This reduces the driver's visibility in dark areas under backlighting conditions and limits the recognition performance of autonomous driving and parking systems.

[0006] To address problems caused by backlighting, numerous methods have been proposed for detecting and correcting backlit images in photographs or videos. However, traditional methods are based on software-based approaches capable of handling limited conditions such as backlighting and light blur. For example, histogram equalization (HE) operates unconditionally across the global domain or region of interest (ROI) of an image. In other words, because traditional contrast enhancement methods operate in the frequency domain or pixel-wise, their correction performance deteriorates when the backlit image itself is problematic. Summary of the Invention

[0007] This disclosure aims to improve the ability to identify solar light sources by segmenting regions of interest that belong to the sky region, accurately identifying solar light sources in the segmented regions of interest, and then performing backlight compensation, thereby providing corrected images.

[0008] The technical objectives of this disclosure are not limited to the foregoing description, and those skilled in the art will clearly understand other technical objectives not mentioned above from the following description. Embodiments of this disclosure provide an apparatus for a backlight compensation device, comprising: at least one memory; and one or more processors, wherein, by executing instructions, the at least one processor is configured to: receive an image from a camera via an in-vehicle system; estimate the position of a horizon in the image using intrinsic and extrinsic parameters of the camera; set a region of interest (ROI) based on the position of the horizon; divide the ROI into multiple blocks and measure the average brightness value of each of the multiple blocks; group the blocks into multiple block groups based on the average brightness value of each block; determine backlight conditions for the ROI according to a predetermined average brightness value of each block in the multiple block groups; and adjust the brightness values ​​of the multiple blocks according to the backlight conditions of the ROI.

[0009] According to embodiments of this disclosure, a method for implementing a backlight compensation device includes: an in-vehicle system receiving an image from a camera; estimating the position of a horizon in the image using intrinsic and extrinsic parameters of the camera; setting a region of interest (ROI) based on the position of the horizon; dividing the ROI into multiple blocks and measuring the average brightness value of each of the multiple blocks; grouping the blocks into multiple block groups based on the average brightness value of each block; determining a backlight condition for the ROI based on a predetermined average brightness value of each block in the multiple block groups; and adjusting the brightness values ​​of the multiple blocks according to the backlight condition of the ROI.

[0010] According to one embodiment of this disclosure, the accuracy of identifying solar light sources can be improved by segmenting only the region of interest corresponding to the sky area and applying more specific conditions within the same resource.

[0011] By performing brightness correction on backlit images in parking or driving environments and improving visibility in dark areas, users' ability to intuitively identify parking environments can be improved.

[0012] Under backlight conditions, it can improve the image recognition performance of objects such as pedestrians, motorcycles and lanes in dark areas.

[0013] The technical effects of this disclosure are not limited to those described above. Those skilled in the art to which this disclosure pertains may learn of other technical effects not mentioned herein from the following description. Attached Figure Description

[0014] Figure 1 This is a block diagram that briefly illustrates a backlight compensation device according to an embodiment of the present disclosure.

[0015] Figure 2 This is a block diagram illustrating an automatic tolerance compensation system according to an embodiment of the present disclosure.

[0016] Figure 3This is a block diagram illustrating an image processing system according to an embodiment of the present disclosure.

[0017] Figure 4 This is a block diagram illustrating a navigation system according to an embodiment of the present disclosure.

[0018] Figure 5A The process of deriving extrinsic parameters using a camera coordinate system is shown.

[0019] Figure 5B The process of deriving the intrinsic parameters of a wide-angle camera is shown.

[0020] Figure 6 V using the vanishing point is shown according to an embodiment of this disclosure. y The process of estimating the height of the horizon using coordinate values.

[0021] Figure 7A A typical automatic exposure (AE) window implemented according to existing methods is shown.

[0022] Figure 7B The method implemented using the new AE window is shown.

[0023] Figure 7C The method implemented using the new AE window is also shown.

[0024] Figure 8A The typical region of interest in an afterimage (AE) is shown using existing methods.

[0025] Figure 8B The region of interest (ROI) for the AE, which reflects the real-time tolerance compensation results, is shown.

[0026] Figure 9A The image shows a scene of the external environment captured by a camera.

[0027] Figure 9B The region of interest (ROI) for the AE, which reflects the real-time tolerance compensation results, is shown.

[0028] Figure 10A The original image without backlight compensation is shown.

[0029] Figure 10B The results show the identification of the solar light source and the application of backlight compensation to each object.

[0030] Figure 11 This is a flowchart illustrating the process of determining the backlight in a region of interest and performing backlight compensation based on the classification conditions according to an embodiment of the present disclosure.

[0031] Figure 12 This is a block diagram schematically illustrating an exemplary computing device that can be used to implement the methods or apparatus according to this disclosure. Detailed Implementation

[0032] Some exemplary embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. In the following description, although elements are shown in different figures, the same reference numerals preferably denote the same elements. Furthermore, in the following description of some embodiments, detailed descriptions of known functions and configurations incorporated therein will be omitted for clarity and brevity. Additionally, various terms such as first, second, A, B, (a), (b) are used only to distinguish one component from another, but do not imply or suggest the substance, order, or sequence of the components. Throughout the specification, when a component “comprises” or “includes” a component, that component means that other components are also included, not excluded, unless otherwise expressly stated. Terms such as “unit” or “module” refer to one or more units for performing at least one function or operation, which may be implemented by hardware, software, or a combination thereof. The following detailed description, together with the accompanying drawings, is intended to describe exemplary embodiments of the invention and is not intended to represent the only embodiments in which the invention can be practiced. Figure 1 This is a block diagram that briefly illustrates a backlight compensation device according to an embodiment of the present disclosure.

[0033] The backlight compensation device 10 according to embodiments of this disclosure may include all or part of a wide-angle camera 100, an automatic tolerance compensation system 120, an image processing system 130, and a navigation system 140. Meanwhile, Figure 1 The constituent elements shown represent functionally different components, and at least one constituent element can be implemented in an integrated form in a real physical environment.

[0034] The wide-angle camera 100 may include at least one or more of a front camera, a rear camera, and a side camera mounted on the vehicle. The camera may include a single camera, a stereo camera, etc.; those skilled in the art can modify and design the type and installation location of the camera in various ways within the scope of this disclosure.

[0035] The Automatic Tolerance Compensation System 120 is a system that generates a coherent top-down view image by synthesizing and correcting images captured by four cameras using various modes and image synthesis algorithms when installing a panoramic monitoring (AVM) system.

[0036] The image processing system 130 receives images from the wide-angle camera 100, processes the received images, and performs backlight compensation.

[0037] The navigation system 140 receives the corrected image from the image processing system 130 and provides the received corrected image to the passenger.

[0038] Figure 2 This is a block diagram illustrating an automatic tolerance compensation system 120 according to an embodiment of the present disclosure.

[0039] The automatic tolerance compensation system 120 may include all or part of the image receiver 121, parameter storage unit 122, tolerance compensation processor 123, and vanishing point calculator 124. Meanwhile, Figure 2 The constituent elements shown represent functionally different components, and at least one constituent element can be implemented in an integrated form in a real physical environment.

[0040] Tolerance refers to the difference between the maximum and minimum values ​​defined for a specific reference value. In other words, tolerance has the same meaning as allowable error.

[0041] Tolerance compensation is one of the fundamental algorithms in a vehicle's AVM (Around View Monitor) system. An AVM is a system that uses four cameras to capture the entire 360-degree surroundings of the vehicle and displays the environment on a single screen, as if viewing the vehicle from above. AVM helps passengers easily check their surroundings without opening windows on cloudy or rainy days when visibility is poor, and allows them to identify obstacles in blind spots not visible in the side mirrors, thus reducing the risk of accidents.

[0042] A Surround View Monitor (SVM) is a device that allows passengers to see the front, rear, left, and right sides of a vehicle through four cameras mounted on it. The goal of an SVM system is to improve the safety and convenience of vehicle operation by providing the driver with an image of the vehicle's surroundings when parking and driving at low speeds. Key functions of an SVM system include displaying images of the vehicle's surroundings, displaying parking distance warnings, and providing tolerance compensation.

[0043] Image receiver 121 receives images of the front, rear, left and right sides of the vehicle from wide-angle camera 100.

[0044] The parameter storage unit 122 acquires and stores the actual external parameters of the camera relative to the design value in real time, and stores the internal parameters acquired from the manufacturing process.

[0045] The tolerance compensation processor 123 acquires the actual external parameters of the camera relative to the design values ​​in real time, which are affected by external physical factors and tolerances. The vanishing point calculator 124 uses the camera's intrinsic and extrinsic parameters to estimate the y-direction pixel coordinates of the horizon.

[0046] Figure 3 This is a block diagram illustrating an image processing system 130 according to an embodiment of the present disclosure.

[0047] The image processing system 130 may include all or part of an image receiver 131, a region of interest setting unit 132, an image converter 133, a controller 134, and an image transmitter 135. Meanwhile, Figure 3The constituent elements shown represent functionally different components, and at least one constituent element can be implemented in an integrated form in a real physical environment.

[0048] Image receiver 131 receives images of the front and rear of the vehicle from wide-angle camera 100.

[0049] The region of interest setting unit 132 receives the coordinate values ​​of the horizon from the automatic tolerance compensation system 120 and sets the region of interest based on the horizon.

[0050] The region of interest setting unit 132 uses the coordinate values ​​of the horizon estimated by reflecting the tolerance compensation results to accurately divide the region of interest window into road area and sky area.

[0051] Image converter 133 converts the RGB data of each pixel of the input image into YUV data.

[0052] Luminance is a measure of the brightness of light emitted by a light source, and it indicates how bright the light source appears when viewed from a particular direction by an observer.

[0053] The controller 134 performs backlight compensation by dividing the region of interest into multiple blocks, calculating the average brightness value of each of the multiple image blocks, setting a block group that meets the conditions for identifying the solar light source, and controlling the brightness of each block.

[0054] The controller 134 identifies only regions of interest (ROIs) within the sky region, divides these ROIs into multiple blocks, and adjusts the number of blocks within each ROI to reduce the resources required for image processing. The controller 134 can use multiple ROI blocks within the same resource to specify conditions for identifying solar light sources. Further specifying these conditions improves the accuracy of solar light source identification.

[0055] Specifically, controller 134 classifies multiple blocks into four groups to select multiple block groups that meet the solar light source identification conditions. Within the overall brightness distribution of the image, blocks whose average brightness value is equal to or higher than a first value are classified into the first group. Within the overall brightness distribution of the image, blocks whose average brightness value is less than the first value but higher than or equal to a second value are classified into the second group. Within the overall brightness distribution of the image, blocks whose average brightness value is less than the second value but higher than or equal to a third value are classified into the third group. Within the overall brightness distribution of the image, blocks whose average brightness value is less than the third value are classified into the fourth group. For example, the first value is defined as 95%, the second value as 90%, and the third value as 20%. The defined average brightness value is not limited to the specific values ​​mentioned above.

[0056] To meet the criteria for solar light source identification, blocks belonging to the first group should be adjacent to each other for more than a first threshold, blocks belonging to the first group should only be adjacent to blocks belonging to the second group, blocks belonging to the second group should be adjacent to each other for a second threshold, and blocks belonging to the second group should always be adjacent to blocks belonging to the first group. Blocks belonging to the first group that are adjacent to blocks belonging to the third group are excluded from the identification criteria. Blocks belonging to the second group are adjacent to blocks belonging to the first group only on one side. The first group is identified as a solar light source even when the brightness of the first group is higher than that of the third group by a certain threshold. The first threshold is defined as 5, and the second threshold is defined as 15; however, the thresholds are not limited to the specific values ​​mentioned above. Blocks belonging to the first group and blocks belonging to the second group can have vertically and / or horizontally symmetrical shapes.

[0057] The controller 134 classifies the backlight condition into four types: first condition, second condition, third condition, or fourth condition.

[0058] The first condition corresponds to a situation where the number of neighboring blocks belonging to the first group is greater than or equal to a first threshold, and the number of neighboring blocks belonging to the second group is greater than or equal to a second threshold. In another embodiment of this disclosure, the blocks belonging to the first group may have a vertically and / or horizontally symmetrical shape. In yet another embodiment of this disclosure, the blocks belonging to the second group may have a vertically and / or horizontally symmetrical shape. The first threshold is defined as 5, and the second threshold is defined as 15; however, the thresholds are not limited to the specific values ​​described above. The first condition corresponds to very strong backlighting conditions.

[0059] The second condition corresponds to a situation where the number of neighboring blocks belonging to the first group is greater than or equal to a first threshold, the number of neighboring blocks belonging to the second group is less than a second threshold, and the brightness ratio exceeds a first ratio. In another embodiment of this disclosure, the blocks belonging to the first group may have a vertically and / or horizontally symmetrical shape. In yet another embodiment of this disclosure, the blocks belonging to the second group may have a vertically and / or horizontally symmetrical shape. The first ratio is a value obtained by dividing the average brightness of the fourth group by the average brightness of the first group, i.e., 0.5. However, it should be noted that the first ratio is not limited to the specific value described above. The second condition corresponds to a weak backlight condition.

[0060] The third condition corresponds to a situation where the number of neighboring blocks belonging to the first group is greater than or equal to a first threshold, the number of neighboring blocks belonging to the second group is less than a second threshold, and the luminance ratio is less than or equal to a first ratio but exceeds a second ratio. In another embodiment of this disclosure, the blocks belonging to the first group may have a vertically and / or horizontally symmetrical shape. In yet another embodiment of this disclosure, the blocks belonging to the second group may have a vertically and / or horizontally symmetrical shape. The number of blocks and the average luminance are not limited to the specific values ​​described above. The second ratio is a value obtained by dividing the average luminance of the fourth group by the average luminance of the first group, i.e., 0.3. However, it should be noted that the second ratio is not limited to the specific values ​​described above. The third condition corresponds to a medium level of backlighting.

[0061] The fourth condition corresponds to a situation where the number of neighboring blocks belonging to the first group is greater than or equal to a first threshold, the number of neighboring blocks belonging to the second group is less than a second threshold, and the brightness ratio is less than or equal to a second ratio. In another embodiment of this disclosure, the blocks belonging to the first group may have a vertically and / or horizontally symmetrical shape. In yet another embodiment of this disclosure, the blocks belonging to the second group may have a vertically and / or horizontally symmetrical shape. However, it should be noted that the ratio is not limited to the specific values ​​described above. The fourth condition corresponds to a strong backlight condition.

[0062] When the backlight condition is determined to be the first condition, the controller 134 reduces the brightness value of the first group by 10%, reduces the brightness value of the second group by 5%, increases the brightness value of the third group by 15%, and increases the brightness value of the fourth group by 30%.

[0063] When the backlight condition is determined to be the second condition, the controller 134 increases the brightness value of the fourth group by 10%.

[0064] When the backlight condition is determined to be the third condition, the controller 134 increases the brightness value of the third group by 5% and the brightness value of the fourth group by 20%.

[0065] When the backlight condition is determined to be the fourth condition, the controller 134 increases the brightness value of the third group by 10% and the brightness value of the fourth group by 25%. Meanwhile, the values ​​used to adjust the brightness values ​​for each group and condition are not limited to the specific examples described above.

[0066] Figure 4 This is a block diagram illustrating a navigation system 140 according to an embodiment of the present disclosure.

[0067] The navigation system 140 may include all or part of the image receiver 141 and the image output unit 142.

[0068] Image receiver 141 can receive real-time images via wired or wireless communication. Furthermore, image receiver 141 can receive the final backlight-compensated image from image processing system 130.

[0069] The image output unit 142 displays the final backlight-compensated image to the passenger. The passenger can visually inspect the image output by the image output unit 142 through a display or monitor installed in the vehicle. The passenger can quickly respond to driving and parking environments by checking the backlight-compensated image.

[0070] Figure 5A and Figure 5B The process of acquiring front and rear camera parameters according to embodiments of the present disclosure is illustrated.

[0071] in, Figure 5A The process of deriving extrinsic parameters using a camera coordinate system is shown.

[0072] During driving, newly acquired camera extrinsic parameters, obtained in real time through automatic tolerance compensation, are stored in parameter storage unit 122. These extrinsic parameters reflect the actual position of the camera, rather than its design specifications.

[0073] in Figure 5B The process of deriving the intrinsic parameters of the wide-angle camera 100 is shown.

[0074] The intrinsic parameters are obtained using the focal length, principal point, and optical axis relative to the camera coordinate system, and the obtained intrinsic parameters are stored in the parameter storage unit 122.

[0075] Figure 6 V using the vanishing point is shown according to an embodiment of this disclosure. y The process of estimating the height of the horizon using coordinate values.

[0076] Use from Figure 5A and Figure 5B The camera parameters obtained from Equation 1 are used to calculate the Y-coordinates of the vanishing point and the horizon. atan2 is a bivariate function that calculates the absolute angle between two points in the four quadrants.

[0077] [Equation 1]

[0078] θ tilt =atan2(V y -C y , F y )

[0079] In the above equation, F y F represents the distance from the center of the camera lens along the y-axis to the image sensor, where F y Always satisfy F y >0.

[0080] [Equation 2]

[0081] V y(real) =Fy(real) ×tanθ (real tilt) +C y(real)

[0082] If the vanishing point is above the optical axis, then V y -C y >0, which establishes equation 2.

[0083] [Equation 3]

[0084] V y(real) =F y(real) ×tan(θ (real tilt) -Π)+C y(real)

[0085] If the vanishing point is below the optical axis, then V y -C y <0, which establishes Equation 3. Based on V y -C y The process of deriving equations 2 and 3 using symbols is a well-known technique in the technical field to which this invention pertains, and will not be described in detail here.

[0086] [Equation 4]

[0087] V y (real) = C y (real)

[0088] If the vanishing point coincides with the principal point, and V y -C y =0, then the y-coordinate of the horizon becomes C. y Therefore, V can be used. y(real) The horizon height is estimated using coordinates, which are the y-coordinates of the vanishing point in the above equation.

[0089] Figure 7A , Figure 7B and Figure 7C The embodiments of this disclosure illustrate how adjusting the number of blocks in a region of interest (ROI) can reduce resources or increase processing power compared to the same resources. Since the controller 134 only determines ROIs belonging to the sky region, the resources required for image processing can be reduced by decreasing and adjusting the number of blocks. Furthermore, by using more ROI blocks within the same resource constraints, the conditions for identifying the sun can be refined.

[0090] in, Figure 7A A typical automatic exposure (AE) window implemented according to existing methods is shown. The total number of blocks in the sky area is 74, accounting for 50% of the resource.

[0091] in, Figure 7B This demonstrates the method implemented using a new After Effects window. (Compared to...) Figure 7A In comparison, blocks are categorized more precisely. 169 blocks belong to the sky region, representing 50% of the resources. By increasing the number of partitions for blocks belonging to the sky region, more than twice the number of blocks can be used compared to existing methods. Although the same resources as the previous ROI are used, more than twice the number of ROI blocks can be utilized.

[0092] in, Figure 7C The method implemented using the new AE window is also shown. Figure 7A In comparison, the number of blocks has decreased. 74 blocks belong to the sky region, representing 25% of the resources. Although the number of blocks corresponding to the sky region is the same as in the previous ROI, only half of the resources can be secured.

[0093] Controller 134 can reduce the required resources by adjusting the number of blocks in the entire region of interest.

[0094] The region of interest setting unit 132 can use the coordinate values ​​of the horizon estimated by adopting the tolerance compensation result to divide the region of interest window into a road region and a sky region.

[0095] Since the controller 134 only determines the exact region of interest belonging to the sky region, the processing capacity based on the same resources can be increased by further dividing the ROI into more blocks, or the required resources can be reduced by reducing the number of ROI blocks.

[0096] Figure 8A and Figure 8B The results of improving the identification of solar light sources by adjusting the number of blocks in the region of interest, according to embodiments of the present disclosure, are shown.

[0097] in, Figure 8A The typical region of interest in an afterimage (AE) is shown using existing methods.

[0098] The sun's shadowed region is indicated by blocks with diagonal lines. A problem with existing methods is that the shape of the sun is difficult to discern due to the large block size.

[0099] in, Figure 8B The region of interest (ROI) for the AE, which reflects the real-time tolerance compensation results, is shown.

[0100] The sun-shaded areas are indicated by blocks with diagonal lines. When the blocks in the region of interest are further subdivided by reflecting the tolerance compensation results, the blocks corresponding to the solar light source form symmetrical shapes, thus allowing for accurate identification of the sun's shape.

[0101] Figure 9A and Figure 9BThe illustration shows the result of identifying a solar light source using the brightness values ​​of blocks in a region of interest according to an embodiment of this disclosure. The solar light source is indicated by blocks having two distinct diagonals. The first group includes blocks with diagonals pointing downwards to the left, and the second group includes blocks with diagonals pointing downwards to the right.

[0102] in, Figure 9A The image shows a scene of the external environment captured by a camera.

[0103] in, Figure 9B The region of interest (ROI) for the image (AE) reflecting the real-time tolerance compensation results is shown. The controller 134 can set up groups of blocks that meet the conditions for identifying solar light sources. Blocks belonging to the first group and blocks belonging to the second group are adjacent to each other to form a solar light source. Since the brightness of the sun decreases from the center of the light source to the periphery, further subdivision of the ROI blocks based on the real-time tolerance compensation results allows for the differentiation and accurate identification of the central and peripheral regions of the sun using blocks adjacent to high-brightness and non-high-brightness blocks compared to the overall brightness of the image.

[0104] Figure 10A and Figure 10B An example of a backlight-compensated image after backlight identification is shown according to an embodiment of the present disclosure.

[0105] in, Figure 10A The original image without backlight compensation is shown.

[0106] in, Figure 10B The results show the identification of the solar light source and the application of backlight compensation to each object.

[0107] The controller 134 can perform at least one of tone mapping, frame interpolation, and motion deblurring.

[0108] Tone mapping refers to the process of mapping a high dynamic range (HDR) image to a low dynamic range (LDR) image. Tone mapping can be performed using either a global tone mapping method (using only one tone mapping operator for the entire image) or a local tone mapping method (tonally mapping each pixel in the image based on its corresponding pixel value and the pixel values ​​of its surrounding pixels).

[0109] Frame interpolation refers to the process of increasing the number of frames in an existing video or a video rendered in real time.

[0110] Motion deblurring refers to the process of removing noticeable stripes from fast-moving objects in consecutive frames of videos and animations or in still images.

[0111] The controller 134 identifies backlighting in the image and then uses tone mapping technology to compensate for the brightness of the image affected by backlighting. Passengers can easily distinguish objects from the backlight-compensated image.

[0112] Figure 11 This is a flowchart illustrating the process of determining the backlight in a region of interest and performing backlight compensation based on the classification conditions according to an embodiment of the present disclosure.

[0113] Image converter 133 converts RGB data to YUV data for each pixel of the input image (S1102).

[0114] The controller 134 calculates the average brightness of each block within the region of interest that belongs to the sky region (S1104).

[0115] Blocks whose average brightness value is higher than or equal to 95% of the overall brightness distribution are designated as the first group (S1106).

[0116] Blocks whose average brightness value is higher than or equal to 90% and less than 95% of the overall brightness distribution are set as the second group (S1108).

[0117] Blocks whose average brightness value is higher than or equal to 20% and less than 90% of the overall brightness distribution are designated as the third group (S1110).

[0118] Blocks whose average brightness value is less than 20% of the overall brightness distribution are designated as the fourth group (S1112).

[0119] The controller 134 determines whether the blocks belonging to the first group are adjacent to each other and less than a first threshold (S1114). If the blocks are adjacent to each other and less than the first threshold, the data is transmitted to the image converter 133.

[0120] Controller 134 determines whether the blocks belonging to the second group are adjacent to each other and less than the second threshold (S1116).

[0121] When blocks belonging to the first group are adjacent to each other for more than a first threshold, and blocks belonging to the second group are adjacent to each other for more than a second threshold, the corresponding situation is classified as the first situation.

[0122] The controller 134 reduces the brightness value of the first group by 10%, reduces the brightness value of the second group by 5%, increases the brightness value of the third group by 15%, and increases the brightness value of the fourth group by 30% (S1118).

[0123] Controller 134 determines whether the luminance ratio exceeds a first ratio (S1120). The luminance ratio is a value obtained by dividing the average luminance of the fourth group by the average intensity of the first group. If the luminance ratio exceeds the first ratio, the corresponding condition is classified as a second condition.

[0124] When the current situation is classified as the second condition, the controller 134 increases the brightness value of the fourth group by 10% (S1122).

[0125] Controller 134 determines whether the luminance ratio exceeds the second ratio (S1124).

[0126] If the brightness ratio is lower than or equal to the first ratio and exceeds the second ratio, the corresponding situation is classified as the third condition. The controller 134 increases the brightness value of the third group by 5% and the brightness value of the fourth group by 20% (S1126).

[0127] If the brightness ratio is lower than or equal to the second ratio, the corresponding situation is classified as the fourth condition. Controller 134 increases the brightness value of the third group by 10% and the brightness value of the fourth group by 25% (S1128).

[0128] Figure 12 This is a block diagram schematically illustrating an exemplary computing device that can be used to implement the methods or apparatus according to this disclosure.

[0129] The computing device 120 may include some or all of the following: memory 1200, processor 1220, storage device 1240, input / output interface 1260, and communication interface 1280. The computing device 120 may structurally and / or functionally include at least a portion of a wide-angle camera 100, an automatic tolerance compensation system 120, an image processing system 130, or a navigation system 140. The computing device 120 may be a fixed computing device such as a desktop computer, server, and AI accelerator, or a mobile computing device such as a laptop computer and smartphone.

[0130] The memory 1200 may store programs that cause the processor 1220 to perform methods or operations according to various embodiments of the present disclosure. For example, the program may include a plurality of instructions executable by the processor 1220, and Figure 11 The method shown can be executed by processor 1220, which executes multiple instructions.

[0131] The memory 1200 may be a single memory or multiple memories. In this case, the information required to perform the methods or operations according to various embodiments of this disclosure may be stored in a single memory or distributed across multiple memories. When the memory 1200 is configured as multiple memories, the multiple memories may be physically separated.

[0132] The memory 1200 may include at least one of volatile memory and non-volatile memory. The volatile memory includes static random access memory (SRAM) or dynamic random access memory (DRAM), and the non-volatile memory includes flash memory.

[0133] Processor 1220 may include at least one core capable of executing at least one instruction. Processor 1220 may execute instructions stored in memory 1200. Processor 1220 may be a single processor or multiple processors.

[0134] Even if the power supply to the computing device 120 is cut off, the storage device 1240 retains the stored data. For example, the storage device 1240 may include non-volatile memory, or may include storage media such as magnetic tape, optical disc, and magnetic disk.

[0135] The program stored in storage device 1240 can be loaded into memory 1200 before being executed by processor 1220. Storage device 1240 can store files written in a programming language, and can load programs created from files by compilers or the like into memory 1200. Storage device 1240 can store data to be processed by processor 1220 and / or data processed by processor 1220.

[0136] The input / output interface 1260 may include input devices such as a keyboard and mouse, and may include output devices such as a display device and a printer. Users can trigger the processor 1220 to execute programs and / or check the processing results of the processor 1220 through the input / output interface.

[0137] The communication interface 1280 can provide access to external networks. For example, the computing device 120 can communicate with other devices through the communication interface 1280.

[0138] Each element of the device or method according to the invention can be implemented in hardware, software, or a combination of hardware and software. The function of the corresponding element can be implemented in software, and a microprocessor can be implemented to execute the software function corresponding to the corresponding element.

[0139] Various embodiments of the systems and techniques described herein can be implemented using digital electronic circuits, integrated circuits, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. Various embodiments may include implementations having one or more computer programs executable on a programmable system. The programmable system includes at least one programmable processor, which may be a dedicated or general-purpose processor, coupled to receive and transfer data and instructions from a storage system, at least one input device, and at least one output device. The computer program (also referred to as a program, software, software application, or code) includes instructions for the programmable processor and is stored in a computer-readable recording medium.

[0140] Computer-readable recording media can include all types of storage devices on which computer-readable data can be stored. Computer-readable recording media can be non-volatile or non-transitory media, such as read-only memory (ROM), random access memory (RAM), compact optical disc ROM (CD-ROM), magnetic tape, floppy disk, or optical data storage devices. Furthermore, computer-readable recording media can further include transient media such as data transmission media. Moreover, computer-readable recording media can be distributed across computer systems connected via a network, and computer-readable program code can be stored and executed in a distributed manner.

[0141] Although the operations in the flowcharts / timing diagrams of this specification are shown to be performed sequentially, this is merely an exemplary description of the technical concept of one embodiment of this disclosure. In other words, those skilled in the art to which one embodiment of this disclosure pertains will understand that various modifications and changes can be made without departing from the basic characteristics of one embodiment of this disclosure; that is, the order shown in the flowcharts / timing diagrams can be changed, and one or more operations can be performed in parallel. Therefore, the flowcharts / timing diagrams are not limited to a temporal order.

[0142] While exemplary embodiments of this disclosure have been described for illustrative purposes, those skilled in the art will understand that various modifications, additions, and substitutions may be made without departing from the spirit and scope of the claimed invention. Therefore, exemplary embodiments of this disclosure have been described for the sake of brevity and clarity. The scope of the technical concept of the embodiments of this disclosure is not limited by the illustrations. Therefore, those skilled in the art will understand that the scope of the claimed invention is not limited to the embodiments explicitly described above, but is limited by the claims and their equivalents.

Claims

1. A backlight compensation device, comprising: At least one memory; as well as At least one processor, Specifically, by executing instructions, the at least one processor is configured to: Images are received from cameras via the vehicle's in-vehicle system. The position of the horizon in the image is estimated using the intrinsic and extrinsic parameters of the camera, and a region of interest (ROI) is set based on the position of the horizon. The region of interest (ROI) is divided into multiple blocks, and the average brightness value of each of the multiple blocks is measured. The blocks are grouped into multiple block groups based on the average brightness value of each block. The backlight conditions of the ROI are determined based on a predetermined average brightness value for each block in the plurality of block groups, and The brightness values ​​of multiple blocks are adjusted according to the backlighting conditions of the region of interest (ROI).

2. The device according to claim 1, wherein, The multiple block groups are classified into Group 1, Group 2, Group 3 and Group 4.

3. The device according to claim 2, wherein, The average brightness value of the first group is higher than or equal to a predetermined first value. The average brightness value of the second group is less than the predetermined first value and higher than or equal to the predetermined second value. The average brightness value of the third group is less than the predetermined second value and higher than or equal to the predetermined third value, and The average brightness value of the fourth group is less than the predetermined third value.

4. The device according to claim 2, wherein, When the number of neighboring blocks belonging to the first group is greater than or equal to a predetermined first threshold, and the number of neighboring blocks belonging to the second group is greater than or equal to a predetermined second threshold, the corresponding situation is classified as the first condition.

5. The device according to claim 4, wherein, When the number of neighboring blocks belonging to the second group is less than the predetermined second threshold, and the brightness ratio obtained by dividing the average brightness of the fourth group by the average brightness of the first group exceeds the predetermined first ratio, the corresponding situation is classified as the second condition.

6. The device according to claim 5, wherein, When the brightness ratio is less than or equal to the predetermined first ratio and exceeds the predetermined second ratio, the corresponding situation will be classified as the third condition. as well as When the brightness ratio is less than or equal to the predetermined second ratio, the corresponding situation is classified as the fourth condition.

7. The device according to claim 4, wherein, When the current situation is classified as the first situation, adjust the brightness values ​​of the first group to the fourth group.

8. The device according to claim 5, wherein, When the current situation is classified as the second situation, adjust the brightness value of the fourth group.

9. The device according to claim 6, wherein, When the current situation is classified as the third situation, adjust the brightness values ​​of the third group and the fourth group, and When the current situation is classified as the fourth situation, the brightness values ​​of the third group and the fourth group are adjusted.

10. A method for implementing a backlight compensation device, the backlight compensation method comprising the following steps: Images are received from cameras via the vehicle's in-vehicle system; The position of the horizon in the image is estimated using the intrinsic and extrinsic parameters of the camera; The region of interest (ROI) is set based on the position of the horizon. The region of interest (ROI) is divided into multiple blocks, and the average brightness value of each of the multiple blocks is measured; The blocks are grouped into multiple block groups based on the average brightness value of each block; The backlight conditions of the ROI are determined based on a predetermined average brightness value for each of the plurality of block groups; as well as Adjust the brightness values ​​of multiple blocks according to the backlight conditions of the region of interest (ROI).

11. The method according to claim 10, wherein, The multiple block groups are classified into Group 1, Group 2, Group 3 and Group 4.

12. The method according to claim 11, wherein, The average brightness value of the first group is higher than or equal to a predetermined first value. The average brightness value of the second group is less than the predetermined first value and higher than or equal to the predetermined second value. The average brightness value of the third group is less than the predetermined second value and higher than or equal to the predetermined third value, and The average brightness value of the fourth group is less than the predetermined third value.

13. The method according to claim 11, wherein, When the number of neighboring blocks belonging to the first group is greater than or equal to a predetermined first threshold, and the number of neighboring blocks belonging to the second group is greater than or equal to a predetermined second threshold, the corresponding situation is classified as the first condition.

14. The method according to claim 13, wherein, When the number of neighboring blocks belonging to the second group is less than the predetermined second threshold, and the brightness ratio obtained by dividing the average brightness of the fourth group by the average brightness of the first group exceeds the predetermined first ratio, the corresponding situation is classified as the second condition.

15. The method according to claim 14, wherein, When the brightness ratio is less than or equal to the predetermined first ratio and exceeds the predetermined second ratio, the corresponding situation will be classified as the third condition. as well as When the brightness ratio is less than or equal to the predetermined second ratio, the corresponding situation is classified as the fourth condition.

16. The method according to claim 13, wherein, When the current situation is classified as the first situation, adjust the brightness values ​​of the first group to the fourth group.

17. The method of claim 14, wherein, When the current situation is classified as the second situation, adjust the brightness value of the fourth group.

18. The method according to claim 15, wherein, When the current situation is classified as the third situation, adjust the brightness values ​​of the third group and the fourth group, and When the current situation is classified as the fourth situation, the brightness values ​​of the third group and the fourth group are adjusted.

Images