Image enhancement method, device and product for vehicle-mounted camera

Abstract

The present disclosure provides image enhancement methods, systems, devices and program products for vehicle-mounted cameras. The image enhancement method comprises: determining whether an image acquired by a vehicle-mounted camera meets a predetermined definition condition; in response to the image not meeting the predetermined definition condition, determining a region in a field of view (FOV) region of the vehicle-mounted camera that needs to be adjusted for illumination; and performing an illumination adjustment process on the region that needs to be adjusted for illumination.

Description

Technical Field

[0001] This disclosure relates to the fields of vehicles and intelligent driving, and more specifically, to image enhancement methods, apparatus, and products for vehicle-mounted cameras. Background Technology

[0002] In the field of intelligent driving, Autonomous Driving (AD) or Advanced Driving Assistance Systems (ADAS) utilize onboard cameras to assist driving. However, certain weather conditions, times of day, or special road conditions can cause significant errors or even render the camera-based assisted driving system unusable. For example, on a sunny day, if the vehicle is traveling directly towards the sun, the strong sunlight can impair the camera's perception. Similarly, when passing through a tunnel, the changing lighting conditions inside and outside the tunnel can cause the onboard camera to "go blind," preventing AD / ADAS from recognizing objects or vehicles ahead and thus affecting the system's ability to make decisions or issue warnings. Summary of the Invention

[0003] According to a first aspect of this disclosure, an image enhancement method for an in-vehicle camera is provided, comprising: determining whether an image acquired by the in-vehicle camera meets a predetermined sharpness condition; in response to the image not meeting the predetermined sharpness condition, determining an area in the field of view (FOV) region of the in-vehicle camera that requires illumination adjustment; and performing illumination adjustment processing on the area requiring illumination adjustment. According to embodiments of this disclosure, the illumination adjustment processing includes at least one of supplemental lighting via an in-vehicle lighting system and shading via a camera lens hood.

[0004] According to embodiments of this disclosure, determining whether an image acquired by a vehicle-mounted camera meets a predetermined sharpness condition includes: generating a grayscale histogram of the image; and determining that the image does not meet the predetermined sharpness condition in response to the number of pixels in the grayscale histogram whose grayscale values ​​are within a predetermined grayscale value range being higher than a threshold.

[0005] According to embodiments of this disclosure, determining the area requiring illumination adjustment within the FOV region of the vehicle-mounted camera includes: calculating the position of the light source within the FOV region.

[0006] According to embodiments of this disclosure, performing illumination adjustment processing on an area requiring illumination adjustment includes: designating the area above the light source position in the FOV area as the area to be shaded; and moving the camera hood to shade the area to be shaded.

[0007] According to embodiments of this disclosure, the method further includes: retracting the camera sunshade in response to the vehicle traveling to a predetermined distance from the intersection.

[0008] According to embodiments of this disclosure, performing illumination adjustment processing on areas requiring illumination adjustment includes: designating the area below the position of the light source in the FOV area as the area requiring supplemental lighting, and supplementing the area requiring supplemental lighting through an on-board lighting system.

[0009] According to embodiments of this disclosure, supplementing illumination to areas requiring supplemental lighting via an onboard lighting system includes: supplementing illumination to areas within the area requiring supplemental lighting that are below a predetermined threshold distance from the ground.

[0010] According to embodiments of this disclosure, the light source can be the sun, and determining the position of the light source in the FOV area further includes: calculating the incident angle and incident position of the light source relative to the windshield of the vehicle based on at least the vehicle's orientation, geographical location, and time information; calculating the relative azimuth angle between the light source and the vehicle in the horizontal direction of the FOV area based on the incident position of the light source relative to the windshield of the vehicle; calculating the effective elevation angle between the light source and the vehicle in the vertical direction of the FOV area based on the incident angle of the light source relative to the windshield of the vehicle; and obtaining the position of the light source in the FOV area based on the ratio of the relative azimuth angle and the effective elevation angle to the vertical FOV angle of the vehicle-mounted camera.

[0011] According to embodiments of this disclosure, the camera hood is made of a semi-transparent material, and the vehicle lighting system is an adaptive high beam (ADB) system.

[0012] According to embodiments of this disclosure, the method further includes: a) acquiring a new image via a vehicle-mounted camera after performing illumination adjustment processing on the area requiring illumination adjustment; b) determining whether the new image meets a predetermined sharpness condition; c) adjusting the position of the light hood and / or adjusting the amount of supplementary light to perform illumination adjustment processing in response to the new image not meeting the predetermined sharpness condition; d) repeating steps a)-c) until the acquired new image meets the predetermined sharpness condition.

[0013] According to a second aspect of this disclosure, a system for image enhancement is provided, comprising: a vehicle-mounted camera; a sensing component configured to determine whether an image acquired by the vehicle-mounted camera meets a predetermined sharpness condition; a control component configured to, in response to the image not meeting the predetermined sharpness condition, determine an area in the field of view (FOV) region of the vehicle-mounted camera that requires illumination adjustment; and an illumination adjustment component for adjusting the illumination of the area requiring illumination adjustment.

[0014] According to embodiments of this disclosure, the control unit is further configured to calculate the position of the light source in the FOV region.

[0015] According to embodiments of this disclosure, the illumination adjustment component includes a camera hood configured to move to block an area above the light source location within the field of view (FOV).

[0016] According to embodiments of this disclosure, the illumination adjustment component includes an adaptive high beam (ADB) system configured to provide supplemental lighting to areas below the light source position within the field of view (FOV).

[0017] According to a third aspect of this disclosure, an image enhancement apparatus for an in-vehicle camera is provided, comprising: at least one processing unit; and a memory storing computer-readable instructions and communicatively connected to the at least one processing unit, the at least one processing unit being configured to execute the method according to the first aspect when executing the computer-readable instructions stored in the memory.

[0018] According to a fourth aspect of this disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the method according to the first aspect.

[0019] According to a fifth aspect of this disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program / instructions are stored, which, when executed by a processor, implement the method according to the first aspect.

[0020] The methods, apparatus, and procedures provided in this disclosure enable low-cost and high-efficiency enhancement of camera-based AD / ADAS image recognition under adverse weather or road conditions, thereby improving the recognition accuracy and decision-making timeliness of AD / ADAS and enhancing vehicle driving safety. Attached Figure Description

[0021] These and / or other aspects and advantages of this disclosure will become clearer and more readily understood from the following detailed description of embodiments of this disclosure taken in conjunction with the accompanying drawings, wherein:

[0022] Figure 1A and Figure 1B A schematic diagram of the operation of a system for image enhancement according to an embodiment of the present disclosure is shown.

[0023] Figure 2 A flowchart of a method for image enhancement according to an embodiment of the present disclosure is shown.

[0024] Figure 3A and Figure 3B The image and its corresponding grayscale histogram are shown in an image acquired by a vehicle-mounted camera in a tunnel exit scenario, according to an embodiment of the present disclosure.

[0025] Figure 4AA flowchart of a method for calculating the position of a light source according to an embodiment of the present disclosure is shown.

[0026] Figure 4B A schematic diagram of a method for calculating the position of a light source according to an embodiment of the present disclosure is shown.

[0027] Figure 5 A schematic diagram of an apparatus for image enhancement according to another embodiment of the present disclosure is shown.

[0028] It should be understood that these accompanying drawings are used to provide a further understanding of the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain this disclosure and do not constitute a limitation thereof. Furthermore, in the drawings, the same reference numerals generally represent the same components or steps. Detailed Implementation

[0029] To better illustrate the technical solutions of this disclosure, the following detailed description is provided in conjunction with the accompanying drawings and specific embodiments. It should be understood that all other embodiments obtained by those skilled in the art based on the embodiments described herein without creative effort should fall within the protection scope of this disclosure. Furthermore, the embodiments described herein are only a part of the embodiments of this disclosure, not all of them. These embodiments are merely illustrative and exemplary, and therefore should not be construed as limiting the scope of this disclosure.

[0030] For vehicle-mounted cameras operating in low-light conditions (e.g., entering or exiting tunnels or under direct sunlight), one possible solution is to automatically adjust the camera's exposure parameters. However, this approach is limited by sensor hardware performance and algorithm response speed, and may struggle to work effectively and accurately in scenarios with drastically changing lighting, leading to insufficient reliability of the AD / ADAS system. Furthermore, other solutions may rely on upgrades to infrastructure services (e.g., installing shading adjustment systems at tunnel entrances and exits) or employ radar for perception compensation. However, these methods often require additional upgrades to infrastructure or vehicle hardware, resulting in higher costs.

[0031] To address the aforementioned issues, this disclosure provides an image enhancement method, apparatus, and product for vehicle-mounted cameras.

[0032] First, combine Figure 1A and Figure 1B A system 100 for image enhancement according to an embodiment of the present disclosure will be described.

[0033] like Figure 1A As shown, system 100 includes one or more cameras 101, sensing and control components 102, and illumination adjustment components 103, which are communicatively or mechanically coupled.

[0034] In embodiments according to this disclosure, during vehicle operation, camera 101 acquires images within its field of view (FOV) at a predefined sampling frequency and sends them to sensing and control unit 102. This predefined sampling frequency can dynamically change during vehicle operation. In one embodiment according to this disclosure, the sampling frequency can vary depending on road conditions or weather conditions. For example, during normal driving on a highway, the sampling frequency can be configured to 30 FPS; while in scenarios such as entering or leaving a tunnel, the sampling frequency can be adjusted to a higher frequency, such as 60 FPS. Furthermore, in extremely bright summer weather conditions, where camera perception may malfunction due to direct sunlight is more likely than under normal lighting conditions, the sampling frequency can be appropriately increased to dynamically allocate computing resources.

[0035] In embodiments according to this disclosure, the sensing and control unit 102 can process the received image to determine whether the image meets the sharpness requirements for image recognition. In embodiments according to this disclosure, if it is determined that the image does not meet the sharpness requirements for image recognition (e.g., the image is overexposed and distorted due to strong light), the sensing and control unit 102 can determine the area in the corresponding field of view (FOV) that needs illumination adjustment based on the specific characteristics of the image.

[0036] After determining the area requiring illumination adjustment, the illumination adjustment component 103 performs illumination adjustment processing on that area. In embodiments according to this disclosure, the illumination adjustment component 103, such as... Figure 1A The area enclosed by the dashed line may include an onboard lighting system (e.g., an adaptive (ADB) high beam system 1031) and / or a camera visor 1032, which are respectively configured as follows: Figure 1B The illustration shows supplemental lighting and / or shading for areas requiring lighting adjustment.

[0037] For example, when the image clarity acquired by the vehicle-mounted camera is poor due to a strong light source, the sensing and control unit 102 can determine the area within the FOV (Field of View) of the camera 101 that requires illumination adjustment based on the position of the light source. In one embodiment, the sensing and control unit 102 can supplement the area below the light source with the ADB (Advanced Driver Assistance) high beam 1031 to reduce the contrast of the acquired image and thus increase the image's recognizability. In another embodiment, the sensing and control unit 102 can move the position of the camera hood 1032 to block the light source and the area above it, thereby reducing the amount of incident light and optimizing the imaging effect. Preferably, in embodiments according to this disclosure, the camera hood can be made of a semi-transparent material, which can effectively block strong light without affecting the recognition of other objects.

[0038] It is understood that the camera 101 can continuously acquire images after the illumination adjustment process, and monitor and analyze them through the sensing and control unit 102.

[0039] Optionally, in embodiments according to this disclosure, system 100 may further include a light sensor 104 configured to detect strong light. In one embodiment, while the vehicle is in motion, if the light sensor 104 continuously detects strong light (e.g., continuously detects a threshold higher than 10,000 cd / m² for more than 1 ms), the system may detect strong light. 2 If the light intensity is high, the sensing and control unit 102 and the illumination adjustment unit 103 initiate the illumination adjustment process as described above. It should be understood that for short-term "strong light" caused by factors such as intersection flashing lights, since it is temporary, illumination adjustment-related processing is not required. Therefore, strong light detection via the light sensor 104 avoids unnecessary processing and resource waste, thereby improving the robustness and detection accuracy of the system 100. Optionally or additionally, in embodiments according to this disclosure, in the case of vehicles entering or exiting tunnels, strong light can be detected based on the contrast between light inside and outside the tunnel. For example, when the ratio of the light intensity inside the tunnel to the light intensity outside the tunnel exceeds 1:100, strong light can be considered detected.

[0040] Next, we will combine Figure 2 The method 200 for image enhancement according to embodiments of the present disclosure will be described in detail.

[0041] like Figure 2 As shown, in step S201, it is determined whether the image acquired by the vehicle-mounted camera meets a predetermined sharpness condition. During vehicle operation, if there is a strong light source or drastic changes in lighting conditions, uneven light intensity or excessive light contrast may occur. This can lead to a sharp reduction in identifiable features in the acquired image, causing the image to fail to meet the sharpness condition required for recognition. In this step, it can be determined whether the image meets the sharpness condition by any appropriate method. In one example implementation, a grayscale histogram of the image can be generated; and in response to the number of pixels in the grayscale histogram whose grayscale values ​​fall within a predetermined grayscale value range being higher than a threshold, it is determined that the image does not meet the predetermined sharpness condition. The following will combine... Figure 3A and Figure 3B A more detailed description of the example implementation is provided.

[0042] When a vehicle is driving directly into the sun or exiting a tunnel, where the light ahead is too strong, the target area in the camera's field of view (FOV) may be overexposed. This will result in an overall larger pixel value in the image, close to the maximum value of 255 (pure white). On the other hand, when entering a tunnel, where the light ahead is too weak, the target area in the camera's FOV may be underexposed. This will result in an overall smaller pixel value in the image, close to the minimum value of 0 (pure black).

[0043] For example, Figure 3A The image shown is an image acquired by a vehicle-mounted camera in a tunnel exit scenario, according to an embodiment of the present disclosure. Figure 3B It shows the relationship with Figure 3A The image shown corresponds to a grayscale histogram. Figure 3B In the diagram, the horizontal axis represents the grayscale value of a pixel, ranging from 0 (pure black) to 255 (pure white), while the vertical axis represents the frequency of pixel occurrence, that is, the frequency of a pixel corresponding to a given grayscale value. For example... Figure 3A As shown, due to the significant difference in lighting conditions inside and outside the tunnel in the tunnel exit scenario, the central area of ​​the image captured by the camera appears almost pure white, therefore the system 100 cannot use this image for image recognition. Accordingly, as... Figure 3B As shown in the figure, the grayscale histogram corresponding to the image is shaped like a "」, with almost all pixels concentrated around pixel value 255 (i.e., pure white area). Therefore, in one embodiment of this disclosure, if the number of pixels with grayscale values ​​in the "240-255" range in the grayscale histogram is higher than, for example, 80%, the image corresponding to the grayscale histogram can be considered not to meet the predetermined sharpness condition.

[0044] Correspondingly, in another embodiment of this disclosure, if the shape of the corresponding grayscale histogram is "L" (not shown in the figure), that is, almost all pixels are concentrated in the "0-15" range (i.e., pure black area), it can also be considered that the image corresponding to the grayscale histogram does not meet the predetermined sharpness condition.

[0045] It should be understood that the predetermined clarity conditions listed above for exiting and entering tunnels are merely examples. Those skilled in the art can adjust the grayscale range and pixel number threshold according to different vehicle environments and road conditions, and this application does not impose any limitations on this.

[0046] Back Figure 2In step S202, in response to the image not meeting the predetermined sharpness condition, areas in the field of view (FOV) of the vehicle camera that require illumination adjustment are determined. As mentioned earlier, when uneven lighting intensity or excessive light contrast occurs during vehicle operation, the image acquired by the vehicle camera will be unclear. To address this, the sharpness of subsequent images acquired by the camera can be improved by adjusting the lighting conditions in the field of view of the vehicle camera. In this step, areas in the camera's field of view that require illumination adjustment can be determined in any suitable manner; this application is not limited in this regard. For example, in a non-limiting embodiment, pixel clusters with gray values ​​exceeding a first threshold (e.g., 240) in the grayscale histogram corresponding to the image can be identified, and the area corresponding to this pixel cluster in the camera's field of view can be further identified as areas requiring reduced brightness. In yet another non-limiting embodiment, pixel clusters with gray values ​​below a second threshold (e.g., 15) in the grayscale histogram corresponding to the image can be identified, and the area corresponding to this pixel cluster in the camera's field of view can be further identified as areas requiring increased brightness.

[0047] In one embodiment of this application, for images containing light sources, the position of the light source within the FOV region can be determined and used as a reference position for corresponding illumination adjustment processing. Determining the area requiring illumination adjustment in this way is more targeted, thereby enabling more efficient and faster adjustment processing. Various appropriate methods can be used to determine the light source position. For example, an image recognition-based algorithm can be used to identify the light source position from an image corresponding to the FOV region of the vehicle-mounted camera. As another example, when the light source is the sun (e.g., the vehicle's direction of travel is directly facing the sun), the light source position within the camera's FOV region can be calculated based on geographical and temporal information related to the vehicle's position using an astronomical model. The following will combine... Figure 4A and Figure 4B A detailed description is provided of a method 400 for obtaining the position of a light source within the field of view (FOV) of a camera when the light source is the sun.

[0048] like Figure 4AAs shown, in step S401, based on at least the vehicle's orientation, geographical location, and time information, the incident angle and position of the light source relative to the vehicle's windshield are calculated. In one embodiment of this disclosure, real-time vehicle data, such as steering wheel sensor data, GPS data, and precise time data, can be obtained via the vehicle's bus interface to determine the vehicle's orientation (e.g., heading angle), geographical location (e.g., latitude and longitude), and time information (e.g., time zone information). Then, combined with windshield-related parameters, such as the windshield's tilt angle and glass material, the incident angle of the light source relative to the vehicle's windshield can be obtained using an astronomical model algorithm. And the incident position P, such as Figure 4B As shown in the figure. In one example embodiment, the Solar Position Algorithm (SPA) of NREL, which is well known in the art, can be used as the astronomical model algorithm to calculate the solar altitude angle and azimuth angle in the global coordinate system, and then converted into the incident angle in the vehicle coordinate system through conventional coordinate transformation. The incident position P will not be elaborated here.

[0049] Then, in step S402, based on the incident position of the light source relative to the vehicle's windshield, the relative azimuth angle between the light source and the vehicle in the horizontal direction within the FOV area can be calculated. The incident position of the light source on the vehicle's windshield indicates the azimuth angle of the light source; therefore, subtracting it from the vehicle's heading angle yields the relative azimuth angle between the light source and the vehicle in the horizontal direction within the FOV area, representing the relative position of the light source in the horizontal direction relative to the vehicle's direction of travel.

[0050] In step S403, based on at least the incident angle of the light source relative to the vehicle's windshield, the effective elevation angle between the light source and the vehicle in the vertical direction of the FOV area can be calculated. Figure 4B As shown, the incident angle of the light source relative to the windshield is... Combined with the incident angle Relevant parameters of the windshield (e.g., the angle between the windshield and the lower edge of the camera's FOV vertical field of view). (etc.), which can obtain the effective elevation angle of the light source in the vertical direction of the camera's FOV area. .

[0051] Furthermore, during vehicle operation, road conditions may involve uphill or downhill sections, causing changes in the camera's field of view (FOV) angle and resulting in a deviation in the relative height of the light source within the FOV area. Therefore, optionally, as a non-limiting example, the vehicle's pitch angle can be considered when calculating the effective height angle of the light source. (Not shown). For example, in this case, after performing the above calculations, it is possible to further subtract... To obtain the final effective elevation angle .

[0052] Finally, in step S404, the position of the light source in the FOV area can be obtained based on the ratio of the relative azimuth angle and the effective elevation angle to the vertical FOV angle of the vehicle-mounted camera. Figure 4B As shown, the field of view (FOV) of the camera in the vertical direction is... Through effective elevation angle With vertical field of view

[0053] The ratio of these values ​​allows us to determine the relative position of the light source to the vehicle in the vertical direction within the field of view (FOV). Optionally or additionally, the above information and the calculation results can be displayed on a visualization screen on the windshield as supplementary information for augmented reality.

[0054] The method described above provides a concise and accurate way to determine the location of light sources within the field of view (FOV). Compared to calculating light source locations through image recognition, this method yields accurate results with less computational resources and faster processing speed.

[0055] Next, in step S203, illumination adjustment processing is performed on the area requiring illumination adjustment. This illumination adjustment processing may include supplementing the area requiring increased brightness as determined in step 202 with supplemental lighting through the vehicle's lighting system, and / or shielding the area requiring reduced brightness using a camera light shield.

[0056] Optionally, in the case where the above image has a light source and the light source position is used as the reference position for calculating the area requiring illumination adjustment, the illumination adjustment process for the area requiring illumination adjustment may include: designating the area below the light source position in the FOV area as the area requiring supplemental lighting, and supplementing the area requiring supplemental lighting through the vehicle lighting system. Wherein, to prevent glare to the reflector of the vehicle in front caused by the ADB high beams during supplemental lighting, as a non-limiting example, supplemental lighting may only be applied to the area below a predetermined threshold above the ground in the area requiring supplemental lighting. This predetermined threshold can be appropriately set according to the height of the vehicle's rearview mirror and the distance to the vehicle in front, for example, it may be an appropriate value between 1 and 1.3 meters.

[0057] Optionally or additionally, the illumination adjustment process for the area requiring illumination adjustment may further include: designating the area above the light source position within the FOV as the area to be shaded; and moving the camera sunshade to shade the area to be shaded. Considering that using a sunshade at locations such as intersections may cause misidentification or missed identification of traffic lights, as a non-limiting example, in response to a vehicle approaching an intersection, for example, at a predetermined distance from the intersection, the sunshade can be retracted. This predetermined distance can be dynamically adjusted according to vehicle speed.

[0058] In embodiments according to this disclosure, as a non-limiting example, the areas requiring supplemental lighting and / or areas requiring shading can be determined based on the lower edge position, upper edge position, center line position, or other suitable position of the light source. Figure 4B As shown, preferably, the lower edge of the light source can be used as the reference position, the area above it is the area to be shaded, and the camera lens hood is moved to the corresponding shading position; the area below it is the area to be illuminated, and ADB high beam is used for illumination. In this way, compared to choosing other light source positions as the reference position, better lighting adjustment effect can be obtained, and subsequent possible repetitive processing can be minimized.

[0059] Back Figure 2 In the image enhancement method for an in-vehicle camera according to embodiments of the present disclosure, illumination adjustment processing can be performed more than once, but rather the corresponding processing is repeated depending on whether the newly acquired image after illumination adjustment is clear. Specifically, in one embodiment of the present disclosure, after illumination adjustment processing is performed on the area requiring illumination adjustment, a new image can be acquired through the in-vehicle camera; then, it is determined whether the new image meets a predetermined clarity condition; in response to the new image not meeting the predetermined clarity condition, the position of the light hood and / or the amount of supplementary light are adjusted again; the above processing is repeated until the acquired new image meets the predetermined clarity condition. In this way, better image clarity can be obtained.

[0060] Figure 5 A schematic diagram of an image enhancement apparatus 500 according to an embodiment of the present disclosure is shown. Figure 5 As shown, the device 500 may include a processor 501 and a memory 502.

[0061] The processor 501 may be any processing-capable device capable of implementing the functions of the various embodiments of this disclosure. For example, it may be a general-purpose processor, digital signal processor (DSP), ASIC, field programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein.

[0062] The memory 502 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, or other removable / non-removable, volatile / non-volatile computer system memory, such as hard disk drives, floppy disks, CD-ROMs, DVD-ROMs, or other optical storage media.

[0063] In this embodiment, memory 502 stores computer program instructions, and processor 501 can execute the instructions stored in memory 502. When the computer program instructions are executed by the processor, the processor performs the image enhancement method of the embodiments of this disclosure. The method for image enhancement is related to the one described above. Figures 2 to 4A - Figure 4B The descriptions are essentially the same, so for the sake of brevity, they will not be repeated.

[0064] According to one aspect of this disclosure, an image enhancement device for an in-vehicle camera is also provided, comprising: an image detection module for determining whether an image acquired by the in-vehicle camera meets a predetermined sharpness condition; an adjustment determination module for determining, in response to the image not meeting the predetermined sharpness condition, an area in the field of view (FOV) region of the in-vehicle camera that requires illumination adjustment; and an illumination adjustment module for performing illumination adjustment processing on the area. For specific processing details of each module, please refer to the above description regarding... Figures 2 to 4A - Figure 4B The methods described herein will not be repeated here.

[0065] The image enhancement method / apparatus according to this disclosure can also be implemented by providing a computer program product containing program code implementing the method or apparatus, or by any storage medium storing such a computer program product.

[0066] As a non-limiting example, the image enhancement method and system according to this disclosure, by being tailored to road conditions and scenarios, can enhance AD / ADAS using cameras in a low-cost and efficient manner, avoiding the "blindness" of onboard cameras caused by strong light affecting the recognition accuracy and decision-making of AD / ADAS, thereby improving vehicle driving safety.

[0067] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the embodiments of this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0068] The block diagrams of devices, apparatuses, devices, and systems involved in the embodiments of this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context explicitly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0069] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0070] It should also be noted that in the apparatus and methods of this disclosure, the components or steps can be disassembled and / or recombined. Such disassembly and / or recombination should be considered as equivalent solutions to this disclosure.

[0071] It will be understood by those skilled in the art that all or any part of the methods and apparatus of this disclosure can be implemented in hardware, firmware, software, or a combination thereof in any computing device (including processors, storage media, etc.) or network of computing devices. The hardware may be a general-purpose processor, digital signal processor (DSP), ASIC, field-programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but alternatively, it may be any commercially available processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration. The software may reside in any form of computer-readable tangible storage medium. By way of example and not limitation, such computer-readable tangible storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other tangible medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible by a computer. If used herein, the discs include compact discs (CDs), laser discs, optical discs, digital universal discs (DVDs), floppy discs, and Blu-ray discs.

[0072] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0073] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0074] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. An image enhancement method for an in-vehicle camera, comprising: Determine whether the images acquired by the vehicle-mounted camera meet the predetermined resolution requirements; In response to the image not meeting the predetermined sharpness condition, the area in the field of view (FOV) of the vehicle-mounted camera that requires illumination adjustment is determined; as well as The area requiring illumination adjustment is then subjected to illumination adjustment processing.

2. The image enhancement method according to claim 1, wherein, The illumination adjustment process includes at least one of supplemental lighting through the vehicle lighting system and shading through a camera hood.

3. The image enhancement method according to claim 1, wherein, The process of determining whether the image acquired by the vehicle-mounted camera meets the predetermined clarity requirements includes: Generate a grayscale histogram of the image; If the number of pixels in the grayscale histogram whose grayscale values ​​fall within a predetermined grayscale value range is higher than a threshold, it is determined that the image does not meet the predetermined sharpness condition.

4. The image enhancement method according to claim 1, wherein, The areas within the FOV (Field of View) of the vehicle-mounted camera that require illumination adjustment include: Determine the position of the light source in the FOV area.

5. The image enhancement method according to claim 4, wherein, The illumination adjustment process for the area requiring illumination adjustment includes: The area above the light source position within the FOV region is designated as the area requiring shading; and A movable camera hood is used to cover the area that needs to be covered.

6. The image enhancement method according to claim 5, further comprising: In response to the vehicle traveling to a predetermined distance from the intersection, the camera's sunshade is retracted.

7. The image enhancement method according to any one of claims 4-6, wherein, The illumination adjustment process for the area requiring illumination adjustment includes: The area below the light source position within the FOV region is considered the area requiring supplemental lighting. The area requiring supplemental lighting is illuminated using the vehicle's lighting system.

8. The image enhancement method according to claim 7, wherein, The method of providing supplemental lighting to the area requiring additional illumination via the vehicle-mounted lighting system includes: Supplemental lighting is applied to areas below a predetermined threshold distance from the ground within the area requiring supplemental lighting.

9. The image enhancement method according to claim 4, wherein, The light source is the sun, and determining the location of the light source within the FOV region further includes: Based on at least the vehicle's orientation, geographical location, and time information, calculate the incident angle and incident position of the light source relative to the vehicle's windshield; Based on the incident position of the light source relative to the windshield of the vehicle, calculate the relative azimuth angle between the light source and the vehicle in the horizontal direction of the FOV area; Based on the incident angle of the light source relative to the vehicle's windshield, calculate the effective elevation angle between the light source and the vehicle in the vertical direction of the FOV area; and The position of the light source in the FOV area is obtained based on the ratio of the relative azimuth angle and the effective elevation angle to the vertical FOV angle of the vehicle-mounted camera.

10. The image enhancement method according to claim 2, wherein, The camera hood is made of a semi-transparent material, and the vehicle lighting system is an adaptive ADB high beam system.

11. The image enhancement method according to claim 2, further comprising: a) After performing illumination adjustment processing on the area requiring illumination adjustment, a new image is acquired through the vehicle-mounted camera; b) Determine whether the new image meets the predetermined sharpness condition; c) In response to the new image not meeting the predetermined sharpness condition, adjust the position of the light shield and / or adjust the amount of supplementary light to perform illumination adjustment processing; d) Repeat steps a)-c) until the acquired new image meets the predetermined sharpness condition.

12. A system for image enhancement, comprising: Vehicle-mounted cameras; The sensing component is configured to determine whether the image acquired by the vehicle-mounted camera meets a predetermined clarity condition; The control unit is configured to determine, in response to the image not meeting the predetermined sharpness condition, an area in the field of view (FOV) of the vehicle-mounted camera that requires illumination adjustment. A light adjustment component is used to adjust the light intensity of the area requiring light adjustment.

13. The system according to claim 12, wherein, The control unit is further configured to calculate the position of the light source in the FOV region.

14. The system of claim 12, wherein the illumination adjustment component includes a camera hood configured to move to block the area above the light source position in the FOV region.

15. The system of claim 12, wherein the illumination adjustment component includes an adaptive ADB high beam system configured to provide supplemental lighting to the area below the light source position in the FOV region.

16. An image enhancement device for a vehicle-mounted camera, comprising: At least one processing unit; and The memory stores computer-readable instructions and is communicatively connected to the at least one processing unit. The at least one processing unit is configured to execute the method according to any one of claims 1-11 when running the computer-readable instructions stored in the memory.

17. A computer program product comprising computer program instructions that, when executed by a processor, implement the method according to any one of claims 1-11.

18. A non-transitory computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the method according to any one of claims 1-11.

Images