System and method for automatically detecting glare of camera

Abstract

The invention relates to the technical field of image analysis, and particularly discloses a camera glare automatic detection system and method, and the method comprises the following steps: S1, obtaining a preview image frame in real time through a camera module of a mobile device after the mobile device enters a shooting interface; s2, performing image analysis on the preview image frame, automatically detecting and identifying whether a glare area exists in the image frame, if not, skipping to S1, and if yes, skipping to S3; s3, calculating the relative orientation of at least one light source generating glare in the real world based on the features of the change of the glare area identified by the plurality of frames; and S4, according to the calculated relative orientation of the light source, generating a guide direction for guiding a user to move the mobile device to avoid glare. According to the technical scheme, automatic glare detection and light source orientation calculation can be achieved, effective evasion guidance is provided, and the imaging quality is improved.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, and in particular to an automatic glare detection system and method for cameras. Background Technology

[0002] With the rapid development of mobile communication and optical imaging technologies, smartphones and other mobile devices have become the main tools for people to record their daily lives and create images. Users' demands for image quality from mobile devices are also increasing, and major device manufacturers are making improving the hardware performance of camera modules and optimizing image processing algorithms one of the core competitive advantages in product iteration.

[0003] In everyday shooting, especially in complex lighting environments such as night scenes, backlighting, or the presence of strong point light sources (e.g., streetlights, the sun, stage lighting), users often encounter lens flare. Lens flare is caused by light from strong light sources entering the camera lens and undergoing multiple reflections or scatterings within the lens assembly, ultimately forming unexpected light spots, ghosting, or light curtains on the image sensor. These imperfections not only severely damage the clarity and contrast of the image, reducing the overall visual appeal, but may even obscure the key subject, significantly reducing the usability of photos or videos.

[0004] Professional photographers typically avoid glare by adjusting shooting angles and postures, or by using hands, light shields, or other objects to block stray light. However, this method requires a high level of professional knowledge and operational skills. For ordinary users without photography expertise, it is often difficult to identify the source of glare and find the correct avoidance method in a short time. They often need to try and adjust repeatedly, which is tedious and affects shooting opportunities.

[0005] While image processing software or algorithms can be used to identify and repair glare areas in photos after shooting, this post-processing method has significant limitations: First, it is a "post-hoc" measure and cannot solve the problem during the shooting and framing stage, so users cannot obtain an original, glare-free "clean" photo; second, when the glare area is large or overlaps with the subject, the algorithmic repair effect is often unsatisfactory, easily producing smearing marks or unnatural artifacts, and may even fail to restore details lost in highlight clipping areas caused by strong light.

[0006] In summary, existing technologies lack a solution that can automatically and in real-time detect glare during mobile device shooting and proactively and intuitively guide ordinary users without professional knowledge to quickly avoid glare, thereby improving image quality at the source of shooting. Summary of the Invention

[0007] One of the objectives of this invention is to provide an automatic glare detection method for cameras, which can realize automatic glare detection and light source orientation calculation, provide effective avoidance guidance, and improve image quality.

[0008] To solve the above-mentioned technical problems, this application provides the following technical solution: The automatic glare detection method for cameras includes the following: S1. After the mobile device enters the shooting interface, the preview image frame is acquired in real time through the camera module of the mobile device. S2. Perform image analysis on the preview image frame, automatically detect and identify whether there is a glare area in the image frame. If there is no glare, jump to S1; if there is, jump to S3. S3. Based on the characteristics of glare area changes identified in several frames, calculate the relative orientation of at least one light source that generates glare in the real world. S4. Based on the calculated relative position of the light source, generate a guiding direction to instruct the user to move the mobile device to avoid glare. S5. Generate guidance information based on the guidance direction, display the guidance information through the shooting interface of the mobile device, and jump to S2.

[0009] Furthermore, in step S2, if a glare area exists, motion data of the mobile device is also acquired, and it is determined whether there is a glare avoidance action based on the motion data. If there is, proceed to S3; if not, end.

[0010] Furthermore, in step S2, a pre-trained neural network model is used to analyze the preview image frame, identify glare in the image, and output a bounding box containing the glare area.

[0011] Furthermore, in step S3, by continuously collecting and analyzing data pairs of the mobile device rotation vector and glare movement vector, a mapping relationship is established to determine the relative orientation of at least one light source that generates glare in the real world.

[0012] Furthermore, in step S5, directional guidance information is generated according to the guidance direction, and the directional guidance information is overlaid on the preview image frame of the shooting interface in real time in the form of an augmented reality overlay. In this context, directional guidance signals are indicated by arrows or indicator lines pointing away from the light source.

[0013] A second objective of this invention is to provide an automatic glare detection system for cameras, comprising: The image acquisition module is used to acquire preview image frames in real time through the camera module of the mobile device after the mobile device enters the shooting interface; The glare analysis module is used to analyze the preview image frame using a pre-trained neural network model, automatically detect and identify whether there is glare in the image frame, and if so, identify the glare area in the image and output the bounding box containing the glare area. The light source analysis module is used to calculate the relative position of at least one light source that generates glare in the real world based on the characteristics of glare area changes identified in several frames; it is also used to generate a guiding direction to guide the user to move the mobile device to avoid glare based on the calculated relative position of the light source. The information generation module is used to generate guidance information based on the guidance direction and display the guidance information through the shooting interface of the mobile device.

[0014] Furthermore, it also includes a motion analysis module, which is used to acquire motion data of the mobile device, determine whether there is a glare avoidance action based on the motion data, and generate a light source analysis instruction if it exists; otherwise, it ends the current analysis.

[0015] Furthermore, the information generation module also generates directional guidance information based on the guidance direction, and overlays the directional guidance information onto the preview image frame of the shooting interface in real time in the form of an augmented reality overlay; wherein, the directional guidance information points away from the light source by arrows or indicator lines.

[0016] This solution combines the movement characteristics of glare areas in consecutive image frames with the motion data of the mobile device itself to establish a mapping relationship, thereby accurately calculating the relative location of the real-world light source that generates glare. Furthermore, by analyzing the mobile device's motion data, it can determine whether the user is performing a "glare avoidance maneuver." Only when a conscious effort to avoid glare is detected will subsequent light source analysis and guidance processes be triggered, reducing system resource consumption. Moreover, glare is detected in real-time on the shooting interface, and augmented reality visual guidance clearly and intuitively instructs the user on how to fine-tune the mobile device's angle to avoid glare. Users do not need professional photography knowledge; they can quickly complete the avoidance operation simply by following the guidance, significantly improving the user experience.

[0017] In summary, this solution not only enables automatic glare detection and light source location calculation, but also allows for intelligent judgment based on user actions and provides intuitive avoidance guidance, effectively solving the technical problems of difficulty in timely glare avoidance and image quality degradation during existing shooting processes. Attached Figure Description

[0018] Figure 1 This is a flowchart of an embodiment of the automatic glare detection method for cameras. Detailed Implementation

[0019] The following detailed description illustrates the specific implementation method: Example 1 like Figure 1 As shown, the automatic glare detection method for cameras in this embodiment includes the following steps: S1. After the mobile device enters the shooting interface, the preview image frame is acquired in real time through the camera module of the mobile device. S2. Analyze the preview image frame using a pre-trained neural network model to automatically detect and identify whether there is glare (e.g., circular or polygonal bright spots) in the image frame. If there is no glare, jump to S1. If there is glare, identify the glare area in the image and output the bounding box containing the glare area. Also acquire the motion data of the mobile device and determine whether there is a glare avoidance action based on the motion data. If there is, jump to S3; if not, end.

[0020] The pre-trained neural network model employs a convolutional neural network (CNN)-based object detection model; in this embodiment, the YOLO series model is specifically used. After inference, the model outputs a target categorized as "glare," along with the bounding box coordinates of the glare area. If these coordinates are higher than a preset confidence threshold (e.g., 0.85), glare is determined to exist in the current preview image frame, and the bounding box's position information is recorded. Conversely, if the model does not output a target categorized as "glare," or if the output target's confidence score is lower than 0.85, glare is determined to be absent in the current frame. For example, when a user uses a mobile device to capture a nighttime street scene, the model automatically detects highly reflective areas in the image frame, such as bright patches created by streetlights illuminating the lens, and outputs the corresponding bounding boxes.

[0021] Motion data comes from the inertial measurement unit (IMU) built into the mobile device, including linear acceleration and angular velocity data collected by accelerometers and gyroscopes. In this embodiment, glare avoidance action refers to a smooth and conscious small angle adjustment made by the user to eliminate or reduce glare, rather than large-amplitude or high-frequency shaking in scenarios such as walking or riding in a vehicle. For example, according to the angular velocity data, the mobile device generates a continuous angular velocity of 2-10 degrees / second around the X-axis (vertical rotation) or Y-axis (horizontal rotation), with a total rotation angle between 1-5 degrees. At the same time, the linear acceleration data shows that the standard deviation of the linear acceleration changes in the three axes is less than 0.2 m / s², which matches the combined characteristics of smooth and conscious small angle adjustments. This is identified as the user adjusting the phone angle to try to avoid glare, thus determining that a glare avoidance action exists.

[0022] S3. Based on the characteristics of glare area changes identified in several frames, calculate the relative orientation of at least one light source that generates glare in the real world. Specifically, the coordinates of the center point of the glare area in each frame are recorded. It also synchronously records the changes in pitch and yaw angles of the mobile device in the motion data from the previous frame to the current frame. Since glare is the reflection or scattering of light sources within the lens, when a device is rotated, the direction in which the glare area moves on the screen usually has a specific mapping relationship with the actual position of the light source. For example, when the phone is tilted upwards, if the light source is above the field of view, the glare point will move downwards.

[0023] For example, the user slightly rotated the phone to the right. =+2 degrees (positive value represents to the right). By analyzing the image frame sequence, it was observed that the center point of the glare area shifted to the left in the screen coordinate system. =-30 pixels. Then, the user tilted the phone upwards again. =+1.5 degrees (positive value represents upward), the center point of the observed glare area shifted downward accordingly. Pixels. By continuously collecting and analyzing data pairs of device rotation vector and glare movement vector, a mapping relationship is established: the relative orientation vector of the light source. The average displacement velocity vector of glare on the screen They are approximately inversely related, that is After accumulating and averaging multiple frames of data to eliminate noise, it can be concluded that the light source causing the glare is located "upper right" in the current camera's pointing direction.

[0024] S4. Based on the calculated relative position of the light source, generate a guiding direction to instruct the user to move the mobile device to avoid glare. The goal of this step is to generate an optimal avoidance direction to guide the user to make the most effective adjustments. In this embodiment, the principle for determining the guiding direction is to guide the user to move the light source away from the "danger zone" that is most likely to produce glare with the smallest possible movement. This danger zone is usually the center and surrounding area of ​​the screen, in order to reduce interference with the user's composition.

[0025] For example, the relative orientation of the light source is determined to be "upper right" of the camera. In the screen's two-dimensional coordinate system, the orientation is quantized as a vector pointing from the center of the screen to the upper right, for example... To avoid glare, the user needs to be guided to move the camera in the opposite direction, specifically towards the "lower left." Therefore, the generated guidance direction vector... that is The reverse vector, i.e. .

[0026] S5. Generate guidance information based on the guidance direction, display the guidance information through the shooting interface of the mobile device, and then jump to S2. Specifically, generate directional guidance information based on the guidance direction, and overlay the directional guidance information on the preview image frame of the shooting interface in real time in the form of an augmented reality overlay; wherein, the directional guidance information points away from the light source through arrows or indicator lines.

[0027] Based on the above method, this embodiment also provides an automatic camera glare detection system, including an image acquisition module, a glare analysis module, a motion analysis module, a light source analysis module, and an information generation module.

[0028] The image acquisition module is used to acquire preview image frames in real time through the camera module of the mobile device after the mobile device enters the shooting interface; The glare analysis module is used to analyze the preview image frame using a pre-trained neural network model, automatically detect and identify whether there is glare (e.g., circular or polygonal bright spots) in the image frame. If it exists, it identifies the glare area in the image and outputs the bounding box containing the glare area. The motion analysis module acquires motion data from the mobile device and determines whether glare avoidance actions exist based on the motion data. If so, it generates a light source analysis command; otherwise, it terminates the current analysis. The motion data comes from the mobile device's built-in inertial measurement unit (IMU), including linear acceleration and angular velocity data collected by accelerometers and gyroscopes.

[0029] The light source analysis module is used to calculate the relative position of at least one light source that generates glare in the real world based on the characteristics of glare area changes identified in several frames; it is also used to generate a guiding direction to guide the user to move the mobile device to avoid glare based on the calculated relative position of the light source.

[0030] The information generation module generates guidance information based on the guidance direction and displays this information through the mobile device's shooting interface. Specifically, it generates directional guidance information based on the guidance direction and overlays this information onto the preview image frame of the shooting interface in real time using an augmented reality overlay; the directional guidance information is indicated by arrows or indicator lines pointing away from the light source.

[0031] This solution combines the motion characteristics of glare areas in consecutive image frames with the motion data of the mobile device itself to establish a mapping relationship, thereby accurately calculating the relative location of the real-world light source that generates the glare. Based on this location, an avoidance direction can be generated, preventing users from aimlessly "shaking their phones" and achieving precise and efficient guidance while minimizing interference with the user's original compositional intentions.

[0032] Furthermore, by analyzing mobile device motion data, it can determine whether the user is performing "glare avoidance actions." Subsequent light source analysis and guidance processes are only triggered when the user is detected consciously avoiding glare. This intelligently distinguishes between unconscious user movements (such as walking) and conscious avoidance intentions, avoiding unnecessary interruptions when the user is not concerned about glare or is performing other operations (such as large camera movements), reducing system resource consumption, and making human-computer interaction more natural.

[0033] Moreover, the camera detects glare in real time on the shooting interface and uses augmented reality visual guidance to clearly and intuitively instruct users on how to fine-tune the angle of their mobile device to avoid glare. Users do not need to have professional photography knowledge; they can simply follow the guidance to quickly complete the avoidance operation, which significantly improves the user experience.

[0034] This solution intervenes during the shooting preview stage, guiding users to eliminate or reduce glare interference before pressing the shutter. Compared to solutions that rely on post-processing, this is a proactive strategy that addresses the problem at its source. It ensures the quality of the original image data captured by the user, avoiding the risks of image quality loss, detail loss, or failed restoration that may occur during post-processing, helping users "shoot well the first time" and improving the success rate of their photos.

[0035] In summary, this solution not only enables automatic glare detection and light source location calculation, but also allows for intelligent judgment based on user actions and provides intuitive avoidance guidance, effectively solving the technical problems of difficulty in timely glare avoidance and image quality degradation during existing shooting processes.

[0036] The above are merely embodiments of the present invention. The invention is not limited to the fields covered by these embodiments. Commonly known structures and characteristics in the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are able to access all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. An automatic glare detection method for cameras, characterized in that, Includes the following: S1. After the mobile device enters the shooting interface, the preview image frame is acquired in real time through the camera module of the mobile device. S2. Perform image analysis on the preview image frame, automatically detect and identify whether there is a glare area in the image frame. If there is no glare, jump to S1; if there is, jump to S3. S3. Based on the characteristics of glare area changes identified in several frames, calculate the relative orientation of at least one light source that generates glare in the real world. S4. Based on the calculated relative position of the light source, generate a guiding direction to instruct the user to move the mobile device to avoid glare. S5. Generate guidance information based on the guidance direction, display the guidance information through the shooting interface of the mobile device, and jump to S2.

2. The automatic glare detection method for cameras according to claim 1, characterized in that: In step S2, if there is a glare area, the motion data of the mobile device is also acquired, and it is determined whether there is a glare avoidance action based on the motion data. If there is, the process jumps to S3. If it does not exist, end.

3. The automatic glare detection method for cameras according to claim 2, characterized in that: In step S2, a pre-trained neural network model is used to analyze the preview image frame, identify glare in the image, and output a bounding box containing the glare area.

4. The automatic glare detection method for cameras according to claim 3, characterized in that: In step S3, by continuously collecting and analyzing data pairs of the mobile device rotation vector and glare movement vector, a mapping relationship is established to determine the relative location of at least one light source that generates glare in the real world.

5. The automatic glare detection method for cameras according to claim 4, characterized in that: In step S5, directional guidance information is generated according to the guidance direction, and the directional guidance information is overlaid on the preview image frame of the shooting interface in real time in the form of an augmented reality overlay. In this context, directional guidance signals are indicated by arrows or indicator lines pointing away from the light source.

6. An automatic glare detection system for cameras, characterized in that, include: The image acquisition module is used to acquire preview image frames in real time through the camera module of the mobile device after the mobile device enters the shooting interface; The glare analysis module is used to analyze the preview image frame using a pre-trained neural network model, automatically detect and identify whether there is glare in the image frame, and if so, identify the glare area in the image and output the bounding box containing the glare area. The light source analysis module is used to calculate the relative location of at least one light source that generates glare in the real world based on the characteristics of glare area changes identified in several frames. It is also used to generate guidance directions to help users move their mobile devices to avoid glare, based on the calculated relative position of the light source. The information generation module is used to generate guidance information based on the guidance direction and display the guidance information through the shooting interface of the mobile device.

7. The automatic camera glare detection system according to claim 6, characterized in that: It also includes a motion analysis module, which is used to acquire motion data of the mobile device, determine whether there is a glare avoidance action based on the motion data, and generate a light source analysis instruction if it exists; otherwise, the current analysis ends.

8. The automatic glare detection system for cameras according to claim 7, characterized in that: The information generation module also generates directional guidance information based on the guidance direction, and overlays the directional guidance information onto the preview image frame of the shooting interface in real time in the form of an augmented reality overlay; wherein, the directional guidance information points away from the light source by arrows or indicator lines.

Images