Camera module smudginess identification system and method
By collecting device motion data and analyzing image features, and combining this with the user's shooting intent, the system dynamically corrects the dirt confidence level, thus solving the problem of image quality degradation caused by dirt during camera module use and improving user experience and image quality.
Patent Information
- Application Number
- CN202511530182.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-24
- Publication Date
- 2026-02-17
AI Technical Summary
Camera modules are prone to getting dirty during use, which can lead to a decrease in image quality and affect the user experience.
By collecting motion data from the device and combining it with the startup behavior of the camera app, the system determines the user's shooting intention and identifies dirt in non-capture scenarios. It then uses a pre-trained image recognition model to analyze image features, dynamically adjusts the dirt confidence level, and generates a reminder message.
It achieves high-precision dirt detection of camera modules, improves the shooting experience and image quality, reduces false alarm rate, and ensures smooth shooting.
Smart Images
Figure CN121544928A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image processing, in particular to a camera module dirt identification system and method. BACKGROUND
[0002] With the wide popularity of smart phones, users pay more and more attention to the shooting effect when choosing a phone, and the imaging quality of the camera module has become an important factor affecting user experience. In order to ensure that the out-of-factory equipment has good imaging ability, the camera module will usually go through strict detection and calibration in the production link to ensure that the lens optical performance and image quality meet the design standards.
[0003] However, in actual use, the camera module is prone to fingerprints, dust, water stains and other dirt due to frequent contact with the environment, being put into a pocket or being held in hand. These dirt can cause problems such as blurred picture, decreased contrast, glare interference, etc., which seriously affect the shooting effect, making it difficult to play the advantages of high standard quality in the early design and out-of-factory detection, and reducing the user experience.
[0004] Therefore, there is an urgent need for a camera module dirt identification system and method that can identify the dirt state of the lens in real time during the shooting process and improve the overall shooting experience. SUMMARY
[0005] One of the purposes of the present application is to provide a camera module dirt identification system that can identify the dirt state of the lens in real time during the shooting process.
[0006] In order to solve the above technical problems, the present application provides the following technical solutions: The camera module dirt identification system comprises: A data acquisition module for acquiring motion state data of a device carrying a camera; An intention analysis module for determining whether the user's shooting intention is a snapshot intention or a non-snapshot intention according to the motion state data when the user performs a shooting operation; A dirt detection module for acquiring images collected by the camera in real time when the shooting intention is a non-snapshot intention, or waiting for the shooting to be completed and then acquiring images collected by the camera in real time when the shooting intention is a non-snapshot intention; and for analyzing the images to determine whether there is dirt in the images and outputting the analysis results; A user feedback module for generating and displaying a reminder information when the analysis results show that there is dirt.
[0007] Further, the intention analysis module is further configured to determine the shooting intention of the user as a snapshot intention or a non-snapshot intention according to the motion state data before entering the photographing application, and if the shooting intention is a non-snapshot intention, continue to determine the shooting intention of the user as a snapshot intention or a non-snapshot intention according to the motion state data after entering the photographing application and the time of taking a picture.
[0008] Further, the intention analysis module is configured to analyze the motion state data within a preset time before starting the photographing application, and if the motion state data exceeds a preset high dynamic threshold, determine that the shooting intention is a snapshot intention, and if the motion state data does not exceed the preset high dynamic threshold, preliminarily determine that the shooting intention is a non-snapshot intention. For a scene preliminarily determined as a non-snapshot intention, the intention analysis module is further configured to continuously monitor the motion state data after entering the photographing application, and if the motion state data continuously falls below a preset stable state threshold within a preset time, finally determine that the shooting intention is a non-snapshot intention, and if the motion state data exceeds the stable state threshold, correct the intention to a snapshot intention.
[0009] Further, the dirt detection module is further configured to locate a suspected dirt area when analyzing the image, obtain weather information, and identify a scene in the image, and comprehensively analyze the confidence of the suspected dirt feature, and if the confidence is higher than a confidence threshold, determine that there is dirt in the image.
[0010] Further, the dirt detection module is further configured to use a pre-trained image recognition model to locate a suspected dirt area in the image preview frame and output a confidence. Further, the dirt detection module is further configured to identify a scene through the image recognition model, including a simple texture scene and a complex texture scene, obtain weather information, analyze global features of the image, identify a low perspective scene and an overexposure interference scene, correct the dirt confidence according to the scene identification result, and obtain a final dirt confidence, and if the final dirt confidence is greater than a preset confidence threshold, determine that there is dirt in the image.
[0011] Further, the dirt detection module is further configured to obtain weather information, analyze global features of the image, and identify a strong light condition, and in the strong light condition, analyze the pixel saturation of the suspected dirt area, and if the proportion of high light overflow pixels in the area exceeds a preset proportion, determine that it is an overexposure interference scene.
[0012] The second object of the present application is to provide a camera module dirt identification method, comprising the following steps: S1, obtaining motion state data of a device carrying a camera, including angular velocity and linear acceleration; S2, determining the shooting intention of the user as a snapshot intention or a non-snapshot intention according to the motion state data when the user performs a photographing operation; S3. When the shooting intention is not to capture the image, acquire the image captured by the camera in real time; or when the shooting intention is not to capture the image, wait for the shooting to be completed and then acquire the image captured by the camera in real time. It is also used to analyze the image, determine whether the image is dirty, and output the analysis results. S4. When the analysis results indicate the presence of dirt, generate and display a reminder message.
[0013] Furthermore, step S2 specifically includes: S201. When the camera application is launched, the motion state data within a preset time before launch is analyzed. If the peak angular velocity or the peak linear acceleration exceeds the preset high dynamic threshold, it is determined to be an intention to capture the image, and the process proceeds to step S3. If it does not exceed the threshold, it is initially determined to be a non-capture intention, and the process proceeds to step S202. S202. For scenarios initially determined to be non-capture intent, continuously monitor motion state data after entering the photo application. If the peak angular velocity or peak linear acceleration remains below the preset stable state threshold within a preset time, it is ultimately determined to be non-capture intent; if the peak angular velocity or peak linear acceleration exceeds the stable state threshold, the intent is corrected to capture intent.
[0014] Furthermore, step S3 specifically includes: S301. Using a pre-trained image recognition model, locate suspected dirty areas in the image preview frame and output the confidence level; S302. The image recognition model identifies scenes, including simple texture scenes and complex texture scenes; it also acquires meteorological information and analyzes global image features to identify low-perspective scenes and overexposure interference scenes. S303. Based on the scene recognition results, adjust the dirt confidence score to obtain the final dirt confidence score; if the final dirt confidence score is greater than the preset confidence score threshold, it is determined that there is dirt in the image.
[0015] This solution collects device motion data and combines it with the camera app's startup behavior to determine whether the user's current shot is intended for capturing a snapshot or not. In snapshot scenarios, the system prioritizes the responsiveness of the shooting process, delaying or skipping dirt detection to reduce the interference of image processing on shooting speed. In non-capture scenarios, dirt identification is performed during the user's framing and waiting time, improving the real-time performance of detection. This solution also employs a pre-trained image recognition model. By analyzing suspected dirty areas in the image and integrating scene recognition and environmental information, the dirt confidence level is dynamically adjusted, effectively reducing the false alarm rate and improving the accuracy of detection results. When dirt is detected on the lens, the system displays a prompt reminding the user to clean the lens promptly, improving the success rate of shooting.
[0016] In summary, this solution, by combining intelligent analysis of user shooting behavior and image features, can achieve high-precision detection of dirt on the camera module while ensuring smooth shooting for users, thus significantly improving the user's shooting experience and image quality. Attached Figure Description
[0017] Figure 1 This is a logic block diagram of an embodiment of a camera module dirt recognition system. Detailed Implementation
[0018] like Figure 1 As shown, the camera module dirt recognition system of this embodiment includes a data acquisition module, an intent analysis module, a dirt detection module, and a user feedback module; The data acquisition module is used to acquire motion state data of the device carrying the camera; in this embodiment, the smart device is a mobile phone, and motion state data including angular velocity and linear acceleration are acquired from the gyroscope and accelerometer of the mobile phone, respectively.
[0019] The intent analysis module is used to determine whether the user's shooting intent is to capture a photo or not, based on motion state data when the user performs a photo-taking operation. In this embodiment, the intent analysis module is used to determine whether the user's shooting intent is a capture intent or a non-capture intent based on the motion state data before entering the camera application. If it is a non-capture intent, the module further determines whether the user's shooting intent is a capture intent or a non-capture intent based on the motion state data after entering the camera application and the framing time.
[0020] Specifically, when the camera application is launched, the intent analysis module analyzes the motion state data within a preset time (e.g., 3 seconds) before launch. If the peak angular velocity or the peak linear acceleration exceeds the preset high dynamic range threshold, it is determined to be an intent to capture the image; if it does not exceed the threshold, it is initially determined to be a non-capture intent.
[0021] For scenarios initially determined to be non-capture intent, the intent analysis module continuously monitors motion data after entering the camera application. If the peak angular velocity or linear acceleration remains below a preset stable state threshold within a preset time period, it indicates the user is composing a stable shot, and the final determination is that it is not a capture intent. If the peak angular velocity or linear acceleration exceeds the stable state threshold, it indicates tracking is in progress, and the intent is corrected to a capture intent. In this embodiment, the peak angular velocity in the high dynamic threshold is 2.0 rad / s, and the peak linear acceleration is 3.0 m / s². 2 The peak angular velocity in the steady-state threshold is 0.5 rad / s, and the peak linear acceleration is 1.0 m / s². 2 In other embodiments, calibration can also be performed based on actual measurement results.
[0022] For example, if a user sees a bird in flight, quickly takes out their phone, raises it, and starts the camera, the shooting intent analysis module analyzes the angular velocity and linear acceleration to determine it as a capture intent. As another example, if a user intends to photograph a building, raises their phone at a normal speed, and starts the camera, the intent analysis module determines it as a non-capture intent. However, if the user spends several seconds composing and adjusting the phone angle, during which time the angular velocity and linear acceleration remain relatively stable, it's determined as a non-capture intent. Conversely, if the user spends several seconds composing and adjusting the phone angle, during which time the angular velocity and linear acceleration change significantly, indicating the user might press the shutter at any moment, it's determined as a capture intent. In this case, the processor prioritizes the image processing flow, and the dirt detection process is temporarily suspended or postponed until after the photo is taken.
[0023] The dirt detection module is used to acquire images captured in real time by the camera when the shooting intention is not to capture images, or to acquire images captured in real time by the camera after the shooting is completed when the shooting intention is not to capture images; it is also used to analyze the images, determine whether there is dirt in the images, and output the analysis results. In this embodiment, when the dirt detection module analyzes the image, it locates the suspected dirty area, obtains meteorological information, and identifies the scene in the image. It comprehensively analyzes the confidence level of the suspected dirty features. If the confidence level is higher than the confidence level threshold, it determines that there is dirt in the image.
[0024] Specifically, a pre-trained image recognition model is used to locate suspected dirty areas in image preview frames that exhibit significant local contrast reduction, blurred edge gradients, and fixed positions across multiple consecutive frames, and outputs a confidence score. In this embodiment, images with suspected dirty areas exhibiting significant local contrast reduction, blurred edge gradients, and fixed positions across multiple consecutive frames are selected to create a training set for training the image recognition model. A significant decrease in contrast can be determined based on actual conditions; for example, the standard deviation of the suspected area should decrease by more than 40% compared to the standard deviation of the surrounding normal area.
[0025] The system also identifies scenes using image recognition models, including simple texture scenes such as skies, solid-color walls, or documents, as well as complex texture scenes such as grass and bushes. It obtains meteorological information from a weather API and analyzes global image features to identify low-perspective scenes such as fog or rain, and strong lighting conditions such as sunny days. In strong lighting conditions, it analyzes the pixel saturation of suspected dirty areas. If the proportion of highlight overflow pixels in this area exceeds a preset ratio (30% in this embodiment), it is determined to be an overexposed interference scene. Based on the scene recognition results, the dirt confidence score is adjusted to obtain the final dirt confidence score. If the final dirt confidence score is greater than a preset confidence score threshold, it is determined that dirt exists in the image.
[0026] In this embodiment, different correction coefficients are set according to the scene and environment. For simple texture scenes, the correction coefficient is 1.2 to enhance confidence; for complex texture scenes, the correction coefficient is 0.7 to reduce confidence; for low-perspective scenes, the correction coefficient is 0.6 to significantly reduce confidence; and for overexposed interference scenes, the correction coefficient is 0.1 to effectively eliminate the interference. Finally, the initial confidence score output by the model is used as a baseline value, and weighted calculations are performed based on different correction coefficients set according to the scene and environment. The final dirt confidence score = baseline value × scene correction coefficient × environment correction coefficient.
[0027] For example, in foggy outdoor scenes, the blurriness is more due to meteorological factors than to dirt. Also, in scenes with strong sunlight, the blurriness may be due to overexposure. By taking into account both scene and image features, the misjudgment rate in foggy, rainy, and backlit conditions can be effectively reduced.
[0028] The user feedback module generates and displays a reminder message when the analysis results indicate the presence of dirt. For example, a text prompt may appear in the camera viewfinder: "The lens seems dirty; wipe it to get a clearer shot." In other embodiments, a prompt box may be drawn in the detected dirt area to visually tell the user the location of the dirt.
[0029] Based on the above system, this embodiment also provides a method for identifying dirt in a camera module, including the following steps: S1. Acquire motion state data of the device carrying the camera, including angular velocity and linear acceleration; S2. Based on motion state data, when the user performs a photo-taking operation, determine whether the user's shooting intention is to capture the moment or not. Specifically, it includes: S201. When the camera application is launched, the motion state data within a preset time (e.g., 3 seconds) before launch is analyzed. If the peak angular velocity or the peak linear acceleration exceeds the preset high dynamic range threshold, it is determined to be an intention to capture the image, and the process proceeds to step S3. If it does not exceed the threshold, it is initially determined to be a non-capture intention, and the process proceeds to step S202.
[0030] S202. For scenarios initially determined to be non-capture intent, continuously monitor motion state data after entering the photo application. If the peak angular velocity or peak linear acceleration remains below the preset stable state threshold within a preset time, it is ultimately determined to be non-capture intent; if the peak angular velocity or peak linear acceleration exceeds the stable state threshold, the intent is corrected to capture intent.
[0031] S3. When the shooting intention is not to capture the image, acquire the image captured by the camera in real time; or when the shooting intention is not to capture the image, wait for the shooting to be completed and then acquire the image captured by the camera in real time. It is also used to analyze the image, determine whether the image is dirty, and output the analysis results. Specifically, it includes: S301. Using a pre-trained image recognition model, locate suspected dirty areas in the image preview frame that have significantly decreased local contrast, blurred edge gradients, and fixed positions in multiple consecutive frames, and output the confidence score. S302. The image recognition model identifies scenes, including simple texture scenes such as sky, solid color walls or documents, and complex texture scenes such as grass and bushes. It also obtains meteorological information from the weather API and analyzes the global features of the image to identify low-perspective scenes such as fog or rain, and strong light conditions such as sunny days. In strong light conditions, it also analyzes the pixel saturation of suspected dirty areas. If the proportion of highlight overflow pixels in the area exceeds a preset ratio (30% in this embodiment), it is determined to be an overexposure interference scene. S303. Based on the scene recognition results, adjust the dirt confidence score to obtain the final dirt confidence score; if the final dirt confidence score is greater than the preset confidence score threshold, it is determined that there is dirt in the image.
[0032] S4. When the analysis results indicate the presence of dirt, generate and display a reminder message.
[0033] This embodiment's solution determines whether a user's current shot is intended for capturing a snapshot or not by collecting the device's angular velocity and linear acceleration, combined with the startup behavior of the camera application. In snapshot scenarios, the system prioritizes the response speed of the shooting process, delaying or skipping dirt detection to avoid consuming excessive computing resources in high real-time scenarios such as snapshots, thus reducing the interference of image processing on shooting speed. In non-snapshot scenarios, dirt identification is performed during the user's composition waiting time. By calling high-performance image processing functions at appropriate times, the real-time performance of detection is improved, achieving a balance between performance and user experience.
[0034] This solution also employs a pre-trained image recognition model. By analyzing suspected dirty areas in the image and combining scene recognition with environmental information (such as heavy fog, rain, strong light, etc.), the dirt confidence level is dynamically corrected, effectively reducing the false alarm rate and improving the accuracy of the detection results.
[0035] When dirt is detected on the lens, the system will display a prompt, reminding the user to clean the lens in time to improve the success rate of shooting.
[0036] In summary, this solution, by combining intelligent analysis of user shooting behavior and image features, can achieve high-precision detection of dirt on the camera module while ensuring smooth shooting for users, thus significantly improving the user's shooting experience and image quality.
[0037] The above are merely embodiments of the present invention. The invention is not limited to the fields covered by these embodiments. Commonly known structures and characteristics in the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are able to access all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.
Claims
1. A camera module dirt recognition system, characterized in that, include: The data acquisition module is used to acquire motion status data of the device carrying the camera; The intent analysis module is used to determine whether the user's shooting intent is to capture a photo or not, based on motion state data when the user performs a photo-taking operation. The dirt detection module is used to acquire images captured by the camera in real time when the shooting intention is not to capture the image, or to acquire images captured by the camera in real time after the shooting is completed when the shooting intention is not to capture the image; it is also used to analyze the images, determine whether there is dirt in the images, and output the analysis results. The user feedback module is used to generate and display a reminder message when the analysis results indicate the presence of dirt.
2. The camera module dirt recognition system according to claim 1, characterized in that: The intent analysis module is also used to determine whether the user's shooting intent is a capture intent or a non-capture intent based on the motion state data before entering the camera application. If it is a non-capture intent, it will further determine whether the user's shooting intent is a capture intent or a non-capture intent based on the motion state data after entering the camera application and the framing time.
3. The camera module dirt recognition system according to claim 2, characterized in that: The intent analysis module is used to analyze motion state data within a preset time before the camera application is launched. If the motion state data exceeds the preset high dynamic threshold, it is determined to be a capture intent. If the limit is not exceeded, it is initially determined that the intention was not to take a snapshot. For scenarios initially determined to be non-capture intent, the intent analysis module is also used to continuously monitor motion state data after entering the camera application. If the motion state data is continuously lower than the preset stable state threshold within a preset time, it is finally determined to be non-capture intent; if the motion state data exceeds the stable state threshold, the intent is corrected to capture intent.
4. The camera module dirt recognition system according to claim 3, characterized in that: The dirt detection module is also used to locate suspected dirty areas when analyzing images, acquire meteorological information, identify scenes in images, and comprehensively analyze the confidence level of suspected dirt features. If the confidence level is higher than the confidence level threshold, it is determined that there is dirt in the image.
5. The camera module dirt recognition system according to claim 4, characterized in that: The dirt detection module is also used to locate suspected dirt areas in the image preview frame using a pre-trained image recognition model and output the confidence level. It is also used to identify scenes through image recognition models, including simple texture scenes and complex texture scenes; it also acquires meteorological information and analyzes global image features to identify low-perspective scenes and overexposed interference scenes; based on the scene recognition results, it corrects the dirt confidence score to obtain the final dirt confidence score. If the final confidence level of dirt is greater than the preset confidence threshold, it is determined that there is dirt in the image.
6. The camera module dirt recognition system according to claim 5, characterized in that: The dirt detection module is also used to acquire meteorological information and analyze global image features to identify strong light conditions. In strong light conditions, it also analyzes the pixel saturation of suspected dirty areas. If the proportion of highlight overflow pixels in the area exceeds a preset ratio, it is determined to be an overexposure interference scene.
7. A method for identifying dirt and grime in a camera module, characterized in that, Includes the following steps: S1. Acquire motion state data of the device carrying the camera, including angular velocity and linear acceleration; S2. Based on motion state data, when the user performs a photo-taking operation, determine whether the user's shooting intention is to capture the moment or not. S3. When the shooting intention is not to capture the image, acquire the image captured by the camera in real time; or when the shooting intention is not to capture the image, wait for the shooting to be completed and then acquire the image captured by the camera in real time. It is also used to analyze the image, determine whether the image is dirty, and output the analysis results. S4. When the analysis results indicate the presence of dirt, generate and display a reminder message.
8. The method for identifying dirt in a camera module according to claim 7, characterized in that: Step S2 specifically includes: S201. When the camera application is launched, the motion state data within a preset time before launch is analyzed. If the peak angular velocity or the peak linear acceleration exceeds the preset high dynamic threshold, it is determined to be an intention to capture the image, and the process proceeds to step S3. If it does not exceed the threshold, it is initially determined to be a non-capture intention, and the process proceeds to step S202. S202. For scenarios initially determined to be non-capture intent, continuously monitor motion state data after entering the photo application. If the peak angular velocity or peak linear acceleration remains below the preset stable state threshold within a preset time, it is ultimately determined to be non-capture intent; if the peak angular velocity or peak linear acceleration exceeds the stable state threshold, the intent is corrected to capture intent.
9. The method for identifying dirt in a camera module according to claim 8, characterized in that: Step S3 specifically includes: S301. Using a pre-trained image recognition model, locate suspected dirty areas in the image preview frame and output the confidence level; S302. The image recognition model identifies scenes, including simple texture scenes and complex texture scenes; it also acquires meteorological information and analyzes global image features to identify low-perspective scenes and overexposure interference scenes. S303. Based on the scene recognition results, adjust the dirt confidence score to obtain the final dirt confidence score; if the final dirt confidence score is greater than the preset confidence score threshold, it is determined that there is dirt in the image.