Image sensor system surface defect detection method and detection device

Abstract

The invention discloses an image sensor system surface defect detection method and a detection device, which are used for surface defect detection of an image sensor or a filter integrated in front of the image sensor, and comprise a light source which provides incident light and directly irradiates the image sensor or the filter integrated in front of the image sensor; when light passes through the image sensor or the defect on the filter in front of the image sensor, the incident light is diffracted or interfered; the image sensor receives an optical signal containing diffraction or interference and converts the optical signal into an electric signal; and the image signal processor is used for converting the electric signals containing diffraction or interference into images, and diffraction or interference light spots on the images are used for judging defects. The method mainly utilizes the diffraction / interference principle of light, can realize low-cost, high-precision and rapid detection of the surface defects of the image sensor system, and is suitable for camera detection and industrial detection scenes.

Description

Technical Field

[0001] The present invention relates to the field of optical detection technology, and in particular to a surface defect detection method and detection device of an image sensor system based on the diffraction or interference phenomenon of light. Background Art

[0002] Image sensor systems generally include lenses, filters, and image sensors (CMOS or CCD). Once surface defects occur, such as contamination defects (dust, particles, fibers, oil stains, or fingerprints) on the filter surface or coating defects (coating demolding or coating damage), or contamination defects (dust, particles, fibers, oil stains, or fingerprints) on the image sensor surface, the image quality will be affected. Therefore, defect detection is necessary.

[0003] There are currently two main detection application scenarios. One is the need to detect whether the image sensor system in the camera is defective during camera maintenance or second-hand camera transactions after the camera leaves the factory; the other is industrial inspection of image sensors or components of image sensors with integrated filters before leaving the factory.

[0004] For camera maintenance after leaving the factory or in second-hand camera transactions, the detection conditions for surface defect detection in existing image sensor systems are complex, relying on personal subjectivity and the constraints of the specific objective environment at the time, with a high misjudgment rate and non-standardization.

[0005] In the industrial inspection of image sensors or image sensors integrated with filters, the traditional method generally adopts the industrial AOI inspection method, which requires additional dedicated high-precision equipment, is costly, has slow inspection speed and low efficiency.

[0006] Therefore, the existing image sensor system surface defect detection methods and devices mentioned above urgently need further improvement in terms of structure, method, and use. The industry is currently striving to develop a new method and device that can detect surface defects in image sensor systems at low cost, with high accuracy, and quickly. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a surface defect detection method and detection device of an image sensor system that can replace the traditional method, so as to achieve low-cost, high-precision and rapid detection.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0009] A method for detecting surface defects of an image sensor system is provided, which is used for detecting surface defects of an image sensor or a filter integrated in front of the image sensor, and comprises:

[0010] Utilizing a light source to directly illuminate the image sensor or directly illuminate a filter integrated in front of the image sensor;

[0011] When light passes through defects on the image sensor or the filter in front of the image sensor, it causes diffraction or interference of the incident light;

[0012] The image sensor receives the light signal containing diffraction or interference and converts it into an electrical signal;

[0013] The image signal processor converts the electrical signal containing diffraction or interference into an image, and the diffraction or interference spots on the image are used to judge defects.

[0014] As a further improvement of the present invention, the image sensor is a CMOS or CCD image sensor;

[0015] And / or, the surface defect of the filter is a pollution defect or a coating defect; the surface defect of the image sensor is a pollution defect;

[0016] And / or, the light source is an LED light source or a laser light source, and the light source is a single beam light source or more than one beam light source.

[0017] Furthermore, when used for surface defect detection of an image sensor system of a camera, the camera includes an image sensor with a filter integrated in the front and an image signal processor connected to the image sensor; the light source directly illuminates the filter through the camera lens interface of the removed camera lens, and then the camera is used for detection;

[0018] Furthermore, when used in industry to detect image sensors alone or filters integrated in front of image sensors, when the image sensor is powered on for testing, a light source is used to directly illuminate the image sensor or the filter integrated in front of the image sensor for defect detection.

[0019] Furthermore, the image output by the image signal processor is further detected by a pre-built surface defect detection model based on deep learning.

[0020] The present invention also provides an image sensor system surface defect detection device for detecting surface defects of an image sensor or a filter integrated in front of the image sensor, comprising:

[0021] A light source is used to provide incident light to directly illuminate the image sensor or directly illuminate a filter integrated in front of the image sensor;

[0022] Image sensors or image sensors with integrated filters in front of them. When light passes through defects in the image sensor or the filter in front of the image sensor, it causes diffraction or interference of the incident light. The image sensor is used to receive the light signal containing diffraction or interference and convert it into an electrical signal.

[0023] The image signal processor converts the electrical signal containing diffraction or interference into an image, and the diffraction or interference spots on the image are used to judge defects.

[0024] Furthermore, when used for surface defect detection of an image sensor system of a camera, the camera includes the filter integrated in front of the image sensor and an image signal processor connected to the image sensor;

[0025] The defect detection device further includes a connecting member, the light source is mounted on a front end of the connecting member, and a rear end of the connecting member is connected to a camera; the light source directly illuminates the filter through a camera lens interface of a disassembled camera lens;

[0026] The camera to be inspected may or may not be part of a surface defect inspection device of an image sensor system.

[0027] Furthermore, the connecting member is a light-shielding tube, the light source is installed at the front end of the light-shielding tube, and the rear end of the light-shielding tube is provided with a connecting end adapted to the lens interface of the camera, and the connecting end is connected to the lens interface of the camera.

[0028] Furthermore, when used for industrial independent detection of image sensors or filters integrated in front of image sensors, the light source is arranged opposite the image sensor or the filter integrated in front of the image sensor, the image sensor to be detected or the image sensor with the filter integrated in front is mounted on the stage, and the image sensor is connected to the image signal processor via a data transmission line.

[0029] Furthermore, the image sensor is a CMOS or CCD image sensor;

[0030] And / or, the surface defect of the filter is a pollution defect or a coating defect; the surface defect of the image sensor is a pollution defect;

[0031] And / or, the light source is an LED light source or a laser light source, and the light source is a single beam light source or more than one beam light source.

[0032] Furthermore, it also includes a surface defect detection model unit based on deep learning, which is used to further detect the image output by the image signal processor.

[0033] By adopting the above technical solution, the present invention has at least the following effects:

[0034] 1. The present invention provides a method and device for detecting surface defects in an image sensor system, primarily for detecting surface defects in an image sensor or a filter integrated in front of an image sensor. This method utilizes the principle of light diffraction / interference, illuminating the image sensor or the filter integrated in front of the image sensor with a light source. When light passes through a defect in the image sensor or the filter in front of the image sensor, it causes diffraction or interference of the incident light. The image sensor receives the light signal containing diffraction or interference and converts it into an electrical signal. An image signal processor converts the electrical signal containing diffraction or interference into an image, and the diffraction or interference spots on the image can be used to determine defects. This method utilizes the image sensor already in the image sensor system to receive the light signal containing diffraction or interference, eliminating the need for expensive, precision equipment. This method enables low-cost, high-precision, and rapid detection of surface defects in the image sensor system.

[0035] 2. This invention can be applied to camera maintenance after shipment or during second-hand camera transactions. It can quickly and accurately identify surface defects in camera filters, facilitating targeted cleaning and repair by repair personnel or photographers. In second-hand camera transactions, it can accurately identify the condition of the camera's filter, reducing transaction disputes.

[0036] 3. The present invention can be applied to industrial inspection, especially in the pre-shipment inspection of image sensors or components of image sensors with integrated filters. Compared with traditional industrial AOI inspection methods, it can reduce the cost of AOI equipment. By adding light source illumination only during routine power-on inspection and synchronously detecting surface defects, the inspection speed can be accelerated.

[0037] 4. The present invention further inputs the image output by the image signal processor into a surface defect detection model based on deep learning, which can more accurately determine the type, location, size, etc. of the defect and achieve process standardization. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The above is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0039] Figure 1 is a schematic diagram of a surface defect detection method using an image sensor system according to the present invention;

[0040] Figure 2 2 is a schematic diagram of the surface defect detection results of the image sensor system in the present invention, wherein (2a) is a particle detection result diagram, (2b) is a dust, oil stain, and fiber detection result diagram, and (2c) is a coating demolding or coating damage detection result diagram;

[0041] Figure 3Schematic diagram of the structure of the surface defect detection device of the image sensor system in Example 1 of the present invention (camera detection scene, no light shielding tube);

[0042] Figure 4 This is a schematic structural diagram of the surface defect detection device of the image sensor system in Example 1 of the present invention (camera detection scene, with light shielding tube);

[0043] Figure 5 Schematic diagram of the structure of the surface defect detection device of the image sensor system (a light-shielding cylinder equipped with a light source) in Example 1 of the present invention;

[0044] Figure 6 This is a schematic diagram of the structure of a surface defect detection device of an image sensor system in Example 2 of the present invention (industrial inspection scenario);

[0045] In the figure: 1-light source; 2-image sensor; 3-filter; 4-connector; 5-connector end adapted to the camera lens interface; 6-data transmission line; 7-stage. DETAILED DESCRIPTION

[0046] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0047] The present invention provides a surface defect detection method for an image sensor system, which is mainly used for surface defect detection of an image sensor or a filter integrated in front of an image sensor, wherein the image sensor is a CMOS or CCD image sensor; the surface defects of the filter are contamination defects or coating defects; the surface defects of the image sensor are contamination defects; contamination defects generally include dust, particles, fibers, oil stains or fingerprints, and coating defects generally include coating demolding or coating damage.

[0048] like Figure 1 As shown, the detection method of the present invention includes: using a light source to directly illuminate an image sensor or a filter integrated in front of the image sensor; when light passes through the image sensor or a defect on the filter in front of the image sensor, it causes diffraction or interference of the incident light; the image sensor receives the light signal containing diffraction or interference and converts it into an electrical signal; the image signal processor converts the electrical signal containing diffraction or interference into an image, and the diffraction or interference light spot on the image (such as Figure 22a is the particle detection result diagram, 2b is the dust, oil stain, and fiber detection result diagram, and 2c is the coating demoulding or coating damage detection result diagram.

[0049] The above-mentioned detection method of the present invention mainly utilizes the diffraction / interference principle of light and applies it to the defect detection of the image sensor system. When light passes through the defect to be detected, diffraction or interference occurs. The image sensor existing in the image sensor system is used to receive the light signal containing diffraction or interference and convert it into an electrical signal (which also contains diffraction or interference signals). Then, a conventional image signal processor can output an image containing diffraction or interference spots. By observing the image (direct observation or magnified observation), the defect information can be directly located, the approximate position, size and type of the defect can be determined, and preliminary detection can be achieved.

[0050] In order to achieve further refined detection, the image containing diffraction or interference spots output by the above-mentioned image signal processor can be input into a pre-built surface defect detection model based on deep learning, wherein the surface defect detection model based on deep learning is constructed according to the conventional construction method of the surface defect detection model, such as including:

[0051] 1. Data training sample collection: Collect a large number of images containing various surface defects (such as dust, particles, fibers, oil stains, fingerprints, coating demolding, coating damage, etc.);

[0052] 2. Use a deep learning-based model for training, such as selecting a suitable convolutional neural network model for training, to build a surface defect detection model based on deep learning.

[0053] By further inputting the image into a surface defect detection model based on deep learning, the type, location, size, etc. of the defect can be determined more accurately, and process standardization can be achieved.

[0054] This invention primarily has two application scenarios: one is post-production camera maintenance or second-hand camera trading, where the image sensor system needs to be inspected for defects; the other is industrial production inspection of image sensors or image sensor components with integrated filters. This will be explained in detail below using specific embodiments (using a CMOS image sensor as an example, but not limited to CMOS).

[0055] Example 1

[0056] This embodiment provides a detection device and a detection method for detecting whether an image sensor system in a camera is defective during camera maintenance or second-hand camera trading after the camera leaves the factory.

[0057] A camera typically consists of a lens, filter, image sensor (CMOS), and image signal processor (ISP). The filter is integrated in front of the image sensor, which is then connected to the ISP. Therefore, the camera's built-in image sensor (CMOS) can be used to inspect filters for surface defects. It should be noted that camera-based inspection primarily focuses on detecting surface defects on the filter. In cameras (primarily those with interchangeable lenses), the filter may be exposed, which can easily lead to surface defects.

[0058] like Figure 3 As shown, a surface defect detection device for an image sensor system in this embodiment is mainly used for defect detection of a filter integrated in front of an image sensor, and includes: a light source 1, an image sensor 2, a filter 3 and an image signal processor (not shown in the figure).

[0059] Among them, the light source 1 is used to provide incident light, directly irradiating the filter 3 integrated in front of the image sensor 2; when the light passes through the defects on the filter 3 in front of the image sensor 2, it will cause the incident light to diffract or interfere; the image sensor 2 is used to receive the light signal containing diffraction or interference and convert it into an electrical signal; the image signal processor converts the electrical signal containing diffraction or interference into an image, and realizes defect detection based on the diffraction or interference spots on the image.

[0060] Specifically, the light source can be an LED or laser light source, and the light source can be a single beam or multiple beams. The LED light source can be a white light LED, a monochromatic LED, or a mixed color LED. Monochromatic light sources, such as blue light or red light, are preferred. Preferably, a single beam is used to directly coaxially illuminate the surface to be inspected.

[0061] Recombination Figure 4 As shown, as a preferred solution, when performing camera inspection, the camera includes a filter 3 integrated in front of an image sensor 2 and an image signal processor connected to the image sensor; the camera is a camera with a detachable lens (e.g., a camera with an interchangeable lens, or a camera with a non-interchangeable but detachable lens); the light source can be directly located opposite the filter. To facilitate operation, the defect detection device can also include a connector 4, with the light source 1 mounted at the front end of the connector 4 and the rear end of the connector 4 connected to the camera; during inspection, the camera lens is removed, and then the connector 4 with the light source 1 mounted thereon is connected to the camera lens interface via its rear end connection end (connection end 5 adapted to the camera lens interface), so that the light source 1 can directly illuminate the filter 3 through the camera lens interface from which the camera lens has been removed. Those skilled in the art will appreciate that the connector 4 can also be connected to other parts of the camera and is not necessarily limited to the camera lens interface.

[0062] Of course, as a most preferred method, the connecting member 4 in this embodiment is in the form of a light-shielding tube, the light source 1 is installed at the front end of the light-shielding tube, and the rear end of the light-shielding tube is provided with a connecting end 5 adapted to the lens interface of the camera, and the connecting end 5 adapted to the lens interface of the camera is connected to the lens interface of the camera.

[0063] Figure 5 This is a schematic structural diagram of the surface defect detection device (light shielding tube with light source installed) of the image sensor system in Example 1 of the present invention, which consists of Figure 5 It can be seen that during camera inspection, the above-mentioned inspection device may only include the light source 1 and the connector 4, and the camera to be inspected is not a component of the surface defect inspection device of the image sensor system, but is only a part to be inspected.

[0064] Example 2

[0065] This embodiment provides an industrial production inspection of components of image sensors or image sensors integrated with filters, which detects surface defects of these two component products to ensure that they meet factory standards.

[0066] It should be noted that in certain specific scenarios in the post-process workshop of image sensor production, the image sensor needs to be powered on for testing (an output signal is required). This embodiment can simultaneously detect its surface defects or the surface defects of the filter integrated in front of it, mainly by adding an incident light source.

[0067] Specifically, combined Figure 6 As shown, this embodiment provides an image sensor system surface defect detection device, including a light source 1, an image sensor 2 and an image signal processor (not shown in the figure).

[0068] Among them, light source 1 is used to provide incident light to directly illuminate image sensor 2; when light passes through defects on image sensor 2 (it should be noted that there is a protective cover in front of a conventional image sensor, which is also a component of the image sensor itself, and there is a certain distance from the protective cover to the photosensitive part of the image sensor), it will cause the incident light to diffract or interfere; image sensor 2 is used to receive light signals containing diffraction or interference and convert them into electrical signals; the image signal processor converts the electrical signal containing diffraction or interference into an image, and realizes defect detection based on the diffraction or interference spots on the image.

[0069] Specifically, the light source can be an LED or laser light source, and the light source can be a single beam or multiple beams. The LED light source can be a white light LED, a monochromatic LED, or a mixed color LED. Monochromatic light sources, such as blue light or red light, are preferred. Preferably, a single beam is used to directly coaxially illuminate the surface to be inspected.

[0070] Recombination Figure 6 As shown, as a preferred solution, in order to facilitate operation, the image sensor 2 is preferably placed on the stage 7 and can transmit signals to an external image signal processor through the data transmission line 6.

[0071] The above-mentioned light source 1 can be directly set opposite the image sensor. When the actual image sensor 2 is powered on for detection, the light source 1 located opposite the image sensor 2 is turned on to illuminate the image sensor 2. The light signal containing diffraction or interference is received by the powered image sensor 2 to be detected and converted into an electrical signal, which is then transmitted to the image signal processor. The electrical signal containing diffraction or interference is converted into an image, and defect detection is achieved based on the diffraction or interference spots on the image.

[0072] The object to be detected in this embodiment may also be replaced by an image sensor with a filter integrated in the front.

[0073] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Those skilled in the art can make some simple modifications, equivalent changes or modifications based on the technical content disclosed above, which all fall within the scope of protection of the present invention.

Claims

1. A surface defect detection method of an image sensor system, characterized in that: Used for surface defect detection of image sensors or filters integrated in front of image sensors, including: Utilizing a light source to directly illuminate the image sensor or directly illuminate a filter integrated in front of the image sensor; When light passes through defects on the image sensor or the filter in front of the image sensor, it causes diffraction or interference of the incident light; The image sensor receives the light signal containing diffraction or interference and converts it into an electrical signal; The image signal processor converts the electrical signal containing diffraction or interference into an image, and the diffraction or interference spots on the image are used to judge defects.

2. The surface defect detection method of the image sensor system according to claim 1, characterized in that: The image sensor is a CMOS or CCD image sensor; And / or, the surface defect of the filter is a pollution defect or a coating defect; the surface defect of the image sensor is a pollution defect; And / or, the light source is an LED light source or a laser light source, and the light source is a single-beam light source or more than a single-beam light source.

3. The surface defect detection method of the image sensor system according to claim 1, characterized in that: When used for surface defect detection of a camera's image sensor system, the camera includes an image sensor with a filter integrated in the front and an image signal processor connected to the image sensor; the light source directly illuminates the filter through the camera lens interface after the camera lens is removed, and then the camera is used for detection; Or, when used in industry to detect image sensors alone or to detect filters integrated in front of image sensors, when the image sensor is powered on for testing, a light source is used to directly illuminate the image sensor or the filter integrated in front of the image sensor for defect detection.

4. The surface defect detection method of an image sensor system according to any one of claims 1 to 3, characterized in that: The image output by the image signal processor is further inspected by a pre-built surface defect detection model based on deep learning.

5. A surface defect detection device of an image sensor system, characterized in that: Used for surface defect detection of image sensors or filters integrated in front of image sensors, including: A light source is used to provide incident light to directly illuminate the image sensor or directly illuminate a filter integrated in front of the image sensor; Image sensors or image sensors with integrated filters in front of them. When light passes through defects in the image sensor or the filter in front of the image sensor, it causes diffraction or interference of the incident light. The image sensor is used to receive the light signal containing diffraction or interference and convert it into an electrical signal. The image signal processor converts the electrical signal containing diffraction or interference into an image, and the diffraction or interference spots on the image are used to judge defects.

6. The surface defect detection device of the image sensor system according to claim 5, characterized in that: When used for surface defect detection of an image sensor system of a camera, the camera includes the filter integrated in front of the image sensor and an image signal processor connected to the image sensor; The defect detection device further includes a connecting member, the light source is mounted on a front end of the connecting member, and a rear end of the connecting member is connected to a camera; the light source directly illuminates the filter through a camera lens interface of a disassembled camera lens; The camera to be inspected may or may not be part of a surface defect inspection device of an image sensor system.

7. The surface defect detection device of the image sensor system according to claim 6, characterized in that: The connecting member is a light-shielding tube, the light source is installed at the front end of the light-shielding tube, and the rear end of the light-shielding tube is provided with a connecting end adapted to the lens interface of the camera, and the connecting end is connected to the lens interface of the camera.

8. The surface defect detection device of the image sensor system according to claim 5, characterized in that: When used for industrial independent detection of image sensors or filters integrated in front of image sensors, the light source is arranged opposite the image sensor or the filter integrated in front of the image sensor, the image sensor to be detected or the image sensor with the filter integrated in front is mounted on the stage, and the image sensor is connected to the image signal processor via a data transmission line.

9. The surface defect detection device of an image sensor system according to any one of claims 5 to 8, characterized in that: The image sensor is a CMOS or CCD image sensor; And / or, the surface defect of the filter is a pollution defect or a coating defect; the surface defect of the image sensor is a pollution defect; And / or, the light source is an LED light source or a laser light source, and the light source is a single beam light source or more than one beam light source.

10. The surface defect detection device of an image sensor system according to any one of claims 5 to 8, characterized in that: It also includes a surface defect detection model unit based on deep learning, which is used to further detect the images output by the image signal processor.

Images