Image sensor defect detection method and system based on space-time correlation analysis

Abstract

The invention provides an image sensor defect detection method and system based on space-time correlation analysis, and electronic equipment. The method comprises the following steps: acquiring an image data set by using an image sensor; extracting spatio-temporal features from the image data set; analyzing and processing the spatio-temporal characteristics by utilizing a spatio-temporal correlation analysis model so as to identify defect categories; and counting defect categories to output a defect detection result. By extracting the spatial-temporal characteristics of the image data set and analyzing and processing the spatial-temporal characteristics by using the spatial-temporal correlation analysis model, the defects can be identified by using the spatial characteristics, and the dynamic defects are distinguished by using the time characteristics, so that the effectiveness of dynamic defect detection is improved, and the problem of how to effectively detect the dynamic defects of the image sensor is solved.

Description

Technical Field

[0001] The present invention relates to the field of image sensor detection technology, and in particular to an image sensor defect detection method and system based on spatiotemporal correlation analysis, and electronic equipment. Background Art

[0002] Image sensor is an important electronic device in the field of optoelectronics and is widely used in many fields such as industrial imaging, machine vision, scientific research, etc. In order to ensure the imaging quality of image sensors in applications, a series of functional tests are usually required before leaving the factory, such as resolution test, dark noise test, full well capacity test, signal-to-noise ratio test, gain test, etc.

[0003] With the development of image sensor technology, the requirements for dynamic imaging performance of image sensors are getting higher and higher. Therefore, it is necessary to effectively detect dynamic defects of image sensors. However, the existing image sensor defect detection methods usually only rely on static image analysis, which makes it difficult to capture instantaneous dynamic defect information, such as pixel flickering, drift, etc., and thus it is difficult to ensure the dynamic imaging performance of image sensors. Summary of the invention

[0004] The object of the present invention is to provide an image sensor defect detection method and system, and electronic equipment based on spatiotemporal correlation analysis, so as to solve the problem of how to effectively detect dynamic defects of image sensors.

[0005] In order to solve the above technical problems, the present invention provides an image sensor defect detection method based on spatiotemporal correlation analysis, comprising: Acquire an image data set using an image sensor; Extracting spatiotemporal features from image datasets; Using the spatiotemporal correlation analysis model, the spatiotemporal features are analyzed and processed to identify the defect categories; Count defect categories to output defect detection results.

[0006] Optionally, in the image sensor defect detection method based on spatiotemporal correlation analysis, the method of acquiring an image data set using an image sensor includes: Using an image sensor to collect images at different time points and in the same environment and / or images at the same time point and in different environments to obtain an initial image data set; The initial image data set is preprocessed to obtain an image data set.

[0007] Optionally, in the image sensor defect detection method based on spatiotemporal correlation analysis, the frequency of collecting images is adjusted according to the estimated frequency and period of occurrence of dynamic defects.

[0008] Optionally, in the image sensor defect detection method based on spatiotemporal correlation analysis, the spatiotemporal features include time series features and space series features; and the method for extracting spatiotemporal features from an image data set includes: From the image dataset, obtain the image acquisition time series; According to the collected time series, the time series characteristics are calculated; Extract target defect pixels from the image data set; The spatial sequence features are calculated based on the target defect pixel and its surrounding spatial neighborhood pixels.

[0009] Optionally, in the image sensor defect detection method based on spatiotemporal correlation analysis, the method of extracting spatiotemporal features from the image data set further includes: Using Fourier transform, spectral features are extracted from image datasets as time series features.

[0010] Optionally, in the image sensor defect detection method based on spatiotemporal correlation analysis, the method of using the spatiotemporal correlation analysis model to analyze and process the spatiotemporal features to identify the defect category includes: The spatiotemporal correlation analysis model is used to analyze the changes in the time series characteristics and spatial sequence characteristics of each pixel on the image sensor to identify dynamic defects and static defects.

[0011] Optionally, in the image sensor defect detection method based on spatiotemporal correlation analysis, the method of analyzing the changes in the time series characteristics and the space series characteristics of each pixel on the image sensor to identify dynamic defects and static defects includes: Set time series threshold and spatial series threshold; If the time series feature exceeds the time series threshold, and the space series feature exceeds the space series threshold, it is determined that the current pixel has a dynamic defect; If the time series feature exceeds the time series threshold, and the space series feature does not exceed the space series threshold, it is determined that the current pixel point has noise; If the time series feature does not exceed the time series threshold, and the space series feature exceeds the space series threshold, it is determined that the current pixel has a static defect.

[0012] Optionally, in the image sensor defect detection method based on spatiotemporal correlation analysis, the defect detection result includes defect category, defect data and defect pixel position.

[0013] In order to solve the above technical problems, the present invention further provides an image sensor defect detection system based on spatiotemporal correlation analysis, which is used to implement the image sensor defect detection method based on spatiotemporal correlation analysis as described in any one of the above items, and the image sensor defect detection system based on spatiotemporal correlation analysis includes: An image acquisition module is used to control the image sensor to capture images to obtain an image data set; Feature extraction module, used to extract spatiotemporal features from image datasets; An analysis and processing module is used to analyze and process the spatiotemporal features using a spatiotemporal correlation analysis model to identify defect categories; The result output module is used to count defect categories and output defect detection results.

[0014] In order to solve the above technical problems, the present invention also provides an electronic device, including a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, the image sensor defect detection method based on spatiotemporal correlation analysis as described in any one of the above items is executed.

[0015] The image sensor defect detection method and system based on spatiotemporal correlation analysis and electronic equipment provided by the present invention include: using an image sensor to obtain an image data set; extracting spatiotemporal features from the image data set; using a spatiotemporal correlation analysis model to analyze and process the spatiotemporal features to identify defect categories; and statistically analyzing the defect categories to output defect detection results. By extracting the spatiotemporal features of the image data set and analyzing and processing the spatiotemporal features using a spatiotemporal correlation analysis model, it is possible to identify defects using spatial features and distinguish dynamic defects using temporal features, thereby improving the effectiveness of dynamic defect detection and solving the problem of how to effectively detect dynamic defects of image sensors. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flow chart of the image sensor defect detection method based on spatiotemporal correlation analysis provided in this embodiment; Figure 2 A schematic diagram of the judgment logic of defect identification provided in this embodiment; Figure 3 A schematic diagram of the structure of an image sensor defect detection system based on spatiotemporal correlation analysis provided in this embodiment. DETAILED DESCRIPTION

[0017] The image sensor defect detection method and system based on spatiotemporal correlation analysis and the electronic device proposed in the present invention are further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In addition, the structure shown in the accompanying drawings is often part of the actual structure. In particular, the emphasis that each drawing needs to show is different, and sometimes different proportions are used.

[0018] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present invention are used to distinguish similar objects in order to describe the embodiments of the present invention, rather than to describe a specific order or sequence. It should be understood that the structures used in this way can be interchanged under appropriate circumstances. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0019] This embodiment provides an image sensor defect detection method based on spatiotemporal correlation analysis. Figure 1 As shown, including: S1, using an image sensor to obtain an image data set; S2, extracting spatiotemporal features from image datasets; S3, using the spatiotemporal correlation analysis model, analyzes and processes the spatiotemporal features to identify the defect category; S4, count the defect categories to output the defect detection results.

[0020] The image sensor defect detection method based on spatiotemporal correlation analysis provided in this embodiment extracts the spatiotemporal features of the image data set and uses the spatiotemporal correlation analysis model to analyze and process the spatiotemporal features. It can identify defects using spatial features and distinguish dynamic defects using temporal features, thereby improving the effectiveness of dynamic defect detection and solving the problem of how to effectively detect dynamic defects of image sensors.

[0021] Specifically, in this embodiment, step S1, the method of obtaining an image data set using an image sensor includes: using the image sensor to collect images at different time points and in the same environment and / or images at the same time point and in different environments to obtain an initial image data set.

[0022] In practical applications, the occurrence frequency and period of dynamic defects can be estimated according to the characteristics of the image sensor, and when acquiring images, the acquisition frequency can be adjusted according to the estimated occurrence frequency and period of dynamic defects.

[0023] Preferably, in order to eliminate noise interference in the image and ensure the accuracy of subsequent feature extraction, in this embodiment, step S1 also includes: preprocessing the initial image data set to obtain an image data set.

[0024] In practical applications, preprocessing may include filtering and denoising. Of course, other processing processes, such as enhancement processing, may also be added according to actual needs.

[0025] Furthermore, in this embodiment, step S2 extracts spatiotemporal features from the image data set, wherein the spatiotemporal features include time series features and space series features.

[0026] In this embodiment, step S2 specifically includes: S21, obtaining an image acquisition time series from an image data set.

[0027] Specifically, in this embodiment, the acquisition time of each image in the image database is obtained, and the images are arranged in chronological order, so as to obtain an acquisition time sequence.

[0028] S22, calculating and obtaining time series characteristics according to the collected time series.

[0029] Specifically, in this embodiment, the pixel value of each pixel point in the image sensor in each image in the image data set is obtained, and multiple pixel values ​​of the same pixel point are arranged according to the acquisition time sequence to obtain a time series; then, the standard deviation of the time series is calculated to obtain the time series feature. In this embodiment, the standard deviation of the time series is recorded as , that is, the time series feature is .

[0030] In practical applications, if the occurrence of dynamic defects has a certain regularity in the time series, Fourier transform can also be used to extract spectral features from the image data set as time series features. The method of extracting spectral features using Fourier transform is well known to those skilled in the art, and this application will not repeat it in detail.

[0031] In other embodiments, the difference between the maximum and minimum pixel gray values ​​within the image acquisition time period can be extracted as a time series feature; or the rate of change of the pixel gray values ​​between adjacent frame images can be extracted as a time series feature. These parameters can reflect pixel flicker defects, and the rate of change can also distinguish pixel drift defects (when there is a drift defect, the rate of change of the pixel gray values ​​between adjacent frame images is lower). In practical applications, it can be selected according to actual needs, and this application does not impose any restrictions on this.

[0032] S23, extracting target defect pixels from the image data set.

[0033] Specifically, in this embodiment, pixel points with large differences in pixel values ​​from surrounding areas can be extracted as target defective pixel points through mean judgment or interpolation judgment. The specific judgment method is well known to those skilled in the art and will not be elaborated in this application.

[0034] Of course, if the detection efficiency is not considered, in order to improve the defect detection accuracy, all pixel points can be processed with spatial sequence features, that is, the target defect pixel points extracted here are all the pixel points of the image sensor.

[0035] S24, calculating and obtaining spatial sequence features according to the target defect pixel and its surrounding spatial neighborhood pixels.

[0036] Specifically, in practical applications, the target defective pixel point can be placed in the middle of the calculation window, and the standard deviation of the pixel values ​​of all pixels in the calculation window can be calculated to obtain the spatial sequence characteristics.

[0037] In order to ensure that there are sufficient and balanced pixels in all directions around the target defective pixel point to participate in the calculation during the calculation process, the size of the calculation window is usually set to N×N, where N≥3. Of course, those skilled in the art can understand that the larger the size of the calculation window, the more data for standard deviation calculation, the lower the calculation efficiency, but the higher the accuracy of the calculation result, so the size of the calculation window needs to be reasonably set according to the actual situation.

[0038] Of course, in other embodiments, an N×M (rectangular) calculation window may also be selected to calculate the spatial sequence feature, and this application does not impose any limitation on this.

[0039] And, in other embodiments, the spatial sequence feature can also be obtained by calculating the mean difference between the target defect pixel and the pixels in the surrounding spatial neighborhood. In practical applications, the selection can be made according to actual needs, and this application does not limit this.

[0040] It should be noted that, in actual applications, the implementation order of step S21 and step S22 can be swapped with that of step S23 and step S24 or performed simultaneously. The change in the order of steps does not affect the implementation of the scheme of this application. The scheme after the order of steps is changed without violating the purpose of this application should also fall within the scope of protection of this application.

[0041] Furthermore, in this embodiment, step S3, using the spatiotemporal correlation analysis model to analyze and process the spatiotemporal features to identify the defect category includes: using the spatiotemporal correlation analysis model to analyze the changes in the time series characteristics and spatial sequence characteristics of each pixel on the image sensor to identify dynamic defects and static defects.

[0042] Specifically, association rules can be obtained according to the image data set collected by the image sensor, and a spatiotemporal association analysis model can be constructed based on the association rules. The spatiotemporal association analysis model can then be used to analyze and process the time series features and space series features, thereby identifying dynamic defects and static defects.

[0043] In practical applications, machine learning methods can be used to construct spatiotemporal correlation analysis models, such as decision trees, neural networks, etc. The method of constructing a spatiotemporal correlation analysis model is well known to those skilled in the art, and will not be described in detail in this application.

[0044] Preferably, in order to ensure the processing accuracy and robustness of the spatiotemporal correlation analysis model, the constructed spatiotemporal correlation analysis model can be trained. In this embodiment, the spatiotemporal correlation analysis model can be automatically optimized by adding a training model to improve the model training efficiency.

[0045] In this embodiment, the determination method for identifying dynamic defects and static defects can be implemented with the help of an abnormality detection algorithm, specifically, Figure 2 As shown, including: Set time series threshold and spatial series threshold; If the time series feature exceeds the time series threshold, and the space series feature exceeds the space series threshold, it is determined that the current pixel has a dynamic defect; If the time series feature exceeds the time series threshold, and the space series feature does not exceed the space series threshold, it is determined that the current pixel point has noise; If the time series feature does not exceed the time series threshold, and the space series feature exceeds the space series threshold, it is determined that the current pixel has a static defect; If the time series feature does not exceed the time series threshold, and the space series feature does not exceed the space series threshold, it is judged that the current pixel is normal and there is no defect.

[0046] Among them, the setting of the time series threshold and the space series threshold needs to be reasonably set according to the characteristics of the image sensor to ensure normal and accurate identification of dynamic defects and static defects.

[0047] S4, count the defect categories to output the defect detection results.

[0048] Specifically, in this embodiment, all pixels of the image sensor are traversed to obtain the defect detection result of each pixel, and the defect detection results of all pixels are counted to obtain the final defect detection result.

[0049] In this embodiment, the defect detection results include but are not limited to defect categories, defect data, defect pixel locations, etc. Among them, defect categories include the above-mentioned dynamic defects, static defects and noise, defect data include pixel values, time series features and spatial series features corresponding to pixels identified as defects, etc., and defect pixel locations can accurately locate defective pixels.

[0050] The image sensor defect detection method based on spatiotemporal correlation analysis provided in this embodiment can not only identify pixel defects in image sensors, but also distinguish dynamic defects, static defects and noise, broadening the application scope of defect detection and being suitable for dynamic defect detection. The image sensor defect detection method based on spatiotemporal correlation analysis provided in this embodiment, with the help of the spatiotemporal correlation analysis model, can accurately identify dynamic defects and static defects at a lower computational cost, thereby improving the efficiency of defect detection; at the same time, the spatiotemporal correlation analysis model can achieve self-optimization through the training model, so that the detection accuracy of the spatiotemporal correlation analysis model gradually improves as the application time goes by.

[0051] This embodiment also provides an image sensor defect detection system based on time-space correlation analysis, which is used to implement the image sensor defect detection method based on time-space correlation analysis as described above. Figure 3 As shown, the image sensor defect detection system based on spatiotemporal correlation analysis includes: An image acquisition module is used to control the image sensor to capture images to obtain an image data set; Feature extraction module, used to extract spatiotemporal features from image datasets; An analysis and processing module is used to analyze and process the spatiotemporal features using a spatiotemporal correlation analysis model to identify defect categories; The result output module is used to count defect categories and output defect detection results.

[0052] The image sensor defect detection system based on spatiotemporal correlation analysis provided in this embodiment extracts the spatiotemporal features of the image data set through a feature extraction module, and analyzes and processes the spatiotemporal features using the spatiotemporal correlation analysis model in the analysis and processing module. It can identify defects using spatial features and distinguish dynamic defects using temporal features, thereby improving the effectiveness of dynamic defect detection and solving the problem of how to effectively detect dynamic defects of image sensors.

[0053] Preferably, in this embodiment, the analysis and processing module also includes a training model, and the training model is used to train and optimize the spatiotemporal correlation analysis model, thereby enabling self-training of the spatiotemporal correlation analysis model and improving the model training efficiency and accuracy.

[0054] In addition, this embodiment also provides an electronic device, including a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, the image sensor defect detection method based on spatiotemporal correlation analysis as described above is executed.

[0055] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other. In addition, the different parts between the various embodiments can also be used in combination with each other, and the present invention is not limited to this.

[0056] The image sensor defect detection method and system based on spatiotemporal correlation analysis and the electronic device provided in this embodiment include: using an image sensor to obtain an image data set; extracting spatiotemporal features from the image data set; using a spatiotemporal correlation analysis model to analyze and process the spatiotemporal features to identify defect categories; and statistically classifying the defect categories to output defect detection results. By extracting the spatiotemporal features of the image data set and analyzing and processing the spatiotemporal features using a spatiotemporal correlation analysis model, it is possible to identify defects using spatial features and distinguish dynamic defects using temporal features, thereby improving the effectiveness of dynamic defect detection and solving the problem of how to effectively detect dynamic defects of image sensors.

[0057] The above description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the claims.

Claims

1. An image sensor defect detection method based on spatiotemporal correlation analysis, characterized in that: include: Acquire an image data set using an image sensor; Extracting spatiotemporal features from image datasets; Using the spatiotemporal correlation analysis model, the spatiotemporal features are analyzed and processed to identify the defect categories; Count defect categories to output defect detection results.

2. The image sensor defect detection method based on spatiotemporal correlation analysis according to claim 1, characterized in that: The method for acquiring an image data set using an image sensor comprises: Using an image sensor to collect images at different time points and in the same environment and / or images at the same time point and in different environments to obtain an initial image data set; The initial image data set is preprocessed to obtain an image data set.

3. The image sensor defect detection method based on spatiotemporal correlation analysis according to claim 2, characterized in that: The frequency of collecting images is adjusted according to the estimated frequency and period of dynamic defects.

4. The image sensor defect detection method based on spatiotemporal correlation analysis according to claim 1, characterized in that: The spatiotemporal characteristics include time series characteristics and space series characteristics; The method for extracting spatiotemporal features from an image data set comprises: From the image dataset, obtain the image acquisition time series; According to the collected time series, the time series characteristics are calculated; Extract target defect pixels from the image data set; The spatial sequence features are calculated based on the target defect pixel and its surrounding spatial neighborhood pixels.

5. The image sensor defect detection method based on spatiotemporal correlation analysis according to claim 4, characterized in that: The method for extracting spatiotemporal features from an image data set also includes: Using Fourier transform, spectral features are extracted from image datasets as time series features.

6. The image sensor defect detection method based on spatiotemporal correlation analysis according to claim 4, characterized in that: The method of analyzing and processing the spatiotemporal features using the spatiotemporal correlation analysis model to identify defect categories includes: The spatiotemporal correlation analysis model is used to analyze the changes in the time series characteristics and spatial sequence characteristics of each pixel on the image sensor to identify dynamic defects and static defects.

7. The image sensor defect detection method based on spatiotemporal correlation analysis according to claim 6, characterized in that: The method of analyzing the changes in the time series characteristics and the space series characteristics of each pixel on the image sensor to identify dynamic defects and static defects includes: Set time series threshold and spatial series threshold; If the time series feature exceeds the time series threshold, and the space series feature exceeds the space series threshold, it is determined that the current pixel has a dynamic defect; If the time series feature exceeds the time series threshold, and the space series feature does not exceed the space series threshold, it is determined that the current pixel point has noise; If the time series feature does not exceed the time series threshold, and the space series feature exceeds the space series threshold, it is determined that the current pixel has a static defect.

8. The image sensor defect detection method based on spatiotemporal correlation analysis according to claim 1, characterized in that: The defect detection result includes defect category, defect data and defect pixel location.

9. An image sensor defect detection system based on spatiotemporal correlation analysis, used to implement the image sensor defect detection method based on spatiotemporal correlation analysis as claimed in any one of claims 1 to 8, characterized in that: The image sensor defect detection system based on spatiotemporal correlation analysis includes: An image acquisition module is used to control the image sensor to capture images to obtain an image data set; Feature extraction module, used to extract spatiotemporal features from image datasets; An analysis and processing module is used to analyze and process the spatiotemporal features using a spatiotemporal correlation analysis model to identify defect categories; The result output module is used to count defect categories and output defect detection results.

10. An electronic device, characterized in that: It includes a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, the image sensor defect detection method based on spatiotemporal correlation analysis as described in any one of claims 1 to 8 is executed.

Images