Image sensor defect identification method and system based on feature fusion, and electronic equipment

Abstract

The invention provides an image sensor defect identification method and system based on feature fusion, and electronic equipment. The method comprises the following steps: obtaining quantum efficiency spectrum data and dark current data of an image sensor; acquiring spectral characteristics of the image sensor according to the quantum efficiency spectral data; acquiring dark current characteristics of the image sensor according to the dark current data; performing feature fusion on the spectral features and the dark current features to obtain a joint feature vector; and obtaining a defect identification result of the image sensor according to the joint feature vector. According to the method, feature extraction and feature fusion are performed on quantum efficiency spectrum data and dark current data, so that relevance between spectrum response abnormity and dark current parameter degradation is established by the obtained joint feature vector, and defects of the image sensor can be quickly and accurately identified by using the joint feature vector; the problem of how to efficiently and accurately identify the defects of the image sensor is solved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, system, and electronic device for identifying defects in image sensors based on feature fusion. Background Technology

[0002] Image sensors utilize the photoelectric conversion function of optoelectronic devices to convert the light image on a photosensitive surface into an electrical signal proportional to the light image. Currently, commonly used image sensors include CCD sensors and CIS sensors. Among them, CIS sensors, due to their compatibility with semiconductor CMOS technology, have the characteristics of high integration, low power consumption, high speed, and low cost, and have developed rapidly in recent years in terms of wide dynamic range and low illumination.

[0003] To ensure the functionality of a CIS image sensor, various tests are typically required. Among these tests, online defect detection is a test that uses images captured by the image sensor to confirm whether there are any abnormalities in the image sensor hardware.

[0004] Currently, conventional online defect detection includes dark current testing and quantum efficiency testing. However, these two tests have different testing conditions, so they can only be performed independently, and the resulting test data are also independent of each other. To improve the accuracy of online defect detection, a significant amount of manpower and time is required to manually integrate and analyze the test data from these two tests. This is not only inefficient and untimely, but also prone to errors due to human oversight, resulting in low reliability. Summary of the Invention

[0005] The purpose of this invention is to provide a method, system, and electronic device for identifying image sensor defects based on feature fusion, so as to solve the problem of how to efficiently and accurately identify image sensor defects.

[0006] To address the aforementioned technical problems, this invention provides a method for defect identification of image sensors based on feature fusion, comprising: Acquire quantum efficiency spectral data and dark current data from image sensors; The spectral characteristics of the image sensor are obtained based on quantum efficiency spectral data; Obtain the dark current characteristics of the image sensor based on dark current data; Spectral features and dark current features are fused to obtain a joint feature vector; Defect identification results of image sensors are obtained based on joint feature vectors.

[0007] Optionally, in the aforementioned image sensor defect identification method based on feature fusion, the method for obtaining the quantum efficiency spectral data and dark current parameters of the image sensor includes: A wide-band light source was used to illuminate the image sensor, and the intensity of reflected light from the image sensor at different wavelengths was measured to obtain quantum efficiency spectral data. In a dark environment, the dark current value generated by the image sensor is measured to obtain dark current data.

[0008] Optionally, in the aforementioned image sensor defect identification method based on feature fusion, the method for obtaining the spectral features of the image sensor based on quantum efficiency spectral data includes: Preprocessing of quantum efficiency spectral data; Constructing a convolutional neural network; A convolutional neural network is used to process the preprocessed quantum efficiency spectral data to extract spectral features.

[0009] Optionally, in the aforementioned image sensor defect identification method based on feature fusion, the method for preprocessing the quantum efficiency spectral data includes: Quantum efficiency spectral data is compared with standard spectral sample data to remove abnormal spectral data; The quantum efficiency spectral data after removing anomalous spectral data is normalized.

[0010] Optionally, in the image sensor defect recognition method based on feature fusion, the constructed convolutional neural network includes multiple convolutional layers with different kernel sizes to extract spectral feature maps at different scales.

[0011] Optionally, in the image sensor defect recognition method based on feature fusion, each convolutional layer is followed by an activation function, which is used to increase the nonlinear expressive power of the convolutional neural network.

[0012] Optionally, in the image sensor defect recognition method based on feature fusion, the constructed convolutional neural network further includes a pooling layer, which is used to downsample the spectral feature map output by the convolutional layer to obtain spectral features.

[0013] Optionally, in the aforementioned image sensor defect identification method based on feature fusion, the method for obtaining the dark current features of the image sensor based on dark current data includes: Noise reduction and filtering are applied to the dark current data. Construct a recurrent neural network; A recurrent neural network is used to process the dark current data after noise reduction and filtering in order to extract dark current features.

[0014] Optionally, in the image sensor defect identification method based on feature fusion, the method for fusing spectral features and dark current features to obtain a joint feature vector includes: Define the spectral feature vector based on spectral characteristics, and define the dark current feature vector based on dark current characteristics; Calculate the vector-level attention weights for spectral features and dark current features; Calculate the dimensional attention weights for spectral features and dark current features; Spectral features and dark current features are weighted using vector-level attention weights and dimension-level attention weights to obtain spectral weighted features and dark current weighted features; The spectral weighted features and dark current weighted features are concatenated to obtain a joint feature vector.

[0015] Optionally, in the aforementioned image sensor defect recognition method based on feature fusion, the method for calculating the vector-level attention weights of spectral features and dark current features includes: Assign the first scalar weights to the entire vector of spectral features and dark current features; The spectral eigenvector and the dark current eigenvector are concatenated to obtain the initial joint eigenvector; The initial joint feature vector is processed using a fully connected network to obtain two scalar vectors; The softmax function is used to process the two scalar vectors to obtain vector-level attention weights for spectral features and dark current features.

[0016] Optionally, in the aforementioned image sensor defect recognition method based on feature fusion, the method for calculating the dimensionality-level attention weights of spectral features and dark current features includes: A second scalar weight is assigned to each dimension of the spectral features and dark current features; Using a neural network, the spectral feature vector and the dark current feature vector are processed separately to obtain the spectral dimension vector and the dark current dimension vector respectively. The sigmoid function is used to process the spectral dimension vector and the dark current dimension vector respectively to obtain the dimensional attention weights of the spectral features and the dark current features.

[0017] Optionally, in the image sensor defect recognition method based on feature fusion, the method of weighting spectral features and dark current features using vector-level attention weights and dimensional attention weights to obtain spectral weighted features and dark current weighted features includes: The spectral feature vector is first multiplied by its dimensional attention weights, and then multiplied by its vector-level attention weights to obtain the spectral weighted features. The dark current feature vector is first multiplied by its dimensional attention weight, and then multiplied by its vector-level attention weight to obtain the dark current weighted feature.

[0018] Optionally, in the aforementioned image sensor defect recognition method based on feature fusion, the method for obtaining the defect recognition result of the image sensor based on the joint feature vector includes: Construct a defect identification network; The joint feature vector is processed using a defect identification network to obtain the probability corresponding to each defect; The defect with the highest probability value is taken as the defect identification result.

[0019] To address the aforementioned technical problems, the present invention also provides an image sensor defect identification system based on feature fusion, used to implement the image sensor defect identification method based on feature fusion as described in any of the preceding claims, wherein the image sensor defect identification system based on feature fusion includes: The data acquisition module is used to acquire quantum efficiency spectral data and dark current data from the image sensor; The feature extraction module is used to obtain the spectral features of the image sensor based on quantum efficiency spectral data and the dark current features of the image sensor based on dark current data. The feature fusion module is used to fuse spectral features and dark current features to obtain a joint feature vector; The defect identification module is used to obtain the defect identification results of the image sensor based on the joint feature vector.

[0020] To address the aforementioned technical problems, the present invention also provides an electronic device, including a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, it executes the image sensor defect identification method based on feature fusion as described in any of the preceding claims.

[0021] This invention provides a method, system, and electronic device for image sensor defect identification based on feature fusion, comprising: acquiring quantum efficiency spectral data and dark current data of an image sensor; acquiring spectral features of the image sensor based on the quantum efficiency spectral data; acquiring dark current features of the image sensor based on the dark current data; fusing the spectral features and dark current features to obtain a joint feature vector; and obtaining the defect identification result of the image sensor based on the joint feature vector. By extracting and fusing features from the quantum efficiency spectral data and dark current data, the resulting joint feature vector establishes a correlation between spectral response anomalies and dark current parameter degradation. This enables rapid and accurate identification of image sensor defects using the joint feature vector, solving the problem of how to efficiently and accurately identify image sensor defects. Attached Figure Description

[0022] Figure 1 This is a flowchart of the image sensor defect recognition method based on feature fusion provided in this embodiment; Figure 2 This is a structural block diagram of the image sensor defect recognition system based on feature fusion provided in this embodiment. Detailed Implementation

[0023] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a more comprehensive overview of the image sensor defect identification method, system, and electronic device based on feature fusion proposed in this invention. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise scales, intended only to facilitate and clarify the illustration of the embodiments of this invention. Furthermore, the structures shown in the drawings are often part of the actual structures. In particular, different figures may emphasize different aspects and may sometimes use different scales.

[0024] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this invention are used to distinguish similar objects in order to describe embodiments of the invention, and are not used to describe a specific order or sequence. It should be understood that such uses of terminology are interchangeable where appropriate. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] This embodiment provides a defect identification method for image sensors based on feature fusion, such as... Figure 1 As shown, it includes: S1, acquire quantum efficiency spectral data and dark current data of the image sensor; S2, Obtain the spectral characteristics of the image sensor based on quantum efficiency spectral data; S3, Obtain the dark current characteristics of the image sensor based on the dark current data; S4, feature fusion of spectral features and dark current features to obtain a joint feature vector; S5, obtain the defect identification results of the image sensor based on the joint feature vector.

[0026] The image sensor defect identification method based on feature fusion provided in this embodiment extracts and fuses features from quantum efficiency spectral data and dark current data, so that the resulting joint feature vector establishes a correlation between spectral response anomalies and dark current parameter degradation. This enables the rapid and accurate identification of image sensor defects using the joint feature vector, thus solving the problem of how to efficiently and accurately identify image sensor defects.

[0027] It should be noted that in practical applications, the order of implementation of steps S2 and S3 can be interchanged or performed simultaneously. Adjusting the order of implementation of steps S2 and S3 does not affect the implementation of the technical solution of this application. Technical solutions with adjusted step order, provided they do not violate the spirit of this application, should also fall within the scope of protection of this application.

[0028] Specifically, in this embodiment, step S1, the method for acquiring the quantum efficiency spectral data and dark current data of the image sensor, includes: S11 uses a wide-band light source to illuminate the image sensor and measures the intensity of reflected light from the image sensor at different wavelengths to obtain quantum efficiency spectral data.

[0029] In practical applications, a spectrometer can be used to accurately measure the intensity of reflected light from an image sensor at different wavelengths, thereby obtaining complete quantum efficiency spectral data.

[0030] S12 measures the dark current value generated by the image sensor in a dark environment to obtain dark current data.

[0031] In practical applications, high-precision current measuring instruments can be used to measure the dark current value generated by image sensors.

[0032] Furthermore, in this embodiment, step S2, the method for obtaining the spectral characteristics of the image sensor based on quantum efficiency spectral data, includes: S21, preprocessing the quantum efficiency spectral data.

[0033] Specifically, considering that fluctuations in light sources and instrument noise can cause errors in quantum efficiency spectral data, this embodiment requires preprocessing the quantum efficiency spectral data to ensure its accuracy. In practical applications, the quantum efficiency spectral data can be compared with standard spectral sample data to remove abnormal spectral data; then, the quantum efficiency spectral data after removing abnormal spectral data can be normalized.

[0034] S22, Construct a convolutional neural network.

[0035] Specifically, in this embodiment, the constructed convolutional neural network includes an input layer, multiple convolutional layers, and pooling layers. The input layer is used to receive preprocessed quantum efficiency spectral data. Each convolutional layer is used to perform convolution processing on the quantum efficiency spectral data to extract spectral feature maps. The pooling layer is used to downsample the spectral feature maps output by the convolutional layers, thereby reducing the data dimensionality and retaining important spectral feature information while reducing computational load.

[0036] Preferably, in this embodiment, the kernel sizes of the multiple convolutional layers are different to extract spectral feature maps at different scales, thereby enriching the spectral features and ensuring the accuracy of the calculation results. In practical applications, the convolutional kernels can be set according to actual image processing needs, such as 1×1, 1×2, 3×3, etc., and this application does not impose any restrictions on this.

[0037] More preferably, in this embodiment, an activation function can be connected after each convolutional layer, the activation function being used to increase the non-linear expressive power of the convolutional neural network.

[0038] S23 uses a convolutional neural network to process the preprocessed quantum efficiency spectral data in order to extract spectral features.

[0039] It should be noted that in practical applications, step S22 can be performed before step S21, so that the convolutional neural network can be trained after it has been constructed. Furthermore, step S22 can be performed simultaneously with step S21, thereby reducing data processing time and improving defect identification efficiency. Technical solutions with adjusted step order without departing from the spirit of this application should also fall within the scope of protection of this application.

[0040] Furthermore, in this embodiment, step S3, the method for obtaining the dark current characteristics of the image sensor based on the dark current data, includes: S31 performs noise reduction filtering on the dark current data.

[0041] Specifically, in this embodiment, the dark current data can be processed by noise reduction filtering methods such as median filtering and Gaussian filtering to remove random noise interference in the dark current data measurement process, thereby improving the accuracy of the dark current data.

[0042] S32, Construct a recurrent neural network.

[0043] Specifically, in this embodiment, considering that dark current data is typically one-dimensional time-series data, a recurrent neural network (RNN) is constructed, with an input layer preceding it. This input layer performs appropriate dimensionality transformation on the dark current data before inputting it into the RNN, ensuring that the RNN can accurately identify and process the dark current data. Furthermore, the RNN is used to capture the changing characteristics of the dark current data, learning the fluctuation patterns, trends, and abnormal change patterns of the dark current over time, thereby obtaining dark current features.

[0044] Recurrent Neural Networks (RNNs) are a class of artificial neural network models specifically designed for processing sequential data. Their core characteristic lies in their "memory" capability: by transmitting information over time through hidden states, the model can capture temporal dependencies within the context. The specific network architecture can be established by those skilled in the art based on existing technology and practical needs, and will not be elaborated upon here.

[0045] S33 uses a recurrent neural network to process the dark current data after noise reduction and filtering in order to extract dark current features.

[0046] It should be noted that in practical applications, step S32 can be performed before step S31, so that the recurrent neural network can be trained after it has been constructed. Furthermore, step S32 can be performed simultaneously with step S31, thereby reducing data processing time and improving defect identification efficiency. Technical solutions with adjusted step order without departing from the spirit of this application should also fall within the scope of protection of this application.

[0047] Furthermore, in this embodiment, step S4, the method for fusing spectral features and dark current features to obtain a joint feature vector, includes: S41, define the spectral eigenvector based on spectral characteristics, and define the dark current eigenvector based on dark current characteristics.

[0048] Specifically, in this embodiment, the shape of the spectral feature vector is defined as [N, D1], where N represents the number of samples, each sample corresponds to one spectral feature vector, and D1 represents the dimension of the spectral feature vector corresponding to each sample. In practical applications, the dimension D1 of the spectral feature vector can be, for example, 128 dimensions.

[0049] Furthermore, the shape of the dark current feature vector is defined as [N, D2], where N represents the number of samples, each sample corresponds to a dark current feature vector, and D2 represents the dimension of the dark current feature vector corresponding to each sample. In practical applications, the dimension D2 of the dark current feature vector can be, for example, 64 dimensions.

[0050] In practical applications, the number of samples N in the spectral feature vector and the dark current feature vector should be kept consistent to ensure the effectiveness of subsequent calculations.

[0051] S42, calculates the vector-level attention weights for spectral features and dark current features.

[0052] Specifically, in this embodiment, firstly, a first scalar weight is assigned to the entire vector of spectral features and dark current features to highlight the more important feature vectors. That is, the first scalar weight is multiplied by the spectral feature vector and the dark current feature vector to obtain the spectral feature vector and the dark current feature vector used in subsequent steps to concatenate them to obtain the initial joint feature vector.

[0053] Then, the spectral eigenvector and the dark current eigenvector are concatenated to obtain an initial joint eigenvector, which has the shape [N, D1 + D2], and in the example above, can be represented as [N, 192]. In this step, the spectral eigenvector and dark current eigenvector used to obtain the initial joint eigenvector are the spectral eigenvector and dark current eigenvector after the first scalar weight allocation.

[0054] Next, a fully connected network is used to process the initial joint feature vector to obtain two scalar vectors. Specifically, in this embodiment, the fully connected network includes two fully connected layers. The first fully connected layer maps the initial joint feature vector from a high dimension to a low dimension, for example, mapping dimension D1+D2 from 192 to 64. At the same time, the ReLU activation function is used to enhance the non-linearity of the feature vector. The second fully connected layer maps the low-dimensional initial joint feature vector to 2 dimensions, for example, mapping dimension D1+D2 from 64 to 2, thereby outputting two scalar scores (scalar vectors).

[0055] Finally, the two scalar vectors are normalized using the softmax function (normalized according to the sample dimension) to obtain the vector-level attention weights of the spectral features and dark current features, denoted as w1 and w2 respectively, and w1 and w2 satisfy: w1+w2=1, and both have the shape [N,1].

[0056] S43, calculates the dimensional attention weights for spectral features and dark current features.

[0057] Specifically, in this embodiment, firstly, a second scalar weight is assigned to each dimension of the spectral features and dark current features to highlight the more important dimensions within the features. That is, the second scalar weight is multiplied by the spectral feature vector and the dark current feature vector to obtain the spectral feature vector and dark current feature vector used in subsequent steps to calculate the dimension weights.

[0058] Then, using a neural network, the spectral feature vector and the dark current feature vector are processed separately to obtain the corresponding spectral dimension vector and dark current dimension vector. Specifically, the neural network is a single-layer neural network. After the spectral feature vector [N,D1] (the spectral feature vector after the second scalar weight allocation) is input into the single-layer neural network, the output is the spectral dimension vector [N,D1] of the same dimension; and after the dark current feature vector [N,D2] (the dark current feature vector after the second scalar weight allocation) is input into the single-layer neural network, the output is the dark current dimension vector [N,D2] of the same dimension.

[0059] Finally, the sigmoid function is used to process the spectral dimension vector and the dark current dimension vector respectively, and the weights are restricted to between 0 and 1 to obtain the dimension-level attention weights of the spectral features and the dark current features, denoted as a1 and a2 respectively, where the shape of a1 is [N,D1] and the shape of a2 is [N,D2].

[0060] S44 uses vector-level attention weights and dimension-level attention weights to weight spectral features and dark current features to obtain spectral weighted features and dark current weighted features.

[0061] Specifically, in this embodiment, the spectral feature vector [N,D1] (the original spectral feature vector without scalar weight allocation) is first multiplied element-wise with its dimension-level attention weight a1 to highlight important dimensions, and then multiplied with its vector-level attention weight w1. It is then expanded to [N,D1] through a broadcasting mechanism to obtain the spectral weighted feature F1_weighted.

[0062] Furthermore, the dark current feature vector [N,D2] (the original dark current feature vector without scalar weight assignment) is first multiplied element-wise with its dimension-level attention weight a2 to highlight important dimensions, and then multiplied with its vector-level attention weight w2, and expanded to [N,D2] through a broadcast mechanism to obtain the dark current weighted feature F2_weighted.

[0063] S45 concatenates the spectral weighted features and the dark current weighted features to obtain a joint feature vector.

[0064] Specifically, in this embodiment, the spectral weighted feature F1_weighted [N,D1] and the dark current weighted feature F2_weighted [N,D2] are spliced ​​together to obtain the fused joint feature vector F_fused. The shape of the joint feature vector is [N,D1+D2]. Taking the above example, the shape of the joint feature vector is [N,192].

[0065] Furthermore, in this embodiment, step S5, the method for obtaining the defect identification result of the image sensor based on the joint feature vector, includes: S51, Construct a defect identification network.

[0066] Specifically, in this embodiment, the defect recognition network includes two fully connected layers, an output layer, and a softmax function. The first fully connected layer maps the joint feature vector to a higher dimension, for example, from 192 dimensions to 128 dimensions, and adds a BatchNorm normalization function and a ReLU activation function to prevent overfitting. The second fully connected layer maps the vector output by the first fully connected layer to 64 dimensions, and adds a BatchNorm normalization function, a ReLU activation function, and a Dropout function to ensure the accuracy of the results. In a specific embodiment, the Dropout rate in the Dropout function can be set to 30%. The output layer maps the vector output by the second fully connected layer to 2 dimensions, which represent the number of defect categories and the probability that the image sensor is normal or defective. The shape of the vector output by the output layer is [N, C], where C represents the number of defect categories.

[0067] It should be noted that in this embodiment, the mapping of the fully connected layer to a high-dimensional or low-dimensional dimension refers to the level of dimensionality. For example, 2-dimensional and 62-dimensional are low-dimensional, and 128-dimensional and 256-dimensional are high-dimensional. It does not refer to the relative level of dimensionality before and after mapping.

[0068] S52 uses a defect identification network to process the joint feature vector to obtain the probability corresponding to each defect.

[0069] Specifically, in this embodiment, the softmax function is applied to the result vector output by the defect identification network output layer to obtain the probability that each sample belongs to each category of defect.

[0070] S53, take the defect corresponding to the highest probability value as the defect identification result.

[0071] In practical applications, a large amount of labeled data can be used to train various network models (such as convolutional neural networks, recurrent neural networks, fully connected networks, and defect recognition networks) required for the technical implementation of this embodiment. The error between the network's predictions and the actual annotations is calculated using the backpropagation algorithm. The network parameters are then adjusted based on this error, enabling the network to continuously learn the mapping relationship between the spectral-dark current joint features and the defects in the image sensor. Furthermore, in practical applications, the network's performance can be periodically evaluated on a validation set. Training strategies and hyperparameters are adjusted based on the evaluation results. Training is stopped when the network's performance on the validation set no longer improves, resulting in the final trained network model.

[0072] This embodiment also provides an image sensor defect recognition system based on feature fusion, used to implement the image sensor defect recognition method based on feature fusion as described above, such as... Figure 2 As shown, the image sensor defect recognition system based on feature fusion includes: The data acquisition module is used to acquire quantum efficiency spectral data and dark current data from the image sensor; The feature extraction module is used to obtain the spectral features of the image sensor based on quantum efficiency spectral data and the dark current features of the image sensor based on dark current data. The feature fusion module is used to fuse spectral features and dark current features to obtain a joint feature vector; The defect identification module is used to obtain the defect identification results of the image sensor based on the joint feature vector.

[0073] The image sensor defect identification system based on feature fusion provided in this embodiment extracts and fuses features from quantum efficiency spectral data and dark current data through a feature extraction module and a feature fusion module. This results in a joint feature vector that establishes a correlation between spectral response anomalies and dark current parameter degradation. As a result, the defect identification module can quickly and accurately identify defects in the image sensor using the joint feature vector, thus solving the problem of how to efficiently and accurately identify image sensor defects.

[0074] Furthermore, this embodiment also provides an electronic device, including a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, it executes the image sensor defect recognition method based on feature fusion as described above.

[0075] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to mutually. In addition, different parts between embodiments can also be combined with each other, and this invention does not limit this.

[0076] This embodiment provides a feature fusion-based image sensor defect identification method, system, and electronic device, including: acquiring quantum efficiency spectral data and dark current data of an image sensor; acquiring spectral features of the image sensor based on the quantum efficiency spectral data; acquiring dark current features of the image sensor based on the dark current data; fusing the spectral features and dark current features to obtain a joint feature vector; and obtaining the defect identification result of the image sensor based on the joint feature vector. By extracting features from the quantum efficiency spectral data and dark current data and fusing them, the resulting joint feature vector establishes a correlation between spectral response anomalies and dark current parameter degradation. This enables rapid and accurate identification of image sensor defects using the joint feature vector, solving the problem of how to efficiently and accurately identify image sensor defects.

[0077] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A defect identification method for image sensors based on feature fusion, characterized in that, include: Acquire quantum efficiency spectral data and dark current data from image sensors; The spectral characteristics of the image sensor are obtained based on quantum efficiency spectral data; Obtain the dark current characteristics of the image sensor based on dark current data; Spectral features and dark current features are fused to obtain a joint feature vector; Defect identification results of image sensors are obtained based on joint feature vectors.

2. The image sensor defect identification method based on feature fusion according to claim 1, characterized in that, The method for acquiring quantum efficiency spectral data and dark current parameters of an image sensor includes: A wide-band light source was used to illuminate the image sensor, and the intensity of reflected light from the image sensor at different wavelengths was measured to obtain quantum efficiency spectral data. In a dark environment, the dark current value generated by the image sensor is measured to obtain dark current data.

3. The image sensor defect identification method based on feature fusion according to claim 1, characterized in that, The method for obtaining the spectral features of an image sensor based on quantum efficiency spectral data includes: Preprocessing of quantum efficiency spectral data; Construct a convolutional neural network; A convolutional neural network is used to process the preprocessed quantum efficiency spectral data to extract spectral features.

4. The image sensor defect identification method based on feature fusion according to claim 3, characterized in that, The method for preprocessing quantum efficiency spectral data includes: Quantum efficiency spectral data is compared with standard spectral sample data to remove abnormal spectral data; The quantum efficiency spectral data after removing anomalous spectral data is normalized.

5. The image sensor defect identification method based on feature fusion according to claim 3, characterized in that, The constructed convolutional neural network includes multiple convolutional layers with different kernel sizes to extract spectral feature maps at different scales.

6. The image sensor defect identification method based on feature fusion according to claim 5, characterized in that, Each convolutional layer is followed by an activation function, which is used to increase the non-linear expressive power of the convolutional neural network.

7. The image sensor defect identification method based on feature fusion according to claim 5, characterized in that, The constructed convolutional neural network further includes a pooling layer, which is used to downsample the spectral feature map output by the convolutional layer to obtain spectral features.

8. The image sensor defect identification method based on feature fusion according to claim 1, characterized in that, The method for obtaining dark current features of an image sensor based on dark current data includes: Noise reduction and filtering are applied to the dark current data. Construct a recurrent neural network; A recurrent neural network is used to process the dark current data after noise reduction and filtering in order to extract dark current features.

9. The image sensor defect identification method based on feature fusion according to claim 1, characterized in that, The method for fusing spectral features and dark current features to obtain a joint feature vector includes: Define the spectral feature vector based on spectral characteristics, and define the dark current feature vector based on dark current characteristics; Calculate the vector-level attention weights for spectral features and dark current features; Calculate the dimensional attention weights for spectral features and dark current features; Spectral features and dark current features are weighted using vector-level attention weights and dimension-level attention weights to obtain spectral weighted features and dark current weighted features; The spectral weighted features and dark current weighted features are concatenated to obtain a joint feature vector.

10. The image sensor defect identification method based on feature fusion according to claim 9, characterized in that, The method for calculating the vector-level attention weights of spectral features and dark current features includes: Assign the first scalar weights to the entire vector of spectral features and dark current features; The spectral eigenvector and the dark current eigenvector are concatenated to obtain the initial joint eigenvector; The initial joint feature vector is processed using a fully connected network to obtain two scalar vectors; The softmax function is used to process the two scalar vectors to obtain vector-level attention weights for spectral features and dark current features.

11. The image sensor defect identification method based on feature fusion according to claim 9, characterized in that, The method for calculating the dimensionality-level attention weights of spectral features and dark current features includes: A second scalar weight is assigned to each dimension of the spectral features and dark current features; Using a neural network, the spectral feature vector and the dark current feature vector are processed separately to obtain the spectral dimension vector and the dark current dimension vector respectively. The sigmoid function is used to process the spectral dimension vector and the dark current dimension vector respectively to obtain the dimensional attention weights of the spectral features and the dark current features.

12. The image sensor defect identification method based on feature fusion according to claim 9, characterized in that, The method for weighting spectral features and dark current features using vector-level attention weights and dimensional attention weights to obtain spectral weighted features and dark current weighted features includes: The spectral feature vector is first multiplied by its dimensional attention weights, and then multiplied by its vector-level attention weights to obtain the spectral weighted features. The dark current feature vector is first multiplied by its dimensional attention weight, and then multiplied by its vector-level attention weight to obtain the dark current weighted feature.

13. The image sensor defect identification method based on feature fusion according to claim 1, characterized in that, The method for obtaining the defect identification result of the image sensor based on the joint feature vector includes: Construct a defect identification network; The joint feature vector is processed using a defect identification network to obtain the probability corresponding to each defect; The defect with the highest probability value is taken as the defect identification result.

14. A feature fusion-based image sensor defect identification system, used to implement the feature fusion-based image sensor defect identification method as described in any one of claims 1 to 13, characterized in that, The feature fusion-based image sensor defect recognition system includes: The data acquisition module is used to acquire quantum efficiency spectral data and dark current data from the image sensor; The feature extraction module is used to obtain the spectral features of the image sensor based on quantum efficiency spectral data and the dark current features of the image sensor based on dark current data. The feature fusion module is used to fuse spectral features and dark current features to obtain a joint feature vector; The defect identification module is used to obtain the defect identification results of the image sensor based on the joint feature vector.

15. An electronic device, characterized in that, It includes a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, it performs the image sensor defect identification method based on feature fusion as described in any one of claims 1 to 13.

Images