Spectroscopic test method and system for lens assemblies

Abstract

The present application belongs to the technical field of lens detection, and provides a spectral testing method and system for a lens assembly, the method comprising: acquiring projection images of different lens assemblies, the projection images being images of an elliptical light spot and a plurality of fringes; evaluating the different lens assemblies, and according to the results, labeling the projection images as qualified or unqualified to form a training set, obtaining a first subset and a second subset based on pixel point recognition; inputting the first subset and the second subset into a lightweight convolutional neural network with depth separable convolution and an efficient convolutional neural network with an attention mechanism, collecting the outputs, and training to obtain a testing model; and when performing spectral testing on the lens assembly, acquiring the projection image thereof, inputting the projection image into the testing model, and obtaining a testing result; the present application can solve the problems of low efficiency and low accuracy in the manual testing of the light transmission performance of the lens in the prior art.

Description

Technical Field

[0001] This invention relates to the field of lens testing technology, and in particular to a spectral testing method and system for lens assemblies. Background Technology

[0002] In recent years, my country's robot market has developed rapidly, with household service robots such as cleaning robots entering the consumer market. Existing cleaning robots' infrared sensor front-collision modules use glass or ordinary plastic lenses. Glass lenses are heavy, fragile, and unsafe, while ordinary plastic lenses have poor scratch resistance and are easily deformed. Therefore, for the sensing module of cleaning robots, existing technology innovatively uses composite lens components. Composite lens components have high hardness and wear resistance, excellent light transmission, and anti-fouling effects. Compared to existing technologies using glass lenses, this allows for a thinner and lighter design with stronger impact resistance, reducing costs while maintaining performance.

[0003] Before these lens assemblies are installed on the target areas of the cleaning robot, they need to undergo performance testing, such as collision resistance and dirt resistance. In particular, to ensure the robot's accurate detection of obstacles and collision avoidance, spectral testing is required to test light transmission performance, especially the light transmission capability in specific wavelengths, to ensure that it does not affect the transmission of light emitted by the sensor. In existing technologies, testing the light transmission performance of lenses relies on manual observation of spectrometers, which is inefficient and prone to low accuracy due to human fatigue. Summary of the Invention

[0004] In view of the above technical problems, the present invention provides a spectral testing method and system for lens components to solve the problems of low efficiency and low accuracy in the prior art of manual testing of lens transmittance.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] According to one aspect of this disclosure, a spectral testing method for lens assemblies is disclosed, the method comprising:

[0007] The projection images of different lens assemblies are obtained. The projection images are images of an elliptical spot and several stripes projected onto an imaging screen after a parallel light beam of a preset wavelength is emitted by the optical system and passes through lens assemblies at different installation angles. Several stripes surround the elliptical spot. The projection images are obtained by an industrial camera capturing the elliptical spot and the stripes on the back of the imaging screen.

[0008] Different lens components are evaluated, and the projected images are labeled as qualified or unqualified according to the results to form a training set. Based on pixel point recognition, the first boundary of the elliptical light spot and the stripe of each projected image in the training set is located. The background outside the first boundary is cropped to obtain a first sub-image. The first sub-images are combined to form a first subset. Pixel point recognition is performed again on each first sub-image in the first subset to locate the second boundary of the elliptical light spot of the first sub-image. The image outside the second boundary is cropped to obtain a second sub-image. The second sub-images are combined to form a second subset.

[0009] The first subset and the second subset are input into a lightweight convolutional neural network with depthwise separable convolutions and an efficient convolutional neural network with an attention mechanism. The output data of the lightweight convolutional neural network and the efficient convolutional neural network are respectively input into two parallel first fully connected layers. The outputs of the first fully connected layers are pooled into a second fully connected layer to train multiple models. During training, the first subset is input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a first model. The second subset is input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a second model. The first subset is input into the lightweight convolutional neural network and the second subset is input into the efficient convolutional neural network to obtain a third model. The second subset is input into the lightweight convolutional neural network and the first subset is input into the efficient convolutional neural network to obtain a fourth model.

[0010] The sensitivity, false negative rate, false negative rate, and specificity of the first model, the second model, the third model, and the fourth model are calculated respectively to obtain the prediction accuracy of each model. The model with the highest accuracy is selected as the test model. When performing spectral testing on the lens assembly, its projection image is obtained and input into the test model to obtain the test results.

[0011] Furthermore, the projected images marked as unqualified include one of the following:

[0012] The stripe distribution is asymmetrical;

[0013] The stripe is connected to the elliptical light spot, and there is a sudden change in brightness at the connection point;

[0014] The stripe shape is irregular or a single stripe is discontinuous;

[0015] The shape of the elliptical light spot is irregular;

[0016] The brightness of the elliptical light spot is uneven;

[0017] The edges of the elliptical light spot are not clear;

[0018] Stray light exists around the elliptical light spot;

[0019] The elliptical light spot contains black dots.

[0020] Furthermore, locating the first boundary specifically includes:

[0021] Obtain the position coordinates of the pixel with the largest width, height, and brightness value in the projected image;

[0022] Based on the imaging size of the elliptical spot and the stripes in the projected image, and the position coordinates of the pixel with the largest brightness value, the coordinates of the first upper left corner and the first lower right corner are obtained, and the first boundary is determined. The imaging size is determined by the optical system.

[0023] The specific steps in locating the second boundary include:

[0024] Obtain the position coordinates of the pixel with the largest width, height, and brightness value in the first sub-image;

[0025] Based on the imaging size of the elliptical spot in the first sub-image and the position coordinates of the pixel with the largest brightness value, the coordinates of the second upper left corner and the second lower right corner are obtained, and the second boundary is determined.

[0026] Furthermore, after obtaining the first upper left corner coordinates and the first lower right corner coordinates, it is verified whether the first upper left corner coordinates and the first lower right corner coordinates exceed the boundary of the projected image. If they do, the upper left corner coordinates or the lower right corner coordinates of the projected image are used as the first upper left corner coordinates or the first lower right corner coordinates.

[0027] Furthermore, when locating the first boundary and the second boundary, one of the following algorithms is calculated: edge detection algorithm, contour detection algorithm, image segmentation algorithm, and Hough transform algorithm.

[0028] Furthermore, after obtaining the first subset and the second subset, the method further includes:

[0029] Filter the first subset and the second subset;

[0030] Data cleaning and data augmentation are performed on the first subset and the second subset to balance the number of samples labeled as qualified and unqualified in the first subset and the second subset.

[0031] Normalize and standardize the first subset and the second subset.

[0032] Furthermore, when calculating the prediction accuracy of each model, the specific steps include:

[0033] Record the spectral test results predicted by each model for the lens assembly and the actual situation, and calculate the sensitivity, the missed diagnosis rate, the false diagnosis rate, and the specificity. The sensitivity includes the case where there is a defect in the actual situation and the spectral test result is defective. The missed diagnosis rate includes the case where there is a defect in the actual situation but the spectral test result is not defective. The false diagnosis rate includes the case where there is no defect in the actual situation but the spectral test result is defective. The specificity includes the case where there is no defect in the actual situation and the spectral test result is also not defective.

[0034] Based on the sensitivity, the missed diagnosis rate, the misdiagnosis rate, and the specificity, the ratio of each model correctly identifying the actual situation as defective and the ratio of identifying it as normal are calculated respectively to obtain the sensitivity ratio and the specificity ratio.

[0035] Based on the sensitivity, the missed diagnosis rate, the false diagnosis rate, and the specificity, the recognition accuracy of each model is obtained. The recognition accuracy is the ratio between the actual defect and the defect in the spectral test result.

[0036] The prediction accuracy of each model is obtained based on the sensitivity ratio, the specificity ratio, and the recognition accuracy.

[0037] Furthermore, when training any one of the first model, the second model, the third model, and the fourth model individually, an original model is trained based on each sample in the first subset and / or the second subset, and the loss value of each sample in the first subset and / or the second subset is recorded during training. The median value is taken as a threshold, so that the samples in the first subset and / or the second subset are divided into two parts respectively. The different parts are used to retrain the model to obtain a first alternative model and a second alternative model. According to the weighted voting strategy, the parameter weights of the original model, the first alternative model, and the second alternative model are fused to obtain the final first model, the second model, the third model, or the fourth model.

[0038] According to another aspect of this disclosure, a spectral testing system for lens assemblies is disclosed, the system comprising:

[0039] The acquisition module is used to acquire projected images of different lens components. The projected image is an image of an elliptical spot and several stripes projected onto an imaging screen after a parallel light beam of a preset wavelength emitted by the optical system passes through lens components at different installation angles. Several stripes surround the elliptical spot. The projected image is obtained by an industrial camera acquiring the elliptical spot and the stripes from the back of the imaging screen.

[0040] The preprocessing module is used to evaluate different lens components. Based on the results, the projected images are labeled as qualified or unqualified to form a training set. Based on pixel recognition, the first boundary of the elliptical light spot and the stripe of each projected image in the training set is located. The background outside the first boundary is cropped to obtain a first sub-image. The first sub-images are combined to form a first subset. Pixel recognition is performed again on each first sub-image in the first subset to locate the second boundary of the elliptical light spot of the first sub-image. The image outside the second boundary is cropped to obtain a second sub-image. The second sub-images are combined to form a second subset.

[0041] The model generation module is used to input the first subset and the second subset into a lightweight convolutional neural network with depthwise separable convolutions and an efficient convolutional neural network with an attention mechanism. The output data of the lightweight convolutional neural network and the efficient convolutional neural network are respectively input into two parallel first fully connected layers. The outputs of the first fully connected layers are aggregated into a second fully connected layer to train multiple models. During training, the first subset is input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a first model; the second subset is input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a second model; the first subset is input into the lightweight convolutional neural network and the second subset is input into the efficient convolutional neural network to obtain a third model; and the second subset is input into the lightweight convolutional neural network and the first subset is input into the efficient convolutional neural network to obtain a fourth model.

[0042] The analysis module is used to calculate the sensitivity, false negative rate, false negative rate, and specificity of the first model, the second model, the third model, and the fourth model respectively, to obtain the prediction accuracy of each model, and select the model with the highest accuracy as the test model. When performing spectral testing on the lens assembly, its projection image is acquired and input into the test model to obtain the test results.

[0043] The technical solution disclosed herein has the following beneficial effects:

[0044] Based on the trained test model, the projected images corresponding to the lens components can be automatically acquired, and then the spectral test of the lens components can be analyzed efficiently and accurately to determine whether the test is qualified, which greatly improves the test efficiency and reduces the production cost.

[0045] By performing secondary cropping and branch training on the projected image, multiple models can be obtained, which can effectively improve the robustness of the model. Furthermore, this method can be applied to lens components of various sizes and with different light transmittance requirements. The final test model obtained has high real-time performance and high accuracy. Attached Figure Description

[0046] Figure 1 This is a flowchart of a spectral testing method for a lens assembly as described in an embodiment of this specification;

[0047] Figure 2 This is a structural block diagram of a spectral testing system for a lens assembly, as described in an embodiment of this specification.

[0048] Figure 3 This refers to the terminal device used in the embodiments of this specification to perform the spectral testing method for the lens assembly;

[0049] Figure 4 This is a computer-readable storage medium storing spectral testing methods for lens assemblies, as described in the embodiments of this specification. Detailed Implementation

[0050] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure may be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., may be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0051] Furthermore, the accompanying drawings are merely illustrative of this disclosure. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0052] like Figure 1 As shown in the embodiments of this specification, a spectral testing method for lens assemblies is provided. The execution subject of the method can be a computer or a server, and the method includes steps S101-104:

[0053] In step S101, projection images of different lens assemblies are acquired. The projection image is an image of an elliptical light spot and several stripes projected onto an imaging screen after a parallel light beam of a preset wavelength is emitted by the optical system and passes through lens assemblies at different installation angles. Several stripes surround the elliptical light spot. The projection image is obtained by an industrial camera capturing the elliptical light spot and the stripes on the back of the imaging screen.

[0054] The lens assembly comprises a lens body, multi-layer coating, and mounting structure. It can be mounted on an adjustable bracket. The optical system emits a parallel light beam with a wavelength identical to that allowed by the lens assembly. The beam passes through the lens assembly and is displayed on an imaging screen with good light transmittance. A filter of the same wavelength is placed on the lens of an industrial camera. Therefore, a projected image can be captured on the back of the imaging screen. Due to the filter settings of the industrial camera, the projected image only shows the projection of the beam of the same wavelength, with other elements appearing as a black background. The lens assembly can be a plane mirror, concave lens, or convex lens. Depending on the type of lens assembly, the distance between the lens and the light source needs to be adjusted to ensure the size of the projection on the imaging screen is suitable. The shape of the projected image is determined by the light source of the optical system and the multi-layer coating of the lenses. However, generally speaking, under interference effects, the lenses will cause elliptical spots and fringes in the projection. The elliptical spot is located at the center, and multiple fringes are regularly diffused and symmetrically distributed around the elliptical spot. When there are light transmission defects or surface defects in the lens assembly, the shape of the spot and fringes will be affected. Therefore, acquiring projected images of the lens assembly at the same focal length but different angles allows for the analysis of the light transmission and other defects of the lens assembly.

[0055] In step S102, different lens components are evaluated, and the projected images are marked as qualified or unqualified according to the results to form a training set. Based on pixel recognition, the first boundary of the elliptical spot and the stripe of each projected image in the training set is located. The background outside the first boundary is cropped to obtain a first sub-image. The first sub-images constitute a first subset. Pixel recognition is performed again on each first sub-image in the first subset to locate the second boundary of the elliptical spot of the first sub-image. The image outside the second boundary is cropped to obtain a second sub-image. The second sub-images constitute a second subset.

[0056] Among them, the projected images marked as unqualified are generally due to asymmetrical stripe distribution, stripes connecting to the elliptical light spot, irregular stripe shape or discontinuous individual stripes, irregular shape of the elliptical light spot, uneven brightness of the elliptical light spot, unclear edges of the elliptical light spot, stray light around the elliptical light spot, and black spots on the elliptical light spot. In particular, when stripes connect to the elliptical light spot, there is a sudden change in brightness at the connection point. The asymmetrical stripe distribution is caused by non-uniform refraction or reflection characteristics of the lens, resulting in the beam of light being deflected when passing through the lens, manifested as an uneven lens surface, uneven thickness, and misalignment of the optical axis. The presence of stray light around the connection between stripes and the elliptical light spot is caused by significant scattering or reflection of the beam of light when passing through the lens, which is caused by micro-scratches or dust on the lens surface or non-uniformity of the lens material. Irregular stripe shape or discontinuous individual stripes are due to uneven scattering of the beam of light when passing through the lens, corresponding to local defects on the lens surface, such as scratches, pits, or contamination. The irregular shape of the elliptical light spot is due to non-ideal refraction of the light beam as it passes through the lens, corresponding to insufficient lens manufacturing precision, uneven surface, or optical center misalignment. Uneven brightness of the elliptical light spot indicates uneven local transmittance of the lens. Indistinct edges of the elliptical light spot indicate an uneven lens surface with minor defects or contaminants. The presence of black spots in the elliptical light spot indicates that the light beam is blocked or scattered by the lens, suggesting the presence of small air bubbles, particles, or other impurities inside or on the surface of the lens. All of these factors affect the lens's transmittance, thus impacting the imaging quality and optical performance of the corresponding sensor. Therefore, after acquiring the projected image, the quality of the corresponding lens assembly can be determined by direct observation and marking. Alternatively, a traditional method can be used: the lens can be re-examined using a spectrometer, and the corresponding projected image of the lens assembly can be labeled as qualified or unqualified, thus obtaining a training set.

[0057] After obtaining the training set, to improve training speed, redundant background in the projected images needs to be removed. Due to the filter settings of industrial cameras, this background is generally black, allowing for rapid identification of projected light spots and stripes in the projected images. This identification can be based on one of the following algorithms: edge detection, contour detection, image segmentation, or Hough transform. After removing the redundant background, the resulting first and second sub-images can be used to train the model. The first and second sub-images have different focuses for detecting different defects. Based on the defect descriptions above, the first sub-image focuses on the lens's transmittance and surface defects, while the second sub-image focuses on the lens's overall transmittance.

[0058] In step S103, the first subset and the second subset are input into a lightweight convolutional neural network with depthwise separable convolutions and an efficient convolutional neural network with an attention mechanism. The output data of the lightweight convolutional neural network and the efficient convolutional neural network are respectively input into two parallel first fully connected layers. The outputs of the first fully connected layers are aggregated into a second fully connected layer to train multiple models. During training, the first subset is input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a first model. The second subset is input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a second model. The first subset is input into the lightweight convolutional neural network and the second subset is input into the efficient convolutional neural network to obtain a third model. The second subset is input into the lightweight convolutional neural network and the first subset is input into the efficient convolutional neural network to obtain a fourth model.

[0059] The lightweight convolutional neural network can be one of MobileNet, SqueezeNet, ShuffleNet, or NASNet-A, which has good computational efficiency and running speed. The efficient convolutional neural network can be one of ResNet, DenseNet, or EfficientNet, which has good performance. These two types form a dual-branch structure, which can train a model with both high accuracy and real-time performance. However, due to differences in optical characteristics, material properties, and detection directions, different lens components require different training methods for their optimal models. For example, some lenses focus on transmittance testing, so the second or third model might be better; others focus on material testing, so the first and fourth models might be better. Such selection is not absolute. Light spots and stripes can simultaneously reflect the transmittance, material properties, and surface defects of the lens. Therefore, it is necessary to train multiple models using various combinations, and then select the appropriate model for the current lens component based on actual performance. Additionally, the two parallel first fully connected layers and the second fully connected layer used for data aggregation can be multi-layered, with the specific number of layers depending on computational power.

[0060] In step S104, the sensitivity, false negative rate, false positive rate, and specificity of the first model, the second model, the third model, and the fourth model are calculated respectively to obtain the prediction accuracy of each model. The model with the highest accuracy is selected as the test model. When performing spectral testing on the lens assembly, its projection image is obtained and input into the test model to obtain the test result.

[0061] In this embodiment, multiple indicators are combined, meaning that different indicators are considered when evaluating the prediction accuracy of each category. Specifically, the calculation of the prediction accuracy of each model includes: recording the spectral test results predicted by each model for the lens assembly and the actual situation; calculating the sensitivity, the false negative rate, the false positive rate, and the specificity; the sensitivity includes cases where there is a defect in the actual situation and the spectral test result is defective; the false negative rate includes cases where there is a defect in the actual situation but the spectral test result is defect-free; the false positive rate includes cases where there is no defect in the actual situation but the spectral test result is defective; and the specificity includes cases where there is no defect in the actual situation and the spectral test result is defect-free. Based on the sensitivity, the false negative rate, the false positive rate, and the specificity, the ratio of each model correctly identifying the actual situation as defective and the ratio of identifying it as normal are calculated respectively to obtain the sensitivity ratio and the specificity ratio. Based on the sensitivity, the false negative rate, the false positive rate, and the specificity, the recognition accuracy of each model is obtained, where the recognition accuracy is the ratio between the defect in the actual situation and the defect in the spectral test result. Based on the sensitivity ratio, the specificity ratio, and the recognition accuracy, the prediction accuracy of each model is obtained. These percentages are all between 0 and 1. Any one of them can be used to evaluate the model's performance individually, or they can be used together for evaluation. The percentages are calculated as follows:

[0062] , To predict accuracy, TP represents sensitivity, TN represents specificity, FN represents false negative rate, and FP represents false positive rate. TP, TN, FN, and FP are the values ​​in the corresponding cases.

[0063] Where TPR is the sensitivity ratio;

[0064] Where TNR is the specificity ratio;

[0065] ,in, For recognition accuracy;

[0066] After obtaining the sensitivity ratio, the specificity ratio, and the recognition accuracy, the corresponding values ​​can be exponentialized using an activation function, weighted, and then summed. The summed result is the prediction accuracy. The weights assigned to the sensitivity ratio and the specificity ratio vary depending on the focus of the current lens component detection.

[0067] In another implementation, when locating the first and second boundaries, in addition to using the algorithm described above, calculations can also be based on experience, specifically including:

[0068] When locating the first boundary, the position coordinates of the pixel with the largest width, height, and brightness value in the projected image are obtained; based on the imaging size of the elliptical light spot and the stripes in the projected image, and the position coordinates of the pixel with the largest brightness value, the coordinates of the first upper left corner and the first lower right corner are obtained, and the first boundary is determined. The imaging size is determined by the optical system.

[0069] When locating the second boundary, the position coordinates of the pixel with the largest width, height, and brightness value in the first sub-image are obtained; based on the imaging size of the elliptical spot in the first sub-image and the position coordinates of the pixel with the largest brightness value, the coordinates of the second upper left corner and the second lower right corner are obtained, and the second boundary is determined.

[0070] Since the distances between the optical system, lens assembly, and imaging screen are known, the imaging size of lens assemblies of the same model is approximately the same. However, due to slight material differences in lens assemblies of the same model, the imaging position may deviate. Therefore, by calculating the pixel with the highest brightness value, the center point of the imaging can be determined. Based on the known imaging size, the first and second boundaries can be determined. In addition, the imaging size can be verified. Due to the overall integrity of the elliptical spot, other pixels with brightness in the elliptical spot are continuously surrounding the pixel with the highest brightness value. Therefore, the second boundary should enclose these pixels. Similarly, the first boundary encloses the stripes and the elliptical spot.

[0071] As a supplement, since the lens assembly is mounted on the bracket, the projection image may be offset when the lens assembly is replaced or rotated. Therefore, after obtaining the first upper left corner coordinate and the first lower right corner coordinate, it is verified whether the first upper left corner coordinate and the first lower right corner coordinate exceed the boundary of the projected image. If they exceed the boundary, the upper left corner coordinate or the lower right corner coordinate of the projected image is used as the first upper left corner coordinate or the first lower right corner coordinate.

[0072] In one embodiment, after obtaining the first subset and the second subset, the method further includes: filtering the first subset and the second subset; cleaning and augmenting the first subset and the second subset to balance the number of samples labeled as qualified and unqualified in the first subset and the second subset; and normalizing and standardizing the first subset and the second subset.

[0073] Among these methods, filtering, cleaning, and enhancement can improve training effectiveness. Normalization and standardization can reduce errors and increase the convergence speed of model training.

[0074] In one embodiment, because some images have relatively obvious features and a close mapping relationship between the light spot and the lens performance, the trained test model can predict the classification results well. However, some images have less obvious features and a poor mapping relationship between the light spot and the lens optical performance, making it difficult for the trained model to make correct predictions. Therefore, the training images in the first subset and the second subset are divided. When training any one of the first model, the second model, the third model, and the fourth model individually, an original model is trained based on each sample in the first subset and / or the second subset, and the loss value of each sample in the first subset and / or the second subset is recorded during training. The median value is used as a threshold, so that the samples in the first subset and / or the second subset are divided into two parts respectively. The different parts are used to retrain the model to obtain a first alternative model and a second alternative model. According to a weighted voting strategy, the parameter weights of the original model, the first alternative model, and the second alternative model are fused to obtain the final first model, the second model, the third model, or the fourth model. The weighted voting strategy can be to iterate through all weight combinations, obtain the prediction accuracy corresponding to each weight combination, and then select the weight combination with the highest prediction accuracy to fuse them to obtain the final model.

[0075] Based on the same content, such as Figure 2 As shown, an exemplary embodiment of this disclosure also provides a spectral testing system for lens assemblies, the system comprising:

[0076] The acquisition module 201 is used to acquire projected images of different lens assemblies. The projected image is an image of an elliptical spot and several stripes projected onto an imaging screen after a parallel light beam of a preset wavelength emitted by the optical system passes through lens assemblies at different installation angles. Several stripes surround the elliptical spot. The projected image is obtained by an industrial camera acquiring the elliptical spot and the stripes from the back of the imaging screen.

[0077] The preprocessing module 202 is used to evaluate different lens components, and according to the results, the projected image is marked as qualified or unqualified to form a training set. Based on pixel point recognition, the first boundary of the elliptical light spot and the stripe of each projected image in the training set is located, the background outside the first boundary is cropped to obtain a first sub-image, the first sub-images are combined to form a first subset, the pixel points of each first sub-image in the first subset are recognized again, the second boundary of the elliptical light spot of the first sub-image is located, the image outside the second boundary is cropped to obtain a second sub-image, and the second sub-images are combined to form a second subset.

[0078] The model generation module 203 is used to input the first subset and the second subset into a lightweight convolutional neural network with depthwise separable convolutions and an efficient convolutional neural network with an attention mechanism. The output data of the lightweight convolutional neural network and the efficient convolutional neural network are respectively input into two parallel first fully connected layers. The outputs of the first fully connected layers are aggregated into a second fully connected layer to train multiple models. During training, the first subset is input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a first model; the second subset is input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a second model; the first subset is input into the lightweight convolutional neural network and the second subset is input into the efficient convolutional neural network to obtain a third model; and the second subset is input into the lightweight convolutional neural network and the first subset is input into the efficient convolutional neural network to obtain a fourth model.

[0079] The analysis module 204 is used to calculate the sensitivity, false negative rate, false negative rate, and specificity of the first model, the second model, the third model, and the fourth model respectively, to obtain the prediction accuracy of each model, and select the model with the highest accuracy as the test model. When performing spectral testing on the lens assembly, its projection image is acquired and input into the test model to obtain the test results.

[0080] The specific details of each module in the above-mentioned device have been described in detail in the method section of the implementation. For any undisclosed details, please refer to the implementation content of the method section, and therefore will not be repeated here.

[0081] In the above implementation, based on the trained test model, the projected image corresponding to the lens assembly can be automatically acquired, and then the spectral test of the lens assembly can be analyzed efficiently and accurately to determine whether it is qualified, which greatly improves the test efficiency and reduces the production cost. By secondary cropping and branch training of the projected image to obtain multiple models, the robustness of the model can be effectively improved. Furthermore, a single method can be applied to lens assemblies of various sizes and with different light transmittance requirements. The final test model obtained has high real-time performance and high accuracy.

[0082] Following the same line of thought, embodiments of this specification also provide a spectral testing device for lens assemblies, such as... Figure 3 As shown.

[0083] The spectral testing equipment used for lens assemblies can be the terminal device or server provided in the above embodiments.

[0084] Spectroscopic testing equipment for lens assemblies can vary considerably depending on configuration and performance. It may include one or more processors 301 and memory 302, with memory 302 storing one or more application programs or data. Memory 302 may include readable media in the form of volatile memory cells, such as random access memory (RAM) and / or cache memory, and may further include read-only memory cells. The application programs stored in memory 302 may include one or more program modules (not shown in the figures), including but not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Furthermore, processor 301 may be configured to communicate with memory 302 and execute a series of computer-executable instructions stored in memory 302 on the spectral testing equipment for lens assemblies. The spectral testing equipment for lens assemblies may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more I / O interfaces (input / output interfaces) 305, one or more external devices 306 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), and may also communicate with one or more devices that enable users to interact with the device, and / or with any device that enables the device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed through I / O interfaces 305. Furthermore, the device can also communicate with one or more networks (e.g., local area networks (LANs)) via wired or wireless interfaces 304.

[0085] Specifically, in this embodiment, the spectral testing device for the lens assembly includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the spectral testing device for the lens assembly, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0086] The projection images of different lens assemblies are obtained. The projection images are images of an elliptical spot and several stripes projected onto an imaging screen after a parallel light beam of a preset wavelength is emitted by the optical system and passes through lens assemblies at different installation angles. Several stripes surround the elliptical spot. The projection images are obtained by an industrial camera capturing the elliptical spot and the stripes on the back of the imaging screen.

[0087] Different lens components are evaluated, and the projected images are labeled as qualified or unqualified according to the results to form a training set. Based on pixel point recognition, the first boundary of the elliptical light spot and the stripe of each projected image in the training set is located. The background outside the first boundary is cropped to obtain a first sub-image. The first sub-images are combined to form a first subset. Pixel point recognition is performed again on each first sub-image in the first subset to locate the second boundary of the elliptical light spot of the first sub-image. The image outside the second boundary is cropped to obtain a second sub-image. The second sub-images are combined to form a second subset.

[0088] The first subset and the second subset are input into a lightweight convolutional neural network with depthwise separable convolutions and an efficient convolutional neural network with an attention mechanism. The output data of the lightweight convolutional neural network and the efficient convolutional neural network are respectively input into two parallel first fully connected layers. The outputs of the first fully connected layers are pooled into a second fully connected layer to train multiple models. During training, the first subset is input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a first model. The second subset is input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a second model. The first subset is input into the lightweight convolutional neural network and the second subset is input into the efficient convolutional neural network to obtain a third model. The second subset is input into the lightweight convolutional neural network and the first subset is input into the efficient convolutional neural network to obtain a fourth model.

[0089] The sensitivity, false negative rate, false negative rate, and specificity of the first model, the second model, the third model, and the fourth model are calculated respectively to obtain the prediction accuracy of each model. The model with the highest accuracy is selected as the test model. When performing spectral testing on the lens assembly, its projection image is obtained and input into the test model to obtain the test results.

[0090] Based on the same idea, exemplary embodiments of this disclosure also provide a computer-readable storage medium storing a program product capable of implementing the methods described above. In some possible embodiments, various aspects of this disclosure can also be implemented as a program product including program code, which, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to the various exemplary embodiments of this disclosure.

[0091] refer to Figure 4As shown, a program product 400 for implementing the above-described method according to an exemplary embodiment of the present disclosure is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0092] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0093] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0094] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0095] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0096] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the method according to the exemplary embodiments of this disclosure.

[0097] Furthermore, the above figures are merely illustrative representations of the processes included in the methods according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0098] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to exemplary embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0099] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A method for spectral testing of a lens assembly, characterized in that, The method includes: The projection images of different lens assemblies are obtained. The projection images are images of an elliptical spot and several stripes projected onto an imaging screen after a parallel light beam of a preset wavelength is emitted by the optical system and passes through lens assemblies at different installation angles. Several stripes surround the elliptical spot. The projection images are obtained by an industrial camera capturing the elliptical spot and the stripes on the back of the imaging screen. Different lens components are evaluated, and the projected images are labeled as qualified or unqualified according to the results to form a training set. Based on pixel point recognition, the first boundary of the elliptical light spot and the stripe of each projected image in the training set is located. The background outside the first boundary is cropped to obtain a first sub-image. The first sub-images are combined to form a first subset. Pixel point recognition is performed again on each first sub-image in the first subset to locate the second boundary of the elliptical light spot of the first sub-image. The image outside the second boundary is cropped to obtain a second sub-image. The second sub-images are combined to form a second subset. The first subset and the second subset are input into a lightweight convolutional neural network with depthwise separable convolutions and an efficient convolutional neural network with an attention mechanism. The output data of the lightweight convolutional neural network and the efficient convolutional neural network are respectively input into two parallel first fully connected layers. The outputs of the first fully connected layers are pooled into a second fully connected layer to train multiple models. During training, the first subset is input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a first model. The second subset is input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a second model. The first subset is input into the lightweight convolutional neural network and the second subset is input into the efficient convolutional neural network to obtain a third model. The second subset is input into the lightweight convolutional neural network and the first subset is input into the efficient convolutional neural network to obtain a fourth model. The sensitivity, false negative rate, false negative rate, and specificity of the first model, the second model, the third model, and the fourth model are calculated respectively to obtain the prediction accuracy of each model. The model with the highest accuracy is selected as the test model. When performing spectral testing on the lens assembly, its projection image is obtained and input into the test model to obtain the test results.

2. A method for spectral testing of a lens assembly according to claim 1, characterized in that, The projected image is marked as unqualified, including one of the following: The stripe distribution is asymmetrical; The stripe is connected to the elliptical light spot, and there is a sudden change in brightness at the connection point; The stripe shape is irregular or a single stripe is discontinuous; The elliptical light spot has an irregular shape; The brightness of the elliptical light spot is uneven; The edges of the elliptical light spot are not clear; Stray light exists around the elliptical light spot; The elliptical light spot contains black dots.

3. The method of claim 1, wherein, When locating the first boundary, the specific steps include: obtaining the position coordinates of the pixel with the largest width, height, and brightness value in the projected image; obtaining the first upper-left corner coordinates and the first lower-right corner coordinates based on the imaging size of the elliptical spot and the stripes in the projected image, and the position coordinates of the pixel with the largest brightness value, to determine the first boundary, wherein the imaging size is determined by the optical system; When locating the second boundary, the specific steps include: obtaining the position coordinates of the pixel with the largest width, height, and brightness value in the first sub-image; obtaining the second upper-left corner coordinates and the second lower-right corner coordinates based on the imaging size of the elliptical spot and the position coordinates of the pixel with the largest brightness value in the first sub-image, to determine the second boundary.

4. The method of claim 3, wherein, After obtaining the first top-left corner coordinates and the first bottom-right corner coordinates, verify whether the first top-left corner coordinates and the first bottom-right corner coordinates exceed the boundary of the projected image. If they do, then use the top-left corner coordinates or the bottom-right corner coordinates of the projected image as the first top-left corner coordinates or the first bottom-right corner coordinates.

5. The method of claim 1, wherein, When locating the first boundary and the second boundary, one of the following algorithms is used: edge detection algorithm, contour detection algorithm, image segmentation algorithm, and Hough transform algorithm.

6. The method of claim 1, wherein, After obtaining the first subset and the second subset, the method further includes: Filter the first subset and the second subset; Data cleaning and data augmentation are performed on the first subset and the second subset to balance the number of samples labeled as qualified and unqualified in the first subset and the second subset. Normalize and standardize the first subset and the second subset.

7. The method of claim 1, wherein, The calculation of the prediction accuracy of each model specifically includes: Record the spectral test results predicted by each model for the lens assembly and the actual situation, and calculate the sensitivity, the missed diagnosis rate, the false diagnosis rate and the specificity. The sensitivity includes the case where there is a defect in the actual situation and the spectral test result is defective. The missed diagnosis rate includes the case where there is a defect in the actual situation but the spectral test result is not defective. The false diagnosis rate includes the case where there is no defect in the actual situation but the spectral test result is defective. The specificity includes the case where there is no defect in the actual situation and the spectral test result is also not defective. Based on the sensitivity, the missed diagnosis rate, the misdiagnosis rate, and the specificity, the ratio of each model correctly identifying the actual situation as defective and the ratio of identifying it as normal are calculated respectively to obtain the sensitivity ratio and the specificity ratio. Based on the sensitivity, the missed diagnosis rate, the false diagnosis rate, and the specificity, the recognition accuracy of each model is obtained, where the recognition accuracy is the ratio between the actual defect and the defect in the spectral test result. The prediction accuracy of each model is obtained based on the sensitivity ratio, the specificity ratio, and the recognition accuracy.

8. The method of claim 1, wherein, In the training of any one of the first model, the second model, the third model and the fourth model, an original model is trained based on each sample in the first subset or / and the second subset, and the loss value of each sample in the first subset or / and the second subset during the training is recorded, and the median is taken as a threshold, so that the samples in the first subset or / and the second subset are divided into two parts respectively, and different parts are used to retrain the model to obtain a first substitute model and a second substitute model, and according to a weighted voting strategy, the parameter weights of the original model, the first substitute model and the second substitute model are fused to obtain the final first model, second model, third model or fourth model.

9. A spectral testing system for a lens assembly, characterized by, The system includes: an acquisition module for acquiring projected images of different lens assemblies, wherein the projected image is an image of an elliptical spot and several stripes projected onto an imaging screen after a parallel beam of a preset wavelength emitted by an optical system passes through lens assemblies at different installation angles, and the several stripes surround the elliptical spot; the projected image is obtained by an industrial camera acquiring the elliptical spot and the stripes from the back of the imaging screen; and a preprocessing module for evaluating different lens assemblies, labeling the projected images as qualified or unqualified based on the results, forming a training set, locating the first boundary of the elliptical spot and the stripes of each projected image in the training set based on pixel point recognition, cropping the background outside the first boundary to obtain a first sub-image, forming a first subset of the first sub-images, performing pixel point recognition again on each of the first sub-images in the first subset, locating the second boundary of the elliptical spot of the first sub-image, cropping the image outside the second boundary to obtain a second sub-image, and forming a second subset of the second sub-images. The model generation module is used to input the first subset and the second subset into a lightweight convolutional neural network with depthwise separable convolutions and an efficient convolutional neural network with an attention mechanism. The output data of the lightweight convolutional neural network and the efficient convolutional neural network are respectively input into two parallel first fully connected layers. The outputs of the first fully connected layers are aggregated into a second fully connected layer to train multiple models. During training, the first subset is input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a first model; the second subset is input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a second model; the first subset is input into the lightweight convolutional neural network and the second subset is input into the efficient convolutional neural network to obtain a third model; and the second subset is input into the lightweight convolutional neural network and the first subset is input into the efficient convolutional neural network to obtain a fourth model. The analysis module is used to calculate the sensitivity, false negative rate, false negative rate, and specificity of the first model, the second model, the third model, and the fourth model respectively, to obtain the prediction accuracy of each model, and select the model with the highest accuracy as the test model. When performing spectral testing on the lens assembly, its projection image is acquired and input into the test model to obtain the test results.

Images