Spectrum testing method and system for lens assembly

Abstract

The invention belongs to the technical field of lens detection, and provides a spectrum test method and system for lens assemblies, and the method comprises the steps: obtaining projection images of different lens assemblies, the projection images being images of elliptical light spots and a plurality of stripes; evaluating different lens components, marking the projection image as qualified or unqualified according to a result, forming a training set, and obtaining a first subset and a second subset based on pixel point identification; inputting the first subset and the second subset into a lightweight convolutional neural network with depth separable convolution and a high-efficiency convolutional neural network with an attention mechanism, collecting and training outputs to obtain a test model, acquiring a projection image of the lens assembly when spectrum testing is performed on the lens assembly, and inputting the projection image into the test model to obtain a test result; according to the invention, the problems of low efficiency and low accuracy of manual testing of the light transmittance performance of the lens in the prior art can be solved.

Description

Technical Field

[0001] The present invention relates to the field of lens detection technology, and in particular to a spectrum testing method and system for lens components. Background Art

[0002] In recent years, my country's robot market has developed rapidly, and household service robots such as cleaning robots have entered the consumer market. The original cleaning robot infrared sensing front collision module components use glass lenses or ordinary plastic lenses. Glass lenses are thick, fragile and unsafe, and ordinary plastic lenses have poor scratch resistance and are easy to deform. Therefore, for the sensing module of the cleaning robot, the existing technology innovatively uses a composite lens assembly, which has high hardness and wear resistance, excellent light transmission and anti-fouling effects. Compared with the existing technology using glass lenses, it can be thinner and lighter, with stronger drop resistance, and reduces costs while ensuring performance.

[0003] Before these lens components are installed on the target parts of the cleaning robot, they need to be tested for performance, such as anti-collision and anti-fouling. In particular, in order to ensure that the robot can accurately detect obstacles and avoid collisions, it is necessary to test the spectrum to test the light transmittance. In particular, the light transmittance of a specific band is required to ensure that it will not affect the transmission of the sensor's emitted light. In the prior art, testing the light transmittance of the lens relies on manual observation of the spectrometer, which is inefficient and has low accuracy due to manual fatigue. Summary of the invention

[0004] In view of the above technical problems, the present invention provides a spectral testing method and system for lens components to solve the problems of low efficiency and accuracy in the manual light transmittance testing of lenses in the prior art.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to one aspect of the present disclosure, a spectral testing method for a lens assembly is disclosed, the method comprising: Acquire projection images of different lens components, wherein the projection images are images of an elliptical light spot and a plurality of stripes projected on an imaging screen after a parallel light beam of a preset wavelength is emitted by an optical system and passes through lens components installed at different angles, wherein a plurality of stripes surround the elliptical light spot, and the projection images are obtained by an industrial camera collecting the elliptical light spot and the stripes on the back of the imaging screen; Evaluate different lens components, and mark the projection images as qualified or unqualified according to the results to form a training set, locate the first boundary of the elliptical light spot and the stripes of each projection image in the training set based on pixel recognition, cut out the background outside the first boundary to obtain a first sub-image, and form a first subset with the first sub-images, perform pixel recognition again on each of the first sub-images in the first subset, locate the second boundary of the elliptical light spot of the first sub-image, cut out the image outside the second boundary to obtain a second sub-image, and form a second subset with the second sub-images; Input the first subset and the second subset into a lightweight convolutional neural network with depthwise separable convolution and an efficient convolutional neural network with an attention mechanism, input the output data of the lightweight convolutional neural network and the efficient convolutional neural network into two parallel first fully connected layers, aggregate the output of the first fully connected layer into the second fully connected layer, and train to obtain multiple models, wherein, during training, the first subset is input into the lightweight convolutional neural network and the efficient convolutional neural network respectively to obtain a first model, the second subset is input into the lightweight convolutional neural network and the efficient convolutional neural network respectively to obtain a second model, the first subset is input into the lightweight convolutional neural network, the second subset is input into the efficient convolutional neural network to obtain a third model, the second subset is input into the lightweight convolutional neural network, the first subset is input into the efficient convolutional neural network to obtain a fourth model; The sensitivity, missed diagnosis rate, misdiagnosis rate and specificity of the first model, the second model, the third model and the fourth model are calculated respectively to obtain the prediction accuracy of each model, and the model with the highest accuracy is selected as the test model. When performing spectral testing on the lens assembly, its projection image is obtained and input into the test model to obtain the test result.

[0007] Furthermore, the projection image marked as unqualified includes one of the following situations: The stripes are distributed asymmetrically; The stripes are connected to the elliptical light spot, and there is a sudden change in brightness at the connection; The stripes are irregular in shape or a single stripe is discontinuous; The shape of the elliptical light spot is irregular; The brightness of the elliptical light spot is uneven; The edge of the elliptical light spot is unclear; There is stray light around the elliptical light spot; The elliptical light spot has a black spot.

[0008] Furthermore, when locating the first boundary, the method specifically includes: Obtaining the position coordinates of the pixel point with the largest width, height and brightness value of the projected image; According to the imaging size of the elliptical spot and the stripes of the projected image and the position coordinates of the pixel point with the maximum brightness value, a first upper left corner coordinate and a first lower right corner coordinate are obtained to determine the first boundary, wherein the imaging size is determined by the optical system; When locating the second boundary, the method specifically includes: Obtaining the position coordinates of the pixel point with the largest width, height and brightness value in the first sub-image; According to the imaging size of the elliptical light spot of the first sub-image and the position coordinates of the pixel point with the maximum brightness value, the second upper left corner coordinates and the second lower right corner coordinates are obtained to determine the second boundary.

[0009] Furthermore, after obtaining the first upper-left corner coordinate and the first lower-right corner coordinate, verify whether the first upper-left corner coordinate and the first lower-right corner coordinate exceed the boundary of the projected image; if so, use the upper-left corner coordinate or the lower-right corner coordinate of the projected image as the first upper-left corner coordinate or the first lower-right corner.

[0010] Furthermore, when locating the first boundary and the second boundary, one of an edge detection algorithm, a contour detection algorithm, an image segmentation algorithm, and a Hough transform algorithm is calculated.

[0011] Further, after obtaining the first subset and the second subset, the method further includes: filtering the first subset and the second subset; Performing data cleaning and data enhancement on the first subset and the second subset so that the number of samples marked as qualified and unqualified in the first subset and the second subset is balanced; The first subset and the second subset are normalized and standardized.

[0012] Furthermore, when calculating the prediction accuracy of each model, it specifically includes: Record the spectral test results predicted by each model for the lens component and the actual situation, and calculate the sensitivity, the missed diagnosis rate, the misdiagnosis rate, and the specificity. The sensitivity includes the situation where there is a defect in the actual situation and the spectral test result is a defect. The missed diagnosis rate includes the situation where there is a defect in the actual situation but the spectral test result is not defective. The misdiagnosis rate includes the situation where the actual situation is not defective but the spectral test result is defective. The specificity includes the situation where the actual situation is not defective and the spectral test result is not defective at the same time. Based on the sensitivity, the missed diagnosis rate, the misdiagnosis rate, and the specificity, the ratio of each model correctly identifying the actual situation as defective and the ratio of each model correctly identifying the actual situation as normal are calculated to obtain the sensitivity ratio and the specificity ratio; Based on the sensitivity, the missed diagnosis rate, the misdiagnosis rate, and the specificity, the recognition accuracy of each model is obtained, and the recognition accuracy is the ratio between the defects in the actual situation and the defects in the spectral test results; Based on the sensitivity ratio, the specificity ratio and the recognition accuracy, the prediction accuracy of each model is obtained.

[0013] Furthermore, when training any one of the first model, the second model, the third model and the fourth model separately, an original model is trained based on each sample in the first subset or / and the second subset, and the loss value of each sample in the first subset or / and the second subset during training is recorded, and the median is taken as the threshold, so that the samples in the first subset or / and the second subset are respectively divided into two parts, and different parts are used to retrain the model to obtain a first substitute model and a second substitute model, and according to a weighted voting strategy, the parameter weights of the original model, the first substitute model and the second substitute model are fused to obtain the final first model, the second model, the third model or the fourth model.

[0014] According to another aspect of the present disclosure, a spectral testing system for a lens assembly is disclosed, the system comprising: A collection module, used to obtain projection images of different lens components, wherein the projection images are images of an elliptical light spot and a plurality of stripes projected on an imaging screen after a parallel light beam of a preset wavelength is emitted by an optical system and passes through lens components installed at different angles, wherein a plurality of stripes surround the elliptical light spot, and the projection images are obtained by an industrial camera collecting the elliptical light spot and the stripes on the back of the imaging screen; a preprocessing module, for evaluating different lens components, marking the projection images as qualified or unqualified according to the results, forming a training set, locating the first boundary of the elliptical light spot and the stripes of each projection image in the training set based on pixel recognition, cutting out the background outside the first boundary to obtain a first sub-image, forming a first subset with the first sub-images, performing pixel recognition again on each of the first sub-images in the first subset, locating the second boundary of the elliptical light spot of the first sub-image, cutting out the image outside the second boundary to obtain a second sub-image, and forming a second subset with the second sub-images; A model generation module, used for inputting the first subset and the second subset into a lightweight convolutional neural network with depthwise separable convolution and an efficient convolutional neural network with an attention mechanism, respectively inputting the output data of the lightweight convolutional neural network and the efficient convolutional neural network into two parallel first fully connected layers, and collecting the output of the first fully connected layer into the second fully connected layer, and training to obtain multiple models, wherein, during training, the first subset is respectively input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a first model, the second subset is respectively input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a second model, the first subset is input into the lightweight convolutional neural network, the second subset is input into the efficient convolutional neural network to obtain a third model, the second subset is input into the lightweight convolutional neural network, and the first subset is input into the efficient convolutional neural network to obtain a fourth model; The analysis module is used to respectively calculate the sensitivity, missed diagnosis rate, misdiagnosis rate and specificity of the first model, the second model, the third model and the fourth model, obtain the prediction accuracy of each model, select the model with the highest accuracy as the test model, and when performing spectral testing on the lens component, obtain its projection image, input it into the test model, and obtain the test result.

[0015] The technical solution disclosed in this disclosure has the following beneficial effects: Based on the trained test model, the projection image corresponding to the lens component can be automatically collected, and then the spectrum test of the lens component can be analyzed efficiently and accurately to determine whether it is qualified, which greatly improves the test efficiency and reduces the production cost. By obtaining multiple models through secondary cropping and branch training of the projected image, the robustness of the model can be effectively improved. A method can be used for lens components of various sizes and different light transmittance requirements. The final test model obtained has high real-time performance and high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of a spectral testing method for a lens assembly in an embodiment of this specification; Figure 2 is a structural block diagram of a spectral testing system for a lens assembly in an embodiment of this specification; Figure 3 A terminal device for executing the spectral testing method for lens components in the embodiments of this specification; Figure 4 The computer-readable storage medium in the embodiment of the present specification stores a spectral testing method for a lens assembly. DETAILED DESCRIPTION

[0017] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0018] In addition, the accompanying drawings are only schematic illustrations of the present disclosure. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0019] like Figure 1 As shown, the embodiment of this specification provides a spectral testing method for a lens assembly, the execution subject of the method can be a computer or a server, and the method includes steps S101-104: In step S101, projection images of different lens assemblies are obtained, wherein the projection images are images of an elliptical light spot and a plurality of stripes projected onto an imaging screen after the optical system emits a parallel light beam of a preset wavelength through lens assemblies installed at different angles, and the plurality of stripes surround the elliptical light spot, and the projection images are obtained by an industrial camera capturing the elliptical light spot and the stripes on the back of the imaging screen.

[0020] Among them, the lens assembly can be a lens body, a multi-layer coating and a mounting structure. The lens assembly can be installed on an angle-adjustable bracket, and the optical system emits a parallel light beam, the wavelength of which is the same as the wavelength allowed to pass through the lens assembly. The light beam passes through the lens assembly and is displayed on the imaging screen. The imaging screen has good light transmittance. A filter of the same wavelength band is set on the lens of the industrial camera. Therefore, a projected image can be taken on the back of the imaging screen. Due to the setting of the filter of the industrial camera, the projected image only has the projection of the light beam of the same wavelength band, and the other pictures are black backgrounds. The lens assembly can be a plane mirror, a concave lens, or a convex lens. When the type of the lens assembly is different, the distance between the lens and the light source needs to be adjusted so that the size of the projection falling on the imaging screen is suitable. The shape of the projected image is determined by the light source of the optical system and the multi-layer coating of the lens. However, in general, under the interference effect, the lens will cause elliptical spots and stripes to appear in the projection. The elliptical spot is located in the center, and multiple stripes are regularly diffused and symmetrically distributed around the elliptical spot. When there are light transmittance defects and surface defects in the lens component, the shape of the spot and stripes will be affected. Therefore, collecting the projection images of the lens component at the same focal length but different angles can analyze the light transmittance and other defects of the lens component.

[0021] In step S102, different lens components are evaluated, and the projection images are marked as qualified or unqualified based on the results to form a training set. Based on pixel recognition, the first boundaries of the elliptical light spot and the stripes of each projection image in the training set are located, and the background outside the first boundary is cropped to obtain a first sub-image. The first sub-images constitute a first subset. Pixel recognition is performed again on each of the first sub-images in the first subset, and the second boundary of the elliptical light spot of the first sub-image is located. The image outside the second boundary is cropped to obtain a second sub-image, and the second sub-images constitute a second subset.

[0022] Among them, the projection images marked as unqualified are generally due to asymmetric stripe distribution, stripes connected to the elliptical spot, irregular stripe shape or discontinuous individual stripes, irregular shape of the elliptical spot, uneven brightness of the elliptical spot, unclear edge of the elliptical spot, stray light around the elliptical spot, black spots in the elliptical spot, and especially, when the stripes are connected to the elliptical spot, there is a sudden change in brightness at the connection. The reason for the asymmetric stripe distribution is that the lens has non-uniform refraction or reflection characteristics, which causes the light beam to deflect when passing through the lens, which is manifested as uneven surface of the lens, uneven thickness, and misaligned optical axis. The reason for the stripes connecting and stray light around the elliptical spot is that the light beam is significantly scattered or reflected when passing through the lens, which is caused by tiny scratches or dust on the surface of the lens and uneven lens material. The irregular stripe shape or discontinuous individual stripes is due to the uneven scattering of the light beam when passing through the lens, corresponding to local defects on the lens surface, such as scratches, pits or pollution. The irregular shape of the elliptical spot is due to the non-ideal refraction of the light beam when passing through the lens, which corresponds to the insufficient processing accuracy of the lens, uneven surface or offset of the optical center. The uneven brightness of the elliptical spot indicates that the local transmittance of the lens is uneven. The unclear edge of the elliptical spot indicates that the surface of the lens is uneven, with tiny flaws or contaminants. The presence of black spots in the elliptical spot is because the light beam is blocked or scattered when passing through the lens, which indicates that there are small bubbles, particles or other impurities inside or on the surface of the lens. All of the above will affect the transmittance of the lens and the imaging quality and optical performance of the corresponding sensor. Therefore, after obtaining the projection image, the corresponding lens component can be judged whether it is qualified by directly observing the projection image, and then marked. It can also be reviewed by the traditional method of spectrometer, and then the projection image corresponding to the lens component is marked as qualified or unqualified to obtain a training set.

[0023] After obtaining the training set, in order to increase the training speed, it is necessary to remove the redundant background in the projection image. Due to the setting of the filter of the industrial camera, the background is generally black, so the projection spot and stripes in the projection image can be quickly identified. The identification can be based on one of the edge detection algorithm, contour detection algorithm, image segmentation algorithm, and Hough transform algorithm. After removing the redundant background, the first sub-image and the second sub-image obtained can be used to train the model, wherein the first sub-image and the second sub-image have different emphases on different defect detection. Combined with the above defect description, it can be known that the first sub-image focuses on the transmittance and surface defects of the lens, and the second sub-image focuses on the general transmittance of the lens.

[0024] In step S103, the first subset and the second subset are input into a lightweight convolutional neural network with depthwise separable convolution and an efficient convolutional neural network with an attention mechanism, the output data of the lightweight convolutional neural network and the efficient convolutional neural network are respectively input into two parallel first fully connected layers, the output of the first fully connected layer is aggregated into the second fully connected layer, and multiple models are trained to obtain, wherein, during training, the first subset is respectively input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a first model, the second subset is respectively input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a second model, the first subset is input into the lightweight convolutional neural network, the second subset is input into the efficient convolutional neural network to obtain a third model, the second subset is input into the lightweight convolutional neural network, the first subset is input into the efficient convolutional neural network to obtain a fourth model.

[0025] Among them, the lightweight convolutional neural network can be one of MobileNet, SqueezeNet, ShuffleNet, and NASNet-A, which has good computing efficiency and running speed, and the efficient convolutional neural network can be one of ResNet, DenseNet, and EfficientNet, which has good performance. The two constitute a dual-branch structure, and a model with both high accuracy and real-time performance can be trained. However, due to the differences in optical properties, material properties, and detection directions of lens components of different models, the training methods of the best models are also different. For example, some lenses focus on transmittance testing, and the second model and the third model may be selected as the best. Some lenses focus on material testing, and the first model and the fourth model may be selected as the best. Such a choice is not absolute. The light spots and stripes can reflect the transmittance, material, and surface defects of the lens at the same time. Therefore, it is necessary to train multiple models with multiple combinations, and then select the model suitable for the current model of the lens component according to the actual performance. In addition, the two parallel first fully connected layers and the second fully connected layer for collecting data can be multi-layered, and the specific number of layers depends on the computing power.

[0026] In step S104, the sensitivity, missed diagnosis rate, misdiagnosis rate and specificity of the first model, the second model, the third model and the fourth model are calculated respectively to obtain the prediction accuracy of each model, and the model with the highest accuracy is selected as the test model. When performing a spectral test on the lens assembly, its projection image is obtained and input into the test model to obtain the test result.

[0027] Among them, this embodiment adopts a combination of multiple indicators, that is, different indicators are considered when evaluating the prediction accuracy of each category. Specifically, when calculating the prediction accuracy of each model, it specifically includes: recording the spectral test results predicted by each model for the lens component and the actual situation, calculating the sensitivity, the missed diagnosis rate, the misdiagnosis rate, and the specificity, the sensitivity includes the situation where there is a defect in the actual situation and the spectral test result is a defect, the missed diagnosis rate includes the situation where there is a defect in the actual situation but the spectral test result is not defective, the misdiagnosis rate includes the situation where the actual situation is not defective but the spectral test result is defective, and the specificity includes the situation where the actual situation is not defective and the spectral test result is not defective at the same time; based on the sensitivity, the missed diagnosis rate, the misdiagnosis rate, and the specificity, respectively calculate the ratio of each model correctly identifying the actual situation as defective and the ratio of identifying it as normal, and obtain the sensitivity ratio and specificity ratio; based on the sensitivity, the missed diagnosis rate, the misdiagnosis rate, and the specificity, obtain the recognition accuracy of each model, the recognition accuracy is the ratio between the defects in the actual situation and the defects in the spectral test result; based on the sensitivity ratio, the specificity ratio and the recognition accuracy, obtain the prediction accuracy of each model. These ratios are percentage values ​​between 0 and 1. Any of them can be used to evaluate the performance of the model alone, or they can be used together to evaluate the performance. The ratio calculation is as follows: , is the prediction accuracy, TP is the sensitivity, TN is the specificity, FN is the missed diagnosis rate, FP is the misdiagnosis rate, and TP, TN, FN, and FP are the values ​​in the corresponding situations; , where TPR is the sensitivity ratio; , where TNR is the specificity ratio; ,in, For recognition accuracy; After obtaining the sensitivity ratio, the specificity ratio and the recognition accuracy, the corresponding values ​​can be indexed using an activation function, and then weighted and added. The addition result is the prediction accuracy. Among them, the weights of the sensitivity ratio and the specificity ratio are assigned differently according to the different focus points of the current lens component detection.

[0028] In another embodiment, when locating the first boundary and the second boundary, in addition to using the above algorithm, calculations may also be performed based on experience, specifically including: When locating the first boundary, the width, height, and position coordinates of the pixel point with the largest brightness value of the projected image are obtained; the first upper left corner coordinates and the first lower right corner coordinates are obtained according to the imaging size of the elliptical light spot and the stripes of the projected image and the position coordinates of the pixel point with the largest brightness value, and the first boundary is determined, and the imaging size is determined by the optical system; When locating the second boundary, the width, height, and position coordinates of the pixel with the largest brightness value of the first sub-image are obtained; based on the imaging size of the elliptical spot of the first sub-image and the position coordinates of the pixel with the largest brightness value, the second upper left corner coordinates and the second lower right corner coordinates are obtained to determine the second boundary.

[0029] Among them, since the distances between the optical system, the lens assembly, and the imaging screen are known, the imaging sizes of the same model of lens assemblies are approximately the same, but due to slight material differences in the same model of lens assemblies, the imaging position may deviate. Therefore, by calculating the pixel with the largest brightness value, the center point of the imaging can be determined, and then based on the known imaging size, the first boundary and the second boundary can be determined; in addition, the imaging size can be verified, that is, due to the integrity of the elliptical light spot, the other pixels with brightness of the elliptical light spot are continuously surrounding the pixel with the maximum brightness value, so the second boundary should enclose these pixels; similarly, the first boundary encloses the stripes and the elliptical light spot.

[0030] In addition, since the lens assembly is installed on the bracket, the projection imaging may be offset when the lens assembly is replaced or rotated. Therefore, after obtaining the first upper left corner coordinate and the first lower right corner coordinate, verify whether the first upper left corner coordinate and the first lower right corner coordinate exceed the boundary of the projected image. If so, the upper left corner coordinate or the lower right corner coordinate of the projected image is used as the first upper left corner coordinate or the first lower right corner.

[0031] In one embodiment, after obtaining the first subset and the second subset, the method further includes: filtering the first subset and the second subset; performing data cleaning and data enhancement on the first subset and the second subset so that the number of samples marked as qualified and unqualified in the first subset and the second subset is balanced; and normalizing and standardizing the first subset and the second subset.

[0032] Among them, filtering, cleaning and enhancement can enhance the training effect. Normalization and standardization can reduce errors and increase the convergence speed of model training.

[0033] In one embodiment, since the features of some images are obvious and the mapping relationship between the light spot and the lens performance is close, the trained test model can well predict the classification results. However, there are also some images with unclear features and poor mapping relationship between the light spot and the lens optical performance. In this case, it is difficult for the trained model to make a correct prediction. Therefore, the training images in the first subset and the second subset are divided. When training any one of the first model, the second model, the third model and the fourth model separately, the original model is trained based on each sample in the first subset or / and the second subset, and the loss value of each sample in the first subset or / and the second subset during training is recorded, and the median is taken as the threshold, so that the samples in the first subset or / and the second subset are respectively divided into two parts, and the different parts are used to retrain the model to obtain the first substitute model and the second substitute model. According to the weighted voting strategy, the parameter weights of the original model, the first substitute model and the second substitute model are fused to obtain the final first model, the second model, the third model or the fourth model. The weighted voting strategy can be to traverse all weight combinations, obtain the prediction accuracy corresponding to each weight combination, and then select the weight combination with the highest prediction accuracy, and fuse them to obtain the final model.

[0034] Based on the same content, such as Figure 2 As shown, an exemplary embodiment of the present disclosure also provides a spectral testing system for a lens assembly, the system comprising: The acquisition module 201 is used to acquire projection images of different lens components, wherein the projection images are images of an elliptical light spot and a plurality of stripes projected on an imaging screen after a parallel light beam of a preset wavelength is emitted by an optical system and passes through lens components installed at different angles, wherein a plurality of stripes surround the elliptical light spot, and the projection images are acquired by an industrial camera acquiring the elliptical light spot and the stripes on the back of the imaging screen; The preprocessing module 202 is used to evaluate different lens components, mark the projection image as qualified or unqualified according to the result, form a training set, locate the first boundary of the elliptical spot and the stripe of each projection image in the training set based on pixel recognition, cut out the background outside the first boundary to obtain a first sub-image, form a first subset with the first sub-images, perform pixel recognition on each of the first sub-images in the first subset again, locate the second boundary of the elliptical spot of the first sub-image, cut out the image outside the second boundary to obtain a second sub-image, and form a second subset with the second sub-images; The model generation module 203 is used to input the first subset and the second subset into a lightweight convolutional neural network with a depthwise separable convolution and an efficient convolutional neural network with an attention mechanism, input the output data of the lightweight convolutional neural network and the efficient convolutional neural network into two parallel first fully connected layers, aggregate the output of the first fully connected layer into the second fully connected layer, and train to obtain multiple models, wherein, during training, the first subset is input into the lightweight convolutional neural network and the efficient convolutional neural network respectively to obtain a first model, the second subset is input into the lightweight convolutional neural network and the efficient convolutional neural network respectively to obtain a second model, the first subset is input into the lightweight convolutional neural network, the second subset is input into the efficient convolutional neural network to obtain a third model, the second subset is input into the lightweight convolutional neural network, the first subset is input into the efficient convolutional neural network to obtain a fourth model; The analysis module 204 is used to respectively calculate the sensitivity, missed diagnosis rate, misdiagnosis rate and specificity of the first model, the second model, the third model and the fourth model, obtain the prediction accuracy of each model, select the model with the highest accuracy as the test model, and when performing spectral testing on the lens component, obtain its projection image and input it into the test model to obtain the test result.

[0035] The specific details of each module in the above device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0036] In the above-mentioned implementation, based on the trained test model, the projection image corresponding to the lens assembly can be automatically collected, and then the spectral test of the lens assembly can be efficiently and accurately analyzed to determine whether it is qualified, which greatly improves the test efficiency and reduces the production cost. By obtaining multiple models through secondary cropping and branch training of the projection image, the robustness of the model can be effectively improved, and a method can be used for lens assemblies of various sizes and different light transmittance requirements, and the final test model obtained has high real-time performance and high accuracy.

[0037] Based on the same idea, the embodiment of this specification also provides a spectrum testing device for lens components, such as Figure 3 shown.

[0038] The spectral testing device for the lens assembly may be a terminal device or a server provided in the above embodiments.

[0039] The spectral testing device for lens components may have relatively large differences due to different configurations or performances, and may include one or more processors 301 and memory 302, and the memory 302 may store one or more storage applications or data. Among them, the memory 502 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) and / or a cache memory unit, and may further include a read-only storage unit. The application stored in the memory 302 may include one or more program modules (not shown in the figure), such program modules include but are not limited to: an operating system, one or more application programs, other program modules and program data, each of these examples or some combination may include the implementation of a network environment. Furthermore, the processor 301 can be configured to communicate with the memory 302 to execute a series of computer executable instructions in the memory 302 on the spectral testing device for lens components. The spectral testing device for lens assemblies may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more I / O interfaces (input and output interfaces) 305, one or more external devices 306 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), and may also communicate with one or more devices that enable a user to interact with the device, and / or communicate with any device that enables the device to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through the I / O interface 305. Furthermore, the device may also communicate with one or more networks (e.g., a local area network (LAN)) through the wired or wireless interface 304.

[0040] Specifically in this embodiment, the spectral testing device for lens components includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions for the spectral testing device for lens components, and the one or more programs are configured to be executed by one or more processors, including computer executable instructions for performing the following: Acquire projection images of different lens components, wherein the projection images are images of an elliptical light spot and a plurality of stripes projected on an imaging screen after a parallel light beam of a preset wavelength is emitted by an optical system and passes through lens components installed at different angles, wherein a plurality of stripes surround the elliptical light spot, and the projection images are obtained by an industrial camera collecting the elliptical light spot and the stripes on the back of the imaging screen; Evaluate different lens components, and mark the projection images as qualified or unqualified according to the results to form a training set, locate the first boundary of the elliptical light spot and the stripes of each projection image in the training set based on pixel recognition, cut out the background outside the first boundary to obtain a first sub-image, and form a first subset with the first sub-images, perform pixel recognition again on each of the first sub-images in the first subset, locate the second boundary of the elliptical light spot of the first sub-image, cut out the image outside the second boundary to obtain a second sub-image, and form a second subset with the second sub-images; Input the first subset and the second subset into a lightweight convolutional neural network with depthwise separable convolution and an efficient convolutional neural network with an attention mechanism, input the output data of the lightweight convolutional neural network and the efficient convolutional neural network into two parallel first fully connected layers, aggregate the output of the first fully connected layer into the second fully connected layer, and train to obtain multiple models, wherein, during training, the first subset is input into the lightweight convolutional neural network and the efficient convolutional neural network respectively to obtain a first model, the second subset is input into the lightweight convolutional neural network and the efficient convolutional neural network respectively to obtain a second model, the first subset is input into the lightweight convolutional neural network, the second subset is input into the efficient convolutional neural network to obtain a third model, the second subset is input into the lightweight convolutional neural network, the first subset is input into the efficient convolutional neural network to obtain a fourth model; The sensitivity, missed diagnosis rate, misdiagnosis rate and specificity of the first model, the second model, the third model and the fourth model are calculated respectively to obtain the prediction accuracy of each model, and the model with the highest accuracy is selected as the test model. When performing spectral testing on the lens assembly, its projection image is obtained and input into the test model to obtain the test result.

[0041] Based on the same idea, the exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0042] refer to Figure 4As shown, a program product 400 for implementing the above method according to an exemplary embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0043] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0044] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0045] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0046] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0047] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the exemplary implementation of the present disclosure.

[0048] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiment of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0049] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0050] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A spectral testing method for lens components, characterized in that: The method comprises: Acquire projection images of different lens components, wherein the projection images are images of an elliptical light spot and a plurality of stripes projected on an imaging screen after a parallel light beam of a preset wavelength is emitted by an optical system and passes through lens components installed at different angles, wherein a plurality of stripes surround the elliptical light spot, and the projection images are obtained by an industrial camera collecting the elliptical light spot and the stripes on the back of the imaging screen; Evaluate different lens components, and mark the projection images as qualified or unqualified according to the results to form a training set, locate the first boundary of the elliptical light spot and the stripes of each projection image in the training set based on pixel recognition, cut out the background outside the first boundary to obtain a first sub-image, and form a first subset with the first sub-images, perform pixel recognition again on each of the first sub-images in the first subset, locate the second boundary of the elliptical light spot of the first sub-image, cut out the image outside the second boundary to obtain a second sub-image, and form a second subset with the second sub-images; Input the first subset and the second subset into a lightweight convolutional neural network with depthwise separable convolution and an efficient convolutional neural network with an attention mechanism, input the output data of the lightweight convolutional neural network and the efficient convolutional neural network into two parallel first fully connected layers, aggregate the output of the first fully connected layer into the second fully connected layer, and train to obtain multiple models, wherein, during training, the first subset is input into the lightweight convolutional neural network and the efficient convolutional neural network respectively to obtain a first model, the second subset is input into the lightweight convolutional neural network and the efficient convolutional neural network respectively to obtain a second model, the first subset is input into the lightweight convolutional neural network, the second subset is input into the efficient convolutional neural network to obtain a third model, the second subset is input into the lightweight convolutional neural network, the first subset is input into the efficient convolutional neural network to obtain a fourth model; The sensitivity, missed diagnosis rate, misdiagnosis rate and specificity of the first model, the second model, the third model and the fourth model are calculated respectively to obtain the prediction accuracy of each model, and the model with the highest accuracy is selected as the test model. When performing spectral testing on the lens assembly, its projection image is obtained and input into the test model to obtain the test result.

2. A spectrum testing method for lens components according to claim 1, characterized in that: The projection images marked as unqualified include one of the following situations: The stripes are distributed asymmetrically; The stripes are connected to the elliptical light spot, and there is a sudden change in brightness at the connection; The stripes are irregular in shape or a single stripe is discontinuous; The shape of the elliptical light spot is irregular; The brightness of the elliptical light spot is uneven; The edge of the elliptical light spot is unclear; There is stray light around the elliptical light spot; The elliptical light spot has a black spot.

3. A spectrum testing method for lens components according to claim 1, characterized in that: When locating the first boundary, the method specifically includes: Obtaining the position coordinates of the pixel point with the largest width, height and brightness value of the projected image; According to the imaging size of the elliptical spot and the stripes of the projected image and the position coordinates of the pixel point with the maximum brightness value, a first upper left corner coordinate and a first lower right corner coordinate are obtained to determine the first boundary, wherein the imaging size is determined by the optical system; When locating the second boundary, the method specifically includes: Obtaining the position coordinates of the pixel point with the largest width, height and brightness value in the first sub-image; According to the imaging size of the elliptical light spot of the first sub-image and the position coordinates of the pixel point with the maximum brightness value, the second upper left corner coordinates and the second lower right corner coordinates are obtained to determine the second boundary.

4. A spectrum testing method for lens components according to claim 3, characterized in that: After obtaining the first upper-left corner coordinate and the first lower-right corner coordinate, verify whether the first upper-left corner coordinate and the first lower-right corner coordinate exceed the boundary of the projected image; if so, use the upper-left corner coordinate or the lower-right corner coordinate of the projected image as the first upper-left corner coordinate or the first lower-right corner.

5. The spectral testing method for lens components according to claim 1, characterized in that: When locating the first boundary and the second boundary, one of an edge detection algorithm, a contour detection algorithm, an image segmentation algorithm, and a Hough transform algorithm is calculated.

6. A spectrum testing method for lens components according to claim 1, characterized in that: After acquiring the first subset and the second subset, the method further includes: filtering the first subset and the second subset; Performing data cleaning and data enhancement on the first subset and the second subset so that the number of samples marked as qualified and unqualified in the first subset and the second subset is balanced; The first subset and the second subset are normalized and standardized.

7. A spectrum testing method for lens components according to claim 1, characterized in that: When calculating the prediction accuracy of each model, it specifically includes: Record the spectral test results predicted by each model for the lens component and the actual situation, and calculate the sensitivity, the missed diagnosis rate, the misdiagnosis rate, and the specificity. The sensitivity includes the situation where there is a defect in the actual situation and the spectral test result is a defect. The missed diagnosis rate includes the situation where there is a defect in the actual situation but the spectral test result is not defective. The misdiagnosis rate includes the situation where the actual situation is not defective but the spectral test result is defective. The specificity includes the situation where the actual situation is not defective and the spectral test result is not defective at the same time. Based on the sensitivity, the missed diagnosis rate, the misdiagnosis rate, and the specificity, the ratio of each model correctly identifying the actual situation as defective and the ratio of each model correctly identifying the actual situation as normal are calculated to obtain the sensitivity ratio and the specificity ratio; Based on the sensitivity, the missed diagnosis rate, the misdiagnosis rate, and the specificity, the recognition accuracy of each model is obtained, and the recognition accuracy is the ratio between the defects in the actual situation and the defects in the spectral test results; Based on the sensitivity ratio, the specificity ratio and the recognition accuracy, the prediction accuracy of each model is obtained.

8. A spectrum testing method for lens components according to claim 1, characterized in that: When training any one of the first model, the second model, the third model and the fourth model separately, an original model is trained based on each sample in the first subset or / and the second subset, and the loss value of each sample in the first subset or / and the second subset during training is recorded, and the median is taken as the threshold, so that the samples in the first subset or / and the second subset are respectively divided into two parts, and different parts are used to retrain the model to obtain a first substitute model and a second substitute model, and according to a weighted voting strategy, the parameter weights of the original model, the first substitute model and the second substitute model are fused to obtain the final first model, the second model, the third model or the fourth model.

9. A spectral testing system for lens components, characterized in that: The system comprises: A collection module, used to obtain projection images of different lens components, wherein the projection images are images of an elliptical light spot and a plurality of stripes projected on an imaging screen after a parallel light beam of a preset wavelength is emitted by an optical system and passes through lens components installed at different angles, wherein a plurality of stripes surround the elliptical light spot, and the projection images are obtained by an industrial camera collecting the elliptical light spot and the stripes on the back of the imaging screen; a preprocessing module, for evaluating different lens components, marking the projection images as qualified or unqualified according to the results, forming a training set, locating the first boundary of the elliptical light spot and the stripes of each projection image in the training set based on pixel recognition, cutting out the background outside the first boundary to obtain a first sub-image, forming a first subset with the first sub-images, performing pixel recognition again on each of the first sub-images in the first subset, locating the second boundary of the elliptical light spot of the first sub-image, cutting out the image outside the second boundary to obtain a second sub-image, and forming a second subset with the second sub-images; A model generation module, used for inputting the first subset and the second subset into a lightweight convolutional neural network with depthwise separable convolution and an efficient convolutional neural network with an attention mechanism, respectively inputting the output data of the lightweight convolutional neural network and the efficient convolutional neural network into two parallel first fully connected layers, and collecting the output of the first fully connected layer into the second fully connected layer, and training to obtain multiple models, wherein, during training, the first subset is respectively input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a first model, the second subset is respectively input into the lightweight convolutional neural network and the efficient convolutional neural network to obtain a second model, the first subset is input into the lightweight convolutional neural network, the second subset is input into the efficient convolutional neural network to obtain a third model, the second subset is input into the lightweight convolutional neural network, and the first subset is input into the efficient convolutional neural network to obtain a fourth model; The analysis module is used to respectively calculate the sensitivity, missed diagnosis rate, misdiagnosis rate and specificity of the first model, the second model, the third model and the fourth model, obtain the prediction accuracy of each model, select the model with the highest accuracy as the test model, and when performing spectral testing on the lens component, obtain its projection image, input it into the test model, and obtain the test result.

Images