Quartz crystal oscillator accurate groove entering detection method and system based on deep learning

Abstract

The invention relates to the technical field of crossing of electronic component manufacturing, machine vision and deep learning, and provides a quartz crystal oscillator accurate groove entering detection method and system based on deep learning, and the method comprises the steps: collecting crystal oscillator images of different detection stations, and carrying out the preprocessing of the crystal oscillator images, and obtaining an input image; constructing a deep learning network model; performing feature extraction on the input image through a backbone network to obtain a high-dimensional feature map; performing deep and shallow layer feature fusion and further feature extraction on the high-dimensional feature map through a neck network to obtain a prediction input feature map; performing feature extraction on the predicted input feature map through a head network to obtain a predicted feature map; performing post-processing on the prediction feature map to obtain an optimal prediction frame; and crystal oscillator groove entering detection is carried out according to the optimal prediction frame. According to the invention, whether the crystal oscillator in the crystal oscillator image enters the groove or not is accurately detected and identified, and the method has the advantages of high detection precision, strong generalization ability, high detection efficiency, low cost and the like.

Description

Technical Field

[0001] The present invention relates to the interdisciplinary field of electronic component manufacturing, machine vision, and deep learning, and in particular to a method and system for accurately detecting the entry of a quartz crystal oscillator into a slot based on deep learning. Background Art

[0002] As a core component in electronic devices, the manufacturing process of quartz crystal oscillators involves multiple complex steps. Precise placement of the crystal oscillator within the tray slot is crucial for ensuring the accuracy and continuity of subsequent production processes. However, traditional methods for inspecting the placement of crystal oscillators in the slot typically rely on manual visual inspection, mechanical sensor detection, or simple image processing methods. Due to the complexity and variability of the crystal oscillator's state on the tray, these methods are susceptible to subjective factors, suffer from low detection efficiency, low detection accuracy, poor robustness and generalization, and are unable to adapt to large-scale production. These methods struggle to meet the high-precision and high-efficiency requirements of modern electronic component manufacturing.

[0003] Traditional quartz crystal slot detection methods have poor adaptability to various light source conditions, such as different types of ring light, coaxial light, and strip light, as well as complex background changes such as damaged trays, dirt, scratches, characters, image noise interference, and different crystal specifications, sizes, and postures, resulting in suboptimal detection results. Furthermore, when the crystal surface exhibits significant changes in color or shadows due to lighting or other factors, or when two or more crystals overlap, detection accuracy can be significantly reduced or even impossible. Detection results are significantly affected by feature changes and exhibit poor robustness. Therefore, traditional image processing algorithms are only suitable for situations with distinct features and simple backgrounds, and recognition effectiveness depends largely on manually defined feature templates.

[0004] Each crystal oscillator and tray of different specifications requires a set of templates and a separate inspection process. This makes it difficult to effectively identify the various states of crystal oscillators of different specifications on different types of trays (such as multiple crystals stacked together, crystals on side beams, crystals with slight overlap, etc.). It has poor adaptability and cannot identify crystals outside the positioning slot. Furthermore, because the characteristics of different slots are relatively similar, the characteristics of crystals in different states vary greatly, leading to problems such as poor template matching accuracy. Existing traditional methods and deep learning methods have low detection accuracy for crystal oscillators of smaller sizes and stacked crystals in a large field of view, and have a high rate of missed detection. Some crystal oscillators in different states have small differences between classes and large differences within classes. It is difficult to achieve high recognition accuracy using traditional methods alone. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides a method and system for accurately detecting the entry of a quartz crystal oscillator into a slot based on deep learning. This method achieves accurate detection and identification of whether the crystal oscillator is slotted in a crystal oscillator image. The method has the advantages of high detection accuracy, strong generalization capability, high detection efficiency, and low cost. It has important practical value in solving the problem of accurately detecting and identifying the entry of a quartz crystal oscillator into a slot, and can be widely applied in the field of electronic component manufacturing to improve production efficiency and product quality.

[0006] The present invention provides a method for accurately detecting the entry of a quartz crystal oscillator into a slot based on deep learning, comprising: S1: Collect crystal oscillator images at different inspection stations and pre-process the crystal oscillator images to obtain input images; S2: Constructing a deep learning network model, wherein the deep learning network model includes a Backbone network, a Neck network, and a Head network; S3: Extract features from the input image through the Backbone network to obtain a high-dimensional feature map; perform deep and shallow feature fusion and further feature extraction on the high-dimensional feature map through the Neck network to obtain a predicted input feature map; perform feature extraction on the predicted input feature map through the Head network to obtain a predicted feature map; S4: The predicted feature map generates a predicted box by decoding the predicted box; the predicted box is redundantly eliminated by NMS non-maximum suppression to obtain the optimal predicted box; S5: Perform crystal oscillator slot detection based on the optimal prediction frame.

[0007] Furthermore, the preprocessing includes geometric transformation, optical transformation, noise addition and labeling information, and the labeling information includes target category, target upper left corner coordinates, target lower right corner coordinates, and the target categories include empty slot and no material OK_1, completely entered slot OK_2, the slot is within the slot range and the slot needs to be adjusted NG_1, and the slot is outside the slot range or the material is reversed and the defective product box needs to be sucked out and thrown away NG_2.

[0008] Furthermore, the Backbone network includes a CBS module, a CSSPX module containing 1 RES Block, a RES-SE module, a CSSPX module containing 2 RES Blocks, a RES-SE module, a CSSPX module containing 4 RES Blocks, a CSSPX module containing 6 RES Blocks, and a CSSPX module containing 8 RES Blocks stacked in sequence.

[0009] Furthermore, the CBS module includes a convolutional layer, a BN layer and a SiLu activation layer stacked in sequence; The RES Block includes a CBS module with a convolution kernel of 1×1, a CBS module with a convolution kernel of 3×3, and a CBS module with a convolution kernel of 1×1, which are stacked in sequence; The working process of the CSSPX module is as follows: The CSSPX module input image is downsampled through a CBS module to obtain the first feature map. The first feature map is passed through a CBS module and multiple ES Block modules, and then through a CBS module for feature extraction to obtain the second feature map. The first feature map is passed through a CBS module for feature extraction to obtain the third feature map. The second feature map and the third feature map are spliced, and then feature extraction is performed through a CBS module to obtain the CSSPX module output feature image.

[0010] Furthermore, the RES-SE module is an EDC-SE module added to the RES Block; The working process of the RES-SE module is as follows: The output feature map of the CBS module with a convolution kernel of 3×3 is passed through the EDC-SE module to obtain a weight vector. The weight vector is weighted channel by channel to the output feature map of the original CBS module with a convolution kernel of 3×3 through multiplication operation. The features of the CBS module with a convolution kernel of 3×3 are then added to the input features of the original RES-SE module to obtain the output features of the RES-SE module. The input features of the EDC-SE module are divided into two paths, one of which is obtained by stacking the global average pooling layer and the CBS module with a convolution kernel of 1×1 in sequence to obtain the fourth feature, and the other is obtained by stacking the global maximum pooling layer and the CBS module with a convolution kernel of 1×1 in sequence to obtain the fifth feature. The fourth and fifth features are spliced ​​and then connected with the activation function to obtain the output features of the EDC-SE module.

[0011] Furthermore, the Neck network uses a deep and shallow feature fusion network structure based on FPN and PAN, and adds SPPF module and CSSPX module at the front. The SPPF module is an improved spatial pyramid pooling layer. The working process of the SPPF module is as follows: The input feature map of the SPPF module first passes through the CBS module with a convolution kernel of 1x1 to extract preliminary features, and then passes through three maximum pooling layers with different pooling kernel sizes for downsampling. The four feature maps obtained are spliced ​​by channel, and the spliced ​​feature maps are then passed through the CBS module with a convolution kernel of 1x1 to extract features to obtain the output feature map of the SPPF module.

[0012] Furthermore, the predicted input feature map outputs predicted feature maps of three sizes through the Head network.

[0013] Furthermore, step S5 includes: S51: Convert the pixel coordinates of the optimal prediction boxes of the three sizes into actual coordinates in the camera coordinate system; S52: Divide the crystal oscillator image in the current detection field of view into regions centered at the center point of the detection region; S53: sorting according to the actual coordinates and encoding the crystal oscillators falling within the area; S54: traverse all actual coordinates and encode all crystal oscillators falling within the area; S55: Check the crystal oscillator slot status according to the marked information and coding.

[0014] Furthermore, in step S53, in, For crystal oscillator encoding, The number of regions divided into for the current image, for The coordinates of the lower left corner of the region, for The coordinates of the lower left corner of the region, The coordinates of any pixel point on the source image.

[0015] The present invention also provides a quartz crystal oscillator precise slot detection system based on deep learning, which is used to execute the above-mentioned quartz crystal oscillator precise slot detection method based on deep learning, including: An image acquisition module, which collects crystal oscillator images of different detection stations and preprocesses the crystal oscillator images to obtain input images; A model construction module, wherein the model construction module constructs a deep learning network model, wherein the deep learning network model includes a Backbone network, a Neck network, and a Head network; A feature extraction module extracts features from the input image through a Backbone network to obtain a high-dimensional feature map; performs deep and shallow feature fusion and further feature extraction on the high-dimensional feature map through a Neck network to obtain a predicted input feature map; and extracts features from the predicted input feature map through a Head network to obtain a predicted feature map; The optimization module is used to predict the feature map and generate a prediction frame by decoding the prediction frame; the prediction frame is redundantly eliminated by NMS non-maximum suppression to obtain the optimal prediction frame; A detection module performs crystal oscillator slot detection according to the optimal prediction frame.

[0016] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: The present invention uses a deep learning algorithm to extract and classify features of crystal oscillator images, ensuring that crystal oscillators in various states under different lines, different fields of view, different sizes, different positions and distances can be accurately detected, significantly improving the accuracy of slot detection, reducing the false detection rate and missed detection rate, helping to reduce production costs and improve the company's market competitiveness.

[0017] The present invention realizes real-time detection of whether multiple crystal oscillators are accurately inserted into the slots under various fields of view, triggers corresponding feedback mechanisms based on the detection results, realizes automatic control and adjustment, reduces manual intervention and downtime, reduces production costs, and improves the automation level and production efficiency of the production line.

[0018] The present invention can adapt to crystal oscillators of different models, sizes, fields of view, and front and back sides in various production environments. It has strong robustness, generalization, and scalability. By continuously updating and optimizing the model, it can adapt to new situations and new demands in the production environment and ensure long-term stable operation.

[0019] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 This is a flow chart of a method for accurately detecting the entry of a quartz crystal oscillator into a slot based on deep learning provided by the present invention.

[0022] Figure 2 It is a structural diagram of the CBS module and RES Block module provided by the present invention.

[0023] Figure 3 It is a structural diagram of the CSSPX module provided by the present invention.

[0024] Figure 4 It is a structural diagram of the EDC-SE module provided by the present invention.

[0025] Figure 5 It is a structural diagram of the RES-SE module provided by the present invention.

[0026] Figure 6 It is a structural diagram of the SPPF module provided by the present invention.

[0027] Figure 7 It is a structural diagram of the deep learning network model provided by the present invention.

[0028] Figure 8 This is a structural diagram of a quartz crystal oscillator precise slot detection system based on deep learning provided by the present invention.

[0029] Reference numerals: 101. Image acquisition module; 102. Model building module; 103. Feature extraction module; 104. Optimization module; 105. Detection module. DETAILED DESCRIPTION

[0030] To make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Obviously, the embodiments described are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0031] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0032] The following combination Figures 1 to 8 The present invention describes a method and system for accurately detecting the entry of a quartz crystal oscillator into a slot based on deep learning.

[0033] like Figure 1 As shown, a method for accurately detecting the entry of a quartz crystal into a slot based on deep learning includes: S1: Collect crystal oscillator images at different inspection stations and pre-process the crystal oscillator images to obtain input images; In some specific embodiments of the present invention, an industrial camera with a resolution of 2448*2048 is used in combination with a FA lens (Factory Automation Lens) to collect image data of crystal oscillators in different states at different inspection stations on the line, including crystal oscillators of different quantities and specifications, different types of trays, different fields of view, empty slots, fully entered slots, not fully entered slots, completely out of slots, overlapping to varying degrees, crystal oscillators scattered at various positions on the tray, and various stacked materials. The inspection stations include glue dispensing slot inspection, fine-tuning slot inspection, and welding slot inspection.

[0034] The collected crystal oscillator images are preprocessed and annotated with high quality using annotation tools, and are marked into two OK categories and two NG categories, namely OK_1: empty slot with no material, OK_2: completely in the slot, NG_1: out of the slot, within the slot range, the slot needs to be adjusted, NG_2: out of the slot, outside the slot range, the material is reversed, and the defective box needs to be sucked out.

[0035] The preprocessing includes geometric transformation, optical transformation, noise addition and labeling information. The labeling information includes target category, target upper left corner coordinates, target lower right corner coordinates. The target categories include empty slot and no material OK_1, completely entered slot OK_2, the slot is within the slot range and the slot needs to be adjusted NG_1, and the slot is outside the slot range or the reverse material needs to be sucked up and the defective product box needs to be thrown away NG_2. The above information will be saved in a txt file.

[0036] The pixel value is divided by 255 and converted from 0-255 to 0-1 for normalization; finally, the image is scaled proportionally, translated and centered, and the excess parts are filled.

[0037] In some specific embodiments of the present invention, the input image is scaled proportionally, and the image resolution is converted from 2448*2048 to a set input image size of 608*608. Then, the scaled image is translated and centered, and the excess part of the scaled image is padded with zeros. The entire process can be completed by an affine transformation matrix, and the calculation expression is: in,( , ) is the coordinate of any pixel point on the target image, is the width of the source image, is the height of the source image, is the width of the target image, is the height of the target image, is the scaling factor, which is the minimum value of the width and height ratio between the target image and the source image, and (x, y) is the coordinate of any pixel point on the source image.

[0038] Through this matrix, we can get the coordinates of any pixel value in the source image corresponding to the position in the target image.

[0039] S2: Construct a deep learning network model, which includes a Backbone network, a Neck network, and a Head network; the deep learning network model is as follows: Figure 7 shown.

[0040] The backbone network includes a CBS module stacked in sequence, a CSSPX module containing one RES Block, a RES-SE module, a CSSPX module containing two RES Blocks, a RES-SE module, a CSSPX module containing four RES Blocks, a CSSPX module containing six RES Blocks, and a CSSPX module containing eight RES Blocks.

[0041] like Figure 2 As shown, the CBS module includes a convolutional layer (Conv), a BN layer, and a SiLu activation layer stacked in sequence; RES Block includes a CBS module with a convolution kernel of 1×1, a CBS module with a convolution kernel of 3×3, and a CBS module with a convolution kernel of 1×1, which are stacked in sequence; like Figure 3 As shown, the working process of the CSSPX module is: The CSSPX module input image is downsampled through the CBS module to obtain the first feature map. The first feature map is passed through a CBS module and multiple ES Block modules, and then through a CBS module for feature extraction to obtain the second feature map. The first feature map is passed through a CBS module for feature extraction to obtain the third feature map. The second feature map and the third feature map are concatenated (Concat), and then feature extracted through a CBS module to obtain the CSSPX module output feature image.

[0042] like Figure 5 As shown, the RES-SE module is to add the EDC-SE module to the RES Block; The working process of the RES-SE module is: The output feature map of the CBS module with a convolution kernel of 3×3 is passed through the EDC-SE module to obtain a weight vector, which is then weighted channel by channel to the output feature map of the original CBS module with a convolution kernel of 3×3 through a multiplication operation (MCL). The features of the CBS module with a convolution kernel of 3×3 are then added (Add) to the input features of the original RES-SE module to obtain the output features of the RES-SE module.

[0043] like Figure 4 As shown in the figure, the input features of the EDC-SE module are divided into two paths. One path is obtained by stacking the global average pooling layer (Global AvgPool) and the CBS module with a convolution kernel of 1×1 in sequence to obtain the fourth feature, and the other path is obtained by stacking the global maximum pooling layer (Global MaxPool) and the CBS module with a convolution kernel of 1×1 in sequence to obtain the fifth feature. The fourth and fifth features are spliced ​​together and then connected with the activation function (Sigmoid) to obtain the output features of the EDC-SE module.

[0044] The Neck network uses a deep and shallow feature fusion network structure based on FPN and PAN, and adds SPPF module and CSSPX module at the front.

[0045] like Figure 6 As shown in Figure 2, the SPPF module is an improved spatial pyramid pooling layer. The working process of the SPPF module is as follows: The input feature map of the SPPF module first passes through the CBS module with a convolution kernel of 1x1 to extract preliminary features, and then passes through three maximum pooling layers with different pooling kernel sizes for downsampling. The four feature maps obtained are concatenated (Concat) by channel, and the concatenated feature maps are then passed through the CBS module with a convolution kernel of 1x1 to extract features to obtain the output feature map of the SPPF module.

[0046] Figure 6 In the figure, the three maximum pooling layers with different pooling kernel sizes are: MaxPool2d (k13, s1, p2), MaxPool2d (k9, s1, p2), and MaxPool2d (k5, s1, p2). Among them, k13 is the size of the pooling kernel of 13×13, and the corresponding receptive field is RF: 13×13. K9 is the size of the pooling kernel of 9×9, and the corresponding receptive field (RF) is RF: 9×9. , k5 is the size of the pooling kernel is 5×5, the corresponding receptive field (RF) is RF:5×5, s1 represents the stride=1, and p2 represents the padding is 2.

[0047] The model is trained using a prepared dataset in a deep learning environment. The model is then targeted and tuned based on the loss curve and various performance indicators on the validation set to obtain the optimal weights and model parameters. Specifically, the model's performance is evaluated using five performance indicators: Precision (P), Recall (R), F1 score, AP (Average Precision), and MAP (Mean Average Precision).

[0048] Before the formal training begins, the model is initialized using pre-trained weights. Warm-up training is used during training, that is, a smaller initial value of the learning rate is used at the beginning of training, so that when the model understands the data distribution as 0 at the beginning, it will not overfit to the small batch images of Batch size. In the present invention, the initial value of the learning rate is set to 0.001. During the first 6 rounds of training, the learning rate gradually increases to 0.01; the input image size is set to 608×608 pixels, the Epoch is set to 200, the Batch size is set to 8, and the weight decay is set to 0.0005. After each Epoch is trained, the network weight file is saved, and the model is monitored in real time using the Tensorboard tool to record the change in the model's loss value and various performance indicators under the validation set. Since the validation loss stabilizes after about 120 rounds and no longer decreases, it means that the model has been fitted at this time. After 120 rounds, the validation loss fluctuates and rises, while the training loss is still decreasing, indicating that the model has been overfitted at this time. Since the model has been fitted in around 120 rounds, and the MAP value comprehensively considers the precision and recall of all categories, the present invention takes the maximum MAP value of each of the 10 rounds around the 120th round. The maximum value of 99.7% occurs in the 117th round, and the weight obtained in this round is used as the final model weight.

[0049] The test set of the present invention contains crystal oscillator images under various complex situations, which basically conforms to the distribution of the overall data set. The MAP value of the final optimized model on the test set can reach 99.5%, and the AP value of each category can reach 99.5%; the average detection speed on the RTX4060 GPU is 45.6 frames / s, and the detection accuracy and detection speed both meet actual production needs; the F1 value is the harmonic mean of the accuracy and recall rate, which changes with the change of the confidence threshold. When the confidence threshold is 0.683, the F1 value reaches a maximum value of 1, and the performance of the model on the test set is the best. After the final model training is completed, the optimal weight file is obtained.

[0050] S3: Extract features from the input image through the Backbone network to obtain a high-dimensional feature map; perform deep and shallow feature fusion and further feature extraction on the high-dimensional feature map through the Neck network to obtain a predicted input feature map; perform feature extraction on the predicted input feature map through the Head network to obtain a predicted feature map; The input image is subjected to feature extraction through the Backbone network. The input image undergoes a series of convolution, batch normalization, activation and other calculations, and is downsampled by 32 times to generate a high-dimensional feature map with a resolution of 19*19. The calculation expression of the high-dimensional feature map is: in, is the height of the high-dimensional feature map, is the width of the high-dimensional feature map, is the height of the input image, is the width of the input image, is the padding width, is the stride used in the convolution, is the width of the convolution kernel, is the height of the convolution kernel. By modifying the size of the convolution kernel as well as padding and stride, the size of the high-dimensional feature map can be adjusted.

[0051] In the model of the present invention, only the stride of the convolution kernel in the first CBS module of the CSSPX module in the Backbone network and the two CBS modules in the Neck network is 2, and 2 times downsampling is performed, while the stride of the rest is 1, and no downsampling is performed.

[0052] like Figure 7As shown in the figure, the input image size is set to 608×608 pixels. After each CSSPX module in the backbone network (X indicates that the CSSPX module has X RES Blocks), the feature map size is reduced to half of its original size. The stride s of the CSSPX modules in the neck and head networks is 1, and no downsampling is performed. After passing through the backbone network, two CBS modules and one SPPF module in the neck network and upsampling, it is concatenated and fused with the 38×38 feature map output by the CSSP6 module in the backbone network. After passing through a CSSP1 module and five CBS modules and upsampling, it is concatenated and fused with the 76×76 feature map output by the CSSP4 module in the backbone network. After passing through a CSSP1 module in the head network, a predicted feature map y1 of size 76×76 is obtained for detecting smaller crystal oscillators. The resulting 76×76 output feature map y1 is then downsampled by a factor of 2 through a CBS module with a stride of 2. The output feature map y2 of size 38×38 is concatenated and fused with the 38×38 feature maps output by the five CBS modules in the Neck network, and then passes through a CSSP1 module in the Head network to obtain a predicted feature map y2 of size 38×38, which is used to detect medium-sized crystal oscillators. Finally, the output feature map y2 of size 38×38 is downsampled by a CBS module with a step size of 2 by a factor of 2, and then concatenated and fused with the 19×19 feature map output by the CBS module before the first upsampling in the Neck network. After passing through a CSSP1 module in the Head network, a predicted feature map y3 of size 19×19 is obtained to detect larger crystal oscillators.

[0053] S4: The predicted feature map generates a predicted box by decoding the predicted box; the predicted box is redundantly eliminated by NMS non-maximum suppression to obtain the optimal predicted box; The number of channels in each of the three predicted feature maps is 27 (each feature point on the feature map predicts three prior boxes, and each prior box requires four category predictions, one confidence prediction, and four position predictions. Therefore, a total of (4+1+4)×3=27-dimensional prediction feature vectors are required, generating a total of (19×19+38×38+76×76)×3=22743 prediction results). The detection box is also based on the preprocessed image (i.e., the 608x608 target image), so the detection box on the target image needs to be mapped back to the source image. This transformation can be achieved using the inverse matrix of the affine transformation matrix, calculated as: in, is the first auxiliary variable, is the second auxiliary variable, is the third auxiliary variable; There are 22743 prediction boxes, most of which are overlapping. Therefore, it is necessary to eliminate redundant prediction boxes and select the optimal prediction box. Therefore, all prediction results need to be sent to Soft NMS for calculation, and the redundant prediction boxes are filtered out to obtain the final detection result. The calculation expression is: in, For the The confidence score of the predicted box, is the prediction box with the highest confidence score among all prediction boxes, For the The predicted boxes to be compared, is the ratio of the intersection and union of two prediction boxes, It is a manually set threshold, usually 0.5; σ is the continuous penalty coefficient.

[0054] In some specific embodiments of the present invention, .

[0055] S5: Perform crystal oscillator slot detection based on the optimal prediction frame; S51: Convert the pixel coordinates of the optimal prediction boxes of the three sizes into actual coordinates in the camera coordinate system; Convert the pixel coordinates in the detection results into actual coordinates in the camera coordinate system. The calculation expression is: in, is the physical size of the unit pixel on the horizontal axis, is the physical size of the unit pixel on the vertical axis, is the focal length, is the distance between the crystal oscillator and the optical center, is the coordinate of the origin of the image coordinate system in the pixel coordinate system, is the coordinate of the origin of the image coordinate system in the pixel coordinate system, the coordinate of the pixel point p in the pixel coordinate system is (u, v), and the coordinate in the camera coordinate system is (x, y, z).

[0056] S52: Divide the crystal oscillator image in the current detection field of view into regions centered at the center point of the detection region; The image under the current detection field of view is divided into multiple areas with the center point of the detection area as the center, and the crystal oscillator coordinates output by the model are sorted, and the crystal oscillators falling in different areas are encoded.

[0057] S53: sorting according to the actual coordinates and encoding the crystal oscillators falling within the area; Sort the crystal coordinates output by the model and encode the crystals in different areas. The calculation expression is: in, For crystal oscillator encoding, The number of regions divided into for the current image, for The coordinates of the lower left corner of the region, for The coordinates of the lower left corner of the region, is the coordinate of any pixel point on the source image.

[0058] S54: traverse all actual coordinates and encode all crystal oscillators falling within the area; Loop through all prediction results, and as long as the center point falls within a certain area, it is considered as the prediction result of the current detection area.

[0059] S55: Check the crystal oscillator slot status according to the marked information and coding; Adjust the NG_1 category into the slot according to the code; throw the NG_2 category into the defective box; after completing all the pick-and-place operations, take photos for re-inspection; if no results are returned or the test times out, sound and light alarms will be triggered; after the second re-inspection shows no NG category, move to the next station on the tray to take photos and continue testing.

[0060] like Figure 8 As shown, a quartz crystal oscillator precise slot detection system based on deep learning is used to perform a quartz crystal oscillator precise slot detection method based on deep learning, including: The image acquisition module 101 collects crystal oscillator images of different detection stations and pre-processes the crystal oscillator images to obtain input images; The model construction module 102 constructs a deep learning network model, wherein the deep learning network model includes a Backbone network, a Neck network, and a Head network; The feature extraction module 103 extracts features from the input image through the Backbone network to obtain a high-dimensional feature map; performs deep and shallow feature fusion and further feature extraction on the high-dimensional feature map through the Neck network to obtain a predicted input feature map; and performs feature extraction on the predicted input feature map through the Head network to obtain a predicted feature map. The optimization module 104 is used to predict the feature map and generate a prediction frame by decoding the prediction frame; the prediction frame is redundantly eliminated by NMS non-maximum suppression to obtain the optimal prediction frame; The detection module 105 performs crystal oscillator slot detection according to the optimal prediction frame.

[0061] Through the collaborative work of the above modules, the present invention uses a deep learning algorithm to extract and classify features of crystal oscillator images, ensuring that crystal oscillators in various states under different lines, different fields of view, different sizes, different positions and distances can be accurately detected, significantly improving the accuracy of slot detection, reducing the false detection rate and missed detection rate, helping to reduce production costs and improve the company's market competitiveness.

[0062] The present invention realizes real-time detection of whether multiple crystal oscillators are accurately inserted into the slots under various fields of view, triggers corresponding feedback mechanisms based on the detection results, realizes automatic control and adjustment, reduces manual intervention and downtime, reduces production costs, and improves the automation level and production efficiency of the production line.

[0063] The present invention can adapt to crystal oscillators of different models, sizes, fields of view, and front and back sides in various production environments. It has strong robustness, generalization, and scalability. By continuously updating and optimizing the model, it can adapt to new situations and new demands in the production environment and ensure long-term stable operation.

[0064] The present invention can be easily embedded into the vision and control systems of crystal oscillator manufacturing equipment to achieve seamless integration with existing production lines. At the same time, by providing rich interface and document support, it is convenient for users to carry out secondary development and integration.

[0065] The implementation of this invention brings opportunities for intelligent upgrades to the crystal oscillator manufacturing process and promotes the further development and innovation of electronic component manufacturing technology. By combining it with technologies such as big data and cloud computing, a higher level of intelligent management and decision support can be achieved, which helps companies maintain technological leadership and cope with increasingly fierce market competition.

[0066] Automated detection and control reduces manual operations, reduces safety risks caused by human factors, and helps to ensure the personal safety of employees and the production safety of enterprises.

[0067] The successful application of this invention will promote industrial upgrading and technological progress in the electronic component manufacturing industry, and promote the development and improvement of related industrial chains, which will help enhance the competitiveness and sustainable development capabilities of the entire industry. By reducing manual intervention and downtime, it will help reduce energy consumption and waste emissions, which is in line with the current global trend of environmental protection and sustainable development.

[0068] In addition to the crystal oscillator manufacturing field, the present invention can be expanded to other fields that require high-precision automated detection and control, such as precision instrument processing, semiconductor manufacturing, electronic product assembly, product appearance defect detection, etc., and has broad application prospects.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for accurately detecting the insertion of a quartz crystal oscillator into a slot based on deep learning, characterized in that: include: S1: Collect crystal oscillator images at different inspection stations and pre-process the crystal oscillator images to obtain input images; S2: Constructing a deep learning network model, wherein the deep learning network model includes a Backbone network, a Neck network, and a Head network; S3: Extract features from the input image through the Backbone network to obtain a high-dimensional feature map; perform deep and shallow feature fusion and further feature extraction on the high-dimensional feature map through the Neck network to obtain a predicted input feature map; perform feature extraction on the predicted input feature map through the Head network to obtain a predicted feature map; S4: The predicted feature map generates a predicted box by decoding the predicted box; the predicted box is redundantly eliminated by NMS non-maximum suppression to obtain the optimal predicted box; S5: Perform crystal oscillator slot detection based on the optimal prediction frame.

2. The method for accurately detecting quartz crystal oscillator slots based on deep learning according to claim 1, characterized in that: The preprocessing includes geometric transformation, optical transformation, noise addition and labeling information. The labeling information includes target category, target upper left corner coordinates, target lower right corner coordinates. The target categories include empty slot and no material OK_1, completely entered slot OK_2, the slot is within the slot range and the slot needs to be adjusted NG_1, and the slot is outside the slot range or the material is reversed and the defective product box needs to be sucked out and thrown away NG_2.

3. The method for accurately detecting quartz crystal oscillator slots based on deep learning according to claim 1, characterized in that: The Backbone network includes a CBS module, a CSSPX module containing one RES Block, a RES-SE module, a CSSPX module containing two RES Blocks, a RES-SE module, a CSSPX module containing four RES Blocks, a CSSPX module containing six RES Blocks, and a CSSPX module containing eight RES Blocks, which are stacked in sequence.

4. The method for accurately detecting quartz crystal oscillator slots based on deep learning according to claim 3, characterized in that: The CBS module includes a convolutional layer, a BN layer and a SiLu activation layer stacked in sequence; The RES Block includes a CBS module with a convolution kernel of 1×1, a CBS module with a convolution kernel of 3×3, and a CBS module with a convolution kernel of 1×1, which are stacked in sequence; The working process of the CSSPX module is as follows: The input image of the CSSPX module is downsampled by the CBS module to obtain the first feature map. The first feature map is passed through a CBS module and multiple ES Block modules, and then through a CBS module for feature extraction to obtain the second feature map. The first feature map is passed through a CBS module for feature extraction to obtain the third feature map. The second feature map and the third feature map are spliced, and then feature extraction is performed through a CBS module to obtain the CSSPX module output feature image.

5. The method for accurately detecting quartz crystal oscillator slots based on deep learning according to claim 4, characterized in that: The RES-SE module is to add the EDC-SE module to the RES Block; The working process of the RES-SE module is as follows: The output feature map of the CBS module with a convolution kernel of 3×3 is passed through the EDC-SE module to obtain a weight vector. The weight vector is weighted channel by channel to the output feature map of the original CBS module with a convolution kernel of 3×3 through multiplication operation. The features of the CBS module with a convolution kernel of 3×3 are then added to the input features of the original RES-SE module to obtain the output features of the RES-SE module. The working process of the EDC-SE module is as follows: The input features of the EDC-SE module are divided into two paths, one of which is obtained by stacking the global average pooling layer and the CBS module with a convolution kernel of 1×1 in sequence to obtain the fourth feature, and the other is obtained by stacking the global maximum pooling layer and the CBS module with a convolution kernel of 1×1 in sequence to obtain the fifth feature. The fourth and fifth features are spliced ​​and then connected with the activation function to obtain the output features of the EDC-SE module.

6. The method for accurately detecting quartz crystal oscillator slots based on deep learning according to claim 5, characterized in that: The Neck network uses a deep and shallow feature fusion network structure based on FPN and PAN, and adds SPPF module and CSSPX module at the front. The SPPF module is an improved spatial pyramid pooling layer. The working process of the SPPF module is as follows: The input feature map of the SPPF module first passes through the CBS module with a convolution kernel of 1x1 to extract preliminary features, and then passes through three maximum pooling layers with different pooling kernel sizes for downsampling. The four feature maps obtained are spliced ​​by channel, and the spliced ​​feature maps are then passed through the CBS module with a convolution kernel of 1x1 to extract features to obtain the output feature map of the SPPF module.

7. The method for accurately detecting quartz crystal oscillator slots based on deep learning according to claim 2, characterized in that: The predicted input feature map outputs predicted feature maps of three sizes through the Head network.

8. The method for accurately detecting quartz crystal oscillator slots based on deep learning according to claim 7, characterized in that: Step S5 includes: S51: Convert the pixel coordinates of the optimal prediction boxes of the three sizes into actual coordinates in the camera coordinate system; S52: Divide the crystal oscillator image in the current detection field of view into regions centered at the center point of the detection region; S53: sorting according to the actual coordinates and encoding the crystal oscillators falling within the area; S54: traverse all actual coordinates and encode all crystal oscillators falling within the area; S55: Check the crystal oscillator slot status according to the marked information and coding.

9. The method for accurately detecting quartz crystal oscillator slots based on deep learning according to claim 8, characterized in that: In step S53, in, For crystal oscillator encoding, The number of regions divided into for the current image, for The coordinates of the lower left corner of the region, for The coordinates of the lower left corner of the region, is the coordinate of any pixel point on the source image.

10. A quartz crystal oscillator precise slot detection system based on deep learning, characterized in that: The method for accurately detecting the insertion of a quartz crystal oscillator into a slot based on deep learning as claimed in any one of claims 1 to 9 comprises: An image acquisition module, which collects crystal oscillator images of different detection stations and preprocesses the crystal oscillator images to obtain input images; A model construction module, wherein the model construction module constructs a deep learning network model, wherein the deep learning network model includes a Backbone network, a Neck network, and a Head network; A feature extraction module extracts features from the input image through a Backbone network to obtain a high-dimensional feature map; performs deep and shallow feature fusion and further feature extraction on the high-dimensional feature map through a Neck network to obtain a predicted input feature map; and extracts features from the predicted input feature map through a Head network to obtain a predicted feature map; The optimization module is used to predict the feature map and generate a prediction frame by decoding the prediction frame; the prediction frame is redundantly eliminated by NMS non-maximum suppression to obtain the optimal prediction frame; A detection module performs crystal oscillator slot detection according to the optimal prediction frame.

Images