Ampoule bottle defect detection method, device and equipment and storage medium

Abstract

The invention discloses an ampoule bottle defect detection method and device, equipment and a storage medium, and the method comprises the steps: obtaining an original ampoule bottle image of a target ampoule bottle, carrying out the image correction of the original ampoule bottle image, and obtaining a to-be-detected ampoule bottle image; performing depth down-sampling and wavelet down-sampling on the to-be-detected ampoule bottle image, and fusing a depth feature and a wavelet feature to obtain an initial feature map; dividing the initial feature map into a plurality of feature convolution groups, and obtaining a reference defect feature map according to defect candidate areas of the feature convolution groups; according to the method, the original image of the ampoule bottle is corrected, sampled and subjected to feature extraction, the defects of the ampoule bottle in the image are highlighted, wavelet sampling and depth sampling are synchronously carried out according to the processed image, detail features are concerned, and the defect image of the target ampoule bottle is obtained. The data volume is reduced, meanwhile, key information is reserved, tiny defects on the ampoule bottle are accurately recognized, and the overall defects of the ampoule bottle are more rapidly and accurately recognized.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and particularly to an ampoule bottle defect detection method, device, equipment and storage medium. Background Art

[0002] With the rapid development of the pharmaceutical industry, drug packaging, as an important link to ensure the quality and safety of drugs, quality control in its production process is particularly crucial. As one of the main packaging forms of liquid drugs, ampoule bottles are widely used because of their good sealing performance, convenience for storage and transportation. The quality of ampoule bottles is not only related to the safety of drugs, but also directly affects the reputation and market competitiveness of pharmaceutical enterprises. Therefore, it is extremely important to effectively detect and control the appearance defects of ampoule bottles.

[0003] The traditional method for detecting the appearance defects of ampoule bottles mainly relies on manual visual inspection. Although this method can, to some extent, detect obvious appearance defects of ampoule bottles, with the expansion of the scale of pharmaceutical enterprises, the efficiency of manual visual inspection is low, making it difficult to meet the production demand of ampoule bottles. Moreover, the judgment criteria of different inspectors may vary, easily leading to inconsistent defect detection results and defect detection errors. In addition, long-term repetitive work is likely to cause fatigue of the inspectors, thereby affecting the accuracy and efficiency of detection.

[0004] The above content is only used to assist in understanding the technical solution of the present application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of the present application is to provide an ampoule bottle defect detection method, device, equipment and storage medium, aiming to solve the technical problem that the current defect detection for ampoule bottles is not accurate enough.

[0006] To achieve the above purpose, the present application proposes an ampoule bottle defect detection method, and the ampoule bottle defect detection method includes: Obtain the original ampoule bottle image of the target ampoule bottle, and perform image correction on the original ampoule bottle image to obtain the ampoule bottle image to be detected. The image correction includes brightness equalization processing, bilateral filtering processing and dilation correction processing; Synchronously perform depth downsampling and wavelet downsampling on the ampoule bottle image to be detected to obtain depth features and wavelet features, and fuse the depth features and the wavelet features to obtain an initial feature map; Divide the initial feature map into multiple feature convolution groups, and obtain a reference defect feature map according to the defect candidate regions of each feature convolution group; Perform feature mapping on the reference defect feature map to obtain the defect image of the target ampoule bottle.

[0007] In one embodiment, the steps of obtaining the original ampoule image of the target ampoule and performing image correction on the original ampoule image to obtain the ampoule image to be detected include: Obtain the original ampoule image of the target ampoule, perform brightness equalization processing on the pixel points in the original ampoule image to obtain an equalized ampoule image; Perform bilateral filtering on the equalized ampoule image to obtain a filtered ampoule image; Perform dilation correction processing on each pixel point of the filtered ampoule image to obtain the ampoule image to be detected.

[0008] In one embodiment, the step of performing bilateral filtering on the equalized ampoule image to obtain a filtered ampoule image includes: Obtain the filtering range of each pixel point according to the positions of the pixel points in the equalized ampoule image; Obtain the spatial proximity factor and pixel value similarity factor of each pixel point according to the filtering range and the pixel point position; Obtain a filtering factor according to the spatial proximity factor and the pixel value similarity factor; Perform bilateral filtering on each pixel point of the equalized ampoule image according to the filtering factor to obtain a filtered ampoule image.

[0009] In one embodiment, the step of performing dilation correction processing on each pixel point of the filtered ampoule image to obtain the ampoule image to be detected includes: Obtain the pixel point mean value according to the filtered ampoule image; Divide the pixel points in the filtered ampoule image into dark pixel points and light pixel points according to the pixel point mean value; Perform positive dilation on the dark pixel points and negative dilation on the light pixel points to obtain the ampoule image to be detected.

[0010] In one embodiment, the step of simultaneously performing depth downsampling and wavelet downsampling on the ampoule image to be detected to obtain depth features and wavelet features and fusing the depth features and the wavelet features to obtain an initial feature map includes: Perform strided convolution on the ampoule image to be detected to obtain depth features; Perform wavelet transform processing on the ampoule image to be detected to obtain wavelet features; Use the depth features and the wavelet features as the initial feature map.

[0011] In one embodiment, the step of performing strided convolution on the image of the ampoule bottle to be detected by the depth downsampling module to obtain depth features includes: Performing image uniform cropping on the image of the ampoule bottle to be detected to obtain a sub-feature map; Performing feature mapping and dimension connection on the sub-feature map to obtain sub-features to be output; Performing strided structure convolution on the sub-features to be output to obtain depth features.

[0012] In one embodiment, the step of dividing the initial feature map into multiple feature convolution groups and obtaining a reference defect feature map according to the defect candidate regions of each feature convolution group includes: Dividing the initial feature map into multiple feature convolution groups, and generating preliminary defect candidate regions according to the feature convolution for defect localization; Performing regional boundary correction on the preliminary defect candidate regions to obtain selected defect candidate regions; Performing separable convolution on the selected defect candidate regions of each feature convolution group based on channel fixation to obtain a reference defect feature map.

[0013] In addition, to achieve the above object, the present application also proposes an ampoule bottle defect detection device, and the ampoule bottle defect detection device includes: A preprocessing module, configured to obtain the original image of the target ampoule bottle, and perform image correction on the original image of the ampoule bottle to obtain an image of the ampoule bottle to be detected, where the image correction includes brightness equalization processing, bilateral filtering processing, and dilation correction processing; A sampling module, configured to simultaneously perform depth downsampling and wavelet downsampling on the image of the ampoule bottle to be detected to obtain depth features and wavelet features, and fuse the depth features and the wavelet features to obtain an initial feature map; A feature extraction module, configured to divide the initial feature map into multiple feature convolution groups, and obtain a reference defect feature map according to the defect candidate regions of each feature convolution group; A defect recognition module, configured to perform feature mapping on the reference defect feature map to obtain a defect image of the target ampoule bottle.

[0014] In addition, to achieve the above object, the present application also proposes an ampoule bottle defect detection device, and the device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the computer program is configured to implement the steps of the ampoule bottle defect detection method as described above.

[0015] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the ampoule defect detection method described above are implemented.

[0016] In addition, to achieve the above object, the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the ampoule defect detection method described above are implemented.

[0017] One or more technical solutions proposed by the present application have at least the following technical effects: By performing image processing such as correction, sampling, and feature extraction on the original image collected from the ampoule, the defects of the ampoule in the image are highlighted; according to the preprocessed image, depth sampling and wavelet sampling are synchronously performed to focus on more detailed features, effectively reducing the data volume while retaining key information, more accurately identifying the tiny defects on the ampoule, and more quickly and accurately identifying the overall defect problem of the ampoule. Description of the Drawings

[0018] The drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0019] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0020] Figure 1 It is a schematic flowchart provided for the first embodiment of the ampoule defect detection method of the present application; Figure 2 It is a schematic diagram of an ampoule defect detection device provided for the first embodiment of the ampoule defect detection method of the present application; Figure 3 It is a schematic diagram of ampoule image acquisition provided for the first embodiment of the ampoule defect detection method of the present application; Figure 4 It is a schematic diagram of a lightweight convolutional network structure provided for the first embodiment of the ampoule defect detection method of the present application; Figure 5 It is a schematic diagram of a regional boundary correction neural network structure provided for the first embodiment of the ampoule defect detection method of the present application; Figure 6 It is a schematic flowchart provided for the second embodiment of the ampoule defect detection method of the present application; Figure 7It is a schematic flowchart provided for the third embodiment of the ampoule defect detection method of this application; Figure 8 It is a schematic module structure diagram of the ampoule defect detection device of the embodiment of this application; Figure 9 It is a schematic device structure diagram of the hardware operating environment involved in the ampoule defect detection method in the embodiment of this application. Specific implementation manners

[0021] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not used to limit this application.

[0022] For a better understanding of the technical solutions of this application, the following will be described in detail in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0023] The main solution of the embodiment of this application is: obtaining the original ampoule image of the target ampoule, performing image correction on the original ampoule image to obtain the ampoule image to be detected; performing depth downsampling and wavelet downsampling on the ampoule image to be detected, and fusing the obtained depth features and wavelet features to obtain an initial feature map; dividing the initial feature map into multiple feature convolution groups, and obtaining a reference defect feature map according to the defect candidate regions of the feature convolution groups; performing feature mapping on the reference defect feature map to obtain the defect image of the target ampoule.

[0024] In this embodiment, for the convenience of description, the following will be described with the ampoule defect detection device as the execution subject.

[0025] Due to the rapid development of the pharmaceutical industry in the prior art, as an important link to ensure the quality and safety of drugs, the quality control of ampoules is particularly crucial. As one of the main packaging forms of liquid drugs, ampoules are widely used because of their good sealing performance, convenient storage and transportation. The quality of ampoules is not only related to the safety of drugs, but also directly affects the reputation and market competitiveness of pharmaceutical enterprises. Therefore, it is extremely important to effectively detect and control the appearance defects of ampoules.

[0026] The detection of the appearance defects of ampoules in the traditional way mainly relies on manual visual inspection. Although this method can, to a certain extent, find obvious appearance defects of ampoules, with the expansion of the scale of pharmaceutical enterprises, the efficiency of manual visual inspection is low, making it difficult to meet the production demand of ampoules. Moreover, the judgment criteria of different inspectors may vary, easily leading to inconsistent defect detection results and defect detection errors. In addition, long-term repetitive work is likely to cause fatigue of the inspectors, thereby affecting the accuracy and efficiency of detection.

[0027] The present application provides a solution. By correcting, sampling, and extracting features from the original image of an ampoule bottle, the defects of the ampoule bottle in the image are highlighted. Based on the processed image, wavelet sampling and depth sampling are synchronously performed, paying attention to the detailed features, while reducing the data volume and retaining the key information, accurately identifying the minute defects on the ampoule bottle, and more quickly and accurately identifying the overall defects of the ampoule bottle.

[0028] As can be seen from the above embodiments, the present application discloses a method, device, equipment, and storage medium for detecting defects of ampoule bottles, which relates to the technical field of image processing, and includes: obtaining the original image of the target ampoule bottle, performing image correction on the original image of the ampoule bottle to obtain the ampoule bottle image to be detected; performing depth downsampling and wavelet downsampling on the ampoule bottle image to be detected, and fusing the depth features and wavelet features to obtain an initial feature map; dividing the initial feature map into multiple feature convolution groups, and obtaining a reference defect feature map according to the defect candidate regions of the feature convolution groups; performing feature mapping on the reference defect feature map to obtain the defect image of the target ampoule bottle. This method highlights the defects of the ampoule bottle in the image by correcting, sampling, and extracting features from the original image of the ampoule bottle, synchronously performs wavelet sampling and depth sampling based on the processed image, pays attention to the detailed features, reduces the data volume while retaining the key information, accurately identifies the minute defects on the ampoule bottle, and more quickly and accurately identifies the overall defects of the ampoule bottle.

[0029] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, an ampoule bottle defect detection device, etc. that can implement the above functions. Hereinafter, taking the ampoule bottle defect detection device as an example, this embodiment and the following embodiments will be described.

[0030] Based on this, the embodiment of the present application provides a method for detecting defects of ampoule bottles, referring to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the method for detecting defects of ampoule bottles in the present application.

[0031] In this embodiment, the method for detecting defects of ampoule bottles includes steps S10 to S40: Step S10, obtaining the original image of the target ampoule bottle, performing image correction on the original image of the ampoule bottle to obtain the ampoule bottle image to be detected, and the image correction includes brightness equalization processing, bilateral filtering processing, and dilation correction processing.

[0032] It can be understood that the target ampoule bottle can be an ampoule bottle that needs to be detected for defects, and it can be a single ampoule bottle image or multiple ampoule bottle images. If it is a single ampoule bottle image, it can be used for training when training the ampoule bottle defect detection model.

[0033] It should be understood that the original ampoule image can be an image directly collected by an image acquisition device without image processing; in this embodiment, an ampoule defect detection device is also involved. The device may include an ampoule detection point frame, an industrial control computer, and a production line interaction module. In specific implementation, reference can be made to Figure 2 , in the schematic diagram of the ampoule defect detection device, the ampoule detection point frame includes a detection point frame body 1. In this embodiment, the detection point frame body 1 is arranged on the ampoule production line 2. For the specific schematic diagram of ampoule image acquisition, reference can be made to Figure 3 , and the ampoule production line 2 conveys the ampoules to be detected; the detection point frame body 1 is provided with a mounting plate 101 for the image acquisition unit 3. In this embodiment, the image acquisition unit 3 uses a high-definition industrial camera, Figure 2 in which the camera body 301 and the lens 302 of the high-definition industrial camera are shown. The image acquisition unit 3 is suspended on the mounting plate 101 of the detection point frame body 1. The detection point frame body 1 is also provided with a light source 11, a light correction unit, and a detection signal triggering unit.

[0034] It should be noted that during the process of image acquisition of the target ampoule, since the acquisition angle may affect the light, clarity, etc., the detection effect of directly using the acquired image for defect detection is not ideal. The image can be corrected first, which is beneficial to the accuracy of subsequent image defect detection.

[0035] It should be emphasized that the detection of the original ampoule image can include processing such as brightness equalization, filtering, and dilation correction of the original ampoule image. It can be understood that the image obtained after the correction processing is the ampoule image to be detected.

[0036] Step S20: Simultaneously perform depth downsampling and wavelet downsampling on the ampoule image to be detected to obtain depth features and wavelet features, and fuse the depth features and the wavelet features to obtain an initial feature map.

[0037] It can be understood that the ampoule defect detection model can be a model that has been trained to accurately perform defect detection.

[0038] It should be understood that the initial feature map can be a fusion map of the depth downsampled feature map and the wavelet downsampled feature map, or it can be understood that the initial feature map includes the feature map obtained by depth downsampling and the feature map obtained by wavelet downsampling.

[0039] It should be noted that the synchronous downsampling attention layer can be image processing that combines the attention mechanism and downsampling technology, where two downsampling methods are used simultaneously for downsampling.

[0040] In a feasible implementation manner, steps A201 to A204 are further included before step S20: Step A201, obtaining a preset defect detection model and a training data set, where the preset defect detection model includes a synchronous downsampling attention layer, a feature refinement and screening layer, a classification layer, and a generative adversarial layer.

[0041] It should be noted that the generative adversarial layer includes a generative adversarial network, which continuously generates new training data according to the training data set and optimizes according to the detection results of the new training data.

[0042] It is worth noting that the training of the preset defect detection model has high requirements for a large amount of high-quality labeled data, and traditional machine learning methods rely on manually designed features, which all increase the cost of data acquisition and processing. At the same time, it is difficult to obtain diverse and comprehensive defect sample data, which limits the generalization ability and application promotion of the model. In this embodiment, before training the multi-task defect detection model, multi-view images of ampoules are collected to form an image sample set, and the image sample set is expanded using data augmentation and transfer learning; in this embodiment, a generative adversarial network (GAN) is introduced for data augmentation to generate diverse and high-quality defect samples, enriching the training set and further improving the generalization ability and accuracy of the detection model.

[0043] It should be understood that the training data set can be generated by collecting images of defective and non-defective ampoules through an image acquisition device.

[0044] It can be understood that the synchronous downsampling attention layer combines the attention mechanism and downsampling technology, aiming to focus on the most important parts of the image and reduce the spatial resolution of the image to reduce the computational complexity; the feature refinement and screening layer extracts useful features from the image after synchronous downsampling processing and further screens and optimizes these features; the classification layer maps the feature map after feature refinement and screening to specific class labels for classification tasks.

[0045] It should be emphasized that the generative adversarial layer is used to optimize the model data during training to facilitate better training of the model. The ampoule defect detection model after the model is trained can have no generative adversarial layer.

[0046] Step A202, obtaining a target data set based on the training data set and the generative adversarial layer, and training the synchronous downsampling attention layer, the feature refinement and screening layer, and the classification layer according to the target data set to obtain a training result.

[0047] It should be noted that obtaining the target dataset based on the training dataset and the generative adversarial layer can be achieved by first training a generative adversarial network (GAN). The generator in the generative adversarial network generates synthetic images similar to real defect images, starting from random noise or low-resolution images and gradually generating high-resolution and realistic defect images. The discriminator distinguishes between the generated images and the real defect images. Through a feedback mechanism, the discriminator helps the generator continuously improve the generation effect.

[0048] Furthermore, use the trained generator to generate a large number of synthetic defect images. The synthetic images can supplement the deficiencies in the original training dataset, increasing the diversity and scale of the dataset. Mix the generated synthetic images with the original training dataset to form a new and richer target dataset. The target dataset not only contains the actually collected defect images but also includes high-quality defect images generated by the generator.

[0049] It should be emphasized that the training result can be the result of defect detection on the ampoule bottle images in the target dataset by a preset defect detection model. For example, the detection result can include the defects in the image, the accuracy of the defect categories, etc.

[0050] Step A203, calculate the cross-entropy loss function according to the training result, and perform backpropagation on the synchronous downsampling attention layer, the feature refinement and screening layer, and the classification layer according to the cross-entropy loss function to obtain an optimized defect detection model.

[0051] It should be noted that calculating the cross-entropy loss function according to the training result can refer to the following formula:

[0052] where i represents different defect categories of the ampoule bottle images to be detected, Y i represents the probability that the model predicts that the image has a defect of category i, and y i represents whether the defect of the image is of the i-th category, which can be represented by 1 or 0.

[0053] It should be noted that performing backpropagation on the synchronous downsampling attention layer, the feature refinement and screening layer, and the classification layer according to the cross-entropy loss function to obtain an optimized defect detection model can be to calculate the gradient of each layer's parameter with respect to the loss using the backpropagation algorithm according to the calculated cross-entropy loss value.

[0054] In a specific implementation, when training a preset ampoule defect detection model, the target data set is divided into a training set and a test set. Based on the backpropagation method, the preset ampoule defect detection model is trained using the training set, and the model parameters of the preset ampoule defect detection model are adjusted by an optimizer. In the training loop, the training set is divided into several batches. For the data of each batch of the training set, first, the gradients of the optimizer are cleared, then the training set data is input into the preset ampoule defect detection model, the cross-entropy loss function is calculated, the cross-entropy loss function is backpropagated, and the model parameters are updated according to the gradients calculated by the backpropagation. When the cross-entropy loss function converges, the trained preset ampoule defect detection model is obtained; and the test of the preset ampoule defect detection model is completed using the test set.

[0055] Step A203: Perform pruning and quantization processing on the optimized defect detection model to obtain an ampoule defect detection model.

[0056] It should be noted that performing pruning and quantization processing on the optimized defect detection model can be to perform pruning and quantization processing on the parameters in the model, where the model parameters can include the convolution kernel size, the number of convolution kernels, the stride, the boundary mode, the bias weight, etc. of the convolutional layer.

[0057] It should be understood that pruning refers to simplifying the model structure by removing unimportant weights or neurons in the model, thereby reducing the computational amount and memory occupancy; quantization refers to converting the floating-point weights and activation values in the model into a low-precision representation (such as 8-bit integers) to reduce the storage requirements and computational complexity of the model.

[0058] It should be emphasized that after the multi-task defect detection model is trained, model parameter pruning and quantization processing are performed on the multi-task defect detection model. Among them, pruning reduces the unnecessary weights and connections of the model, thereby reducing the computational cost; quantization processing improves the efficiency and speed of the model by compressing or streamlining the model and reducing the computational amount and storage space of the model.

[0059] In this embodiment, the training data is enhanced through a generative adversarial layer, and the enhanced images are mixed to train the model to obtain better training results. After the model is trained, the computational cost is reduced through pruning and quantization processing, and the computational amount and storage space of the model are reduced, effectively improving the detection efficiency and detection speed of the model while ensuring the detection accuracy of the model.

[0060] The above are only feasible implementation manners before step S20 provided in this embodiment, and this embodiment does not make specific limitations on the specific implementation manners before step S20.

[0061] Step S30: Divide the initial feature map into multiple feature convolution groups, and obtain a reference defect feature map according to the defect candidate regions of each feature convolution group.

[0062] It can be understood that the feature extraction of the initial feature map to obtain the reference defect feature map can be performed through the feature refinement and screening layer of the ampoule bottle defect detection model. The reference defect feature map can be a feature map with defects extracted from the initial feature map, which contains the key features that best represent the defects.

[0063] It should be noted that the feature refinement and screening layer can be a lightweight convolutional network. Specifically, reference can be made to Figure 4 , Figure 4 In the lightweight convolutional network structure diagram, the feature extraction network includes a first convolutional layer Conv1, a max pooling layer MaxPooling1, a second convolutional layer Conv2, and a third convolutional layer Conv3 connected in sequence. After each convolutional layer in the first convolutional layer Conv1, the second convolutional layer Conv2, and the third convolutional layer Conv3, a ReLU activation function layer is connected; after the image of the ampoule bottle is successively subjected to convolutional operations by the first convolutional layer Conv1, the max pooling layer Max Pooling1, the second convolutional layer Conv2, and the third convolutional layer Conv3, image features are extracted to generate a feature map.

[0064] In a feasible implementation manner, step S30 may include steps A31 to A33: Step A31: Divide the initial feature map into multiple feature convolution groups, and generate preliminary defect candidate regions according to the feature convolution for defect localization.

[0065] It can be understood that the generation of preliminary defect candidate regions according to the initial feature map can be performed through the feature refinement and screening layer of the ampoule bottle defect detection model.

[0066] It should be noted that when performing grouped convolution on the initial feature map, when the convolution is divided into G groups, the number of parameters of the convolution kernel is 1 / G of the original. Preliminary candidate regions are obtained by defect localization according to the features extracted from each group respectively, and then the preliminary candidate regions are further screened. The selected defect candidate regions of multiple feature groups are convolved to obtain the finally output reference curve feature map.

[0067] It should be noted that the generation of preliminary defect candidate regions according to the fused image can be referred to Figure 4 , Figure 4The middle output end includes three parts. In the figure, the first part "classification" represents preliminary classification, the second part "Bounding Box Regression" represents preliminary bounding box regression, and the third part "Localization" represents preliminary localization. Through Figure 3 the feature extraction network shown, preliminary defect candidate regions are generated.

[0068] Among them, in the object detection task, preliminary classification refers to the process of predicting the category of candidate regions or anchor boxes in the image, which can automatically extract features from the image and judge which category each candidate region belongs to based on these features; bounding box regression aims to adjust the position and size of the candidate regions generated by the preliminary classification stage to make them more accurately match the actual position of the target object, so as to obtain more accurate target localization. This step is crucial for improving the accuracy of object detection; preliminary localization refers to determining the approximate position of the target in the image to achieve more precise target localization.

[0069] In the specific implementation, the input image size is 12*12*3. The first convolutional layer Conv1 uses a 3x3 convolutional kernel with a stride of 1 and outputs a feature map with 10 channels. The second convolutional layer Conv2 uses a 3x3 convolutional kernel with a stride of 1 and outputs a feature map with 16 channels. The third convolutional layer Conv3 uses a 3x3 convolutional kernel with a stride of 1 and outputs a feature map with 32 channels. For the max pooling layer, Max Pooling1 in the figure uses a 2x2 pooling window with a stride of 2, which can reduce the computational complexity of the upper layer. The feature map output by the convolutional layer often contains a lot of data. The max pooling layer selects the maximum value within its pooling window as the output, reducing the spatial dimension of the feature map, thereby reducing the computational amount of the subsequent layer (upper layer) and the resources required for computing. In addition, to ensure translational invariance, the max pooling layer is implemented on a 2x2 window and can accurately produce the same result as the layer connected after the convolutional layer in possible translational configurations, which means that when the image is translated within a certain range, the results obtained after passing through the max pooling layer are similar. This feature helps the model to still be able to recognize well when the target object has a small translation in the image recognition and other tasks.

[0070] Step A32: Perform regional boundary correction on the preliminary defect candidate regions to obtain selected defect candidate regions.

[0071] It can be understood that performing regional boundary correction on the preliminary defect candidate regions can be to further process the preliminary defect candidate regions, adjust their boundaries to obtain a more accurate defect position, thereby improving the accuracy of the defect candidate regions and making them more accurately cover the actual defects. The candidate regions after boundary correction have high position accuracy.

[0072] It should be noted that the regional boundary correction can be to apply geometric transformations (such as translation, scaling, rotation, etc.) to the preliminary defect candidate regions through set transformation, which can adjust their shapes and sizes to better match the positions of actual defects; it can also be to fine-tune the candidate regions by combining the surrounding context information. This embodiment does not make any limitations in this regard, and can be selected or adjusted according to the actual situation.

[0073] In a specific implementation, the regional boundary correction is a convolutional neural network with a complexity greater than that of the feature extraction network. For the specific structure diagram of the regional boundary correction neural network, reference can be made to Figure 5 , Figure 5 where Conv4 represents the fourth convolutional layer, Max Pooling represents the maximum pooling layer, Conv5 represents the fifth convolutional layer, Max Pooling3 represents the maximum pooling layer, Conv6 represents the sixth convolutional layer and the fully connected layer. The refined screening network finely screens the preliminary defect candidate regions and corrects the boundaries of the preliminary candidate regions to generate selected defect candidate regions.

[0074] Step A33: Convolve the selected defect candidate regions based on channel fixation to obtain a reference defect feature map.

[0075] It can be understood that fixing the number of channels can make the channels (or feature dimensions) be particularly concerned or remain unchanged during the convolution process, which can better retain key features and reduce unnecessary calculations.

[0076] It should be understood that convolution can further extract and optimize the features in the selected defect candidate regions to generate a final reference defect feature map.

[0077] In this embodiment, the feature refinement and screening layer of the ampoule defect detection model generates preliminary defect candidate regions from the initial feature map, then corrects the boundaries of these regions to obtain more accurate selected defect candidate regions, and finally extracts high-quality reference defect feature maps through convolution operations based on channel fixation, effectively improving the accuracy and reliability of defect detection, ensuring that the model can accurately identify and locate subtle defects in ampoules, while reducing false alarms and computational complexity. The finally generated reference defect feature map provides a basis for subsequent defect classification.

[0078] The above is only a feasible implementation manner of step S30 provided by this embodiment. This embodiment does not make specific limitations on the specific implementation manner of step S30.

[0079] Step S40: Perform feature mapping on the reference defect feature map to obtain the defect image of the target ampoule.

[0080] It should be noted that the defect image of the target ampoule bottle obtained by performing feature mapping on the reference defect feature map can be carried out through the classification layer of the ampoule bottle defect detection model.

[0081] It should be further noted that the classification layer performs a non-linear transformation on the input feature map through a series of learning parameters (such as weights and biases), and each output unit corresponds to a type of defect or background category. By comparing the activation values of each unit, the most likely defect type is determined. Based on the result of the feature mapping, the position of the defect in the original ampoule bottle image is located, and the spatial information of the feature map is inversely mapped back to the original image coordinate system to accurately identify the position of the defect.

[0082] This embodiment provides an ampoule bottle defect detection method. By correcting, sampling, and extracting features from the original image of the ampoule bottle, the defects of the ampoule bottle in the image are highlighted. According to the processed image, wavelet sampling and depth sampling are synchronously performed, focusing on the detailed features, reducing the amount of data while retaining the key information, accurately identifying the tiny defects on the ampoule bottle, and more quickly and accurately identifying the overall defects of the ampoule bottle.

[0083] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar content as in the above-mentioned first embodiment can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 6 , step S10 further includes steps S11 to S13: Step S11, obtaining the original ampoule bottle image of the target ampoule bottle, and performing brightness equalization processing on the pixel points in the original ampoule bottle image to obtain the equalized ampoule bottle image.

[0084] It can be understood that performing brightness equalization processing on the pixel points in the original ampoule bottle image is mainly to improve the contrast and brightness distribution of the image, making the details in the image more clearly visible.

[0085] It should be understood that brightness equalization can redistribute the gray values in the image to improve the image contrast, converting the gray histogram of the original image (i.e., the frequency of each gray level) into a uniform distribution form, thereby expanding the overall brightness range of the image.

[0086] In a specific implementation, first, count the number of occurrences of each gray level in the image, and draw a gray histogram of the original image based on this data. Based on the original histogram, calculate the cumulative distribution function, which represents the cumulative proportion of all pixel numbers less than or equal to a certain gray value. Use the cumulative distribution function as the transformation function; map each pixel value in the original image to a new gray value, which is evenly distributed across the entire gray scale range, can increase the dynamic range of the image, thereby improving the contrast; finally, generate a new image using the above mapping relationship, where the gray value of each pixel is replaced with its corresponding equalized gray value.

[0087] Step S12: Perform bilateral filtering on the equalized ampoule image to obtain a filtered ampoule image.

[0088] It can be understood that bilateral filtering is a non-linear filtering method that combines two factors, spatial proximity and pixel value similarity, to smooth the image. This method can reduce noise while keeping edges clear.

[0089] It should be understood that when the ampoule image is processed by brightness equalization, bilateral filtering can help further improve the image quality. Equalization may enhance the contrast in some areas, and at the same time may also amplify the noise in the image. Bilateral filtering can effectively reduce this noise, making the image cleaner, so as to more accurately identify the defects in the image.

[0090] In a feasible implementation, step S12 may include steps A121 to A124: Step A121: Obtain the filtering range of each pixel point according to the positions of each pixel point in the equalized ampoule image.

[0091] It should be noted that the filtering range can be determined according to a pre-determined filtering radius. Determining the filtering range according to the filtering radius can simply be to take the area around each pixel point with a distance not exceeding the preset distance from the pixel point as the filtering range.

[0092] Furthermore, the preset distance can be calculated according to the filtering radius, specifically l=(2r + 1)*(2r + 1), where l represents the preset distance and r represents the filtering radius; take the range with a distance less than l from each pixel point as the filtering range.

[0093] Step A122: Obtain the spatial proximity factor and pixel value similarity factor of each pixel point according to the filtering range and the pixel point position.

[0094] It should be noted that the calculation of the spatial proximity factor can refer to the following formula:

[0095] Among them, i and j represent the coordinate points of each pixel position; k and l represent the coordinates of adjacent pixel positions; σ s represents the spatial standard deviation parameter, which controls the spatial expansion range of the filter.

[0096]

[0097] Among them, I(i, j) represents the gray value of each pixel. If it is a color image, it is the color vector; I(k, l) represents the gray value or color vector of adjacent pixels; σ r represents the standard deviation parameter of the pixel value, which determines the sensitivity of the filter to the difference in pixel values.

[0098] Step A123: Obtain a filtering factor according to the spatial proximity factor and the pixel value similarity factor.

[0099] It should be noted that multiplying the spatial proximity factor and the pixel value similarity factor can obtain the filtering factor. Specifically, the following formula can be referred to:

[0100] Among them, H(i, j, k, l) represents the filtering factor.

[0101] Step A124: Perform bilateral filtering on each pixel of the equalized ampoule image to obtain a filtered ampoule image.

[0102] It should be noted that the calculation formula of the pixel after bilateral filtering of each pixel can be referred to the following formula:

[0103] Among them, g(i, j) represents the filtered pixel, S(i, j) represents the filtering range, and f(k, l) represents the adjacent pixel.

[0104] It can be understood that bilateral filtering of each pixel in the equalized ampoule image to obtain the filtered pixels constitutes a filtered ampoule image.

[0105] It is worth noting that when detecting whether there are cracks, breakages or other defects in the ampoule, it is very important to keep the edges clear. Bilateral filtering can smooth the image while ensuring that these key features are not affected. By removing unnecessary noise and maintaining key features, the image processed by bilateral filtering is more conducive to accurate automated analysis, such as defect detection or content recognition, etc.

[0106] In this embodiment, the bilateral filter adjusts the value of each pixel in the equalized ampoule image by considering both the above-mentioned spatial proximity and pixel value similarity. It not only takes into account the physical distance between pixels but also the color or brightness difference between them, which can effectively remove noise and smooth the image while well preserving the important features in the image.

[0107] The above is only a feasible implementation manner of step S12 provided in this embodiment, and this embodiment does not make specific limitations on the specific implementation manner of step S12.

[0108] Step S13, perform dilation correction processing on each pixel point of the filtered ampoule image to obtain the ampoule image to be detected.

[0109] It can be understood that dilation correction is usually used to correct problems such as loss of image details or deformation caused by noise or other factors. Repairing the edges and contours of the filtered image can obtain a clearer and more accurate image.

[0110] It should be noted that dilation is a basic operation in morphological operations, mainly used to expand the foreground area in the image (usually the white or high-brightness area). The dilation operation slides the structuring element (also called the kernel) on the image and increases the pixel value according to the position of the structuring element. The dilation operation can fill small holes, connect adjacent foreground areas, and expand the object boundary.

[0111] It should be understood that the ampoule image to be detected can be an image obtained by performing dilation correction on the filtered ampoule image.

[0112] In a feasible implementation manner, step S13 may include steps A131 to A133: Step A131, obtain the pixel point mean value according to the filtered ampoule image.

[0113] It can be understood that the pixel point mean value can be calculated according to the pixel values of each pixel point. Specifically, it can refer to the following formula:

[0114] Among them, Q represents the pixel point mean value, P represents the pixel values of each pixel point, and n represents the total number of pixel points.

[0115] Step A132, divide the pixel points in the filtered ampoule image into dark pixel points and light pixel points according to the pixel point mean value.

[0116] It can be understood that the pixel points with pixel values greater than the pixel point mean value are used as dark pixel points, and the pixel points with pixel values less than or equal to the pixel point mean value are used as light pixel points.

[0117] Step A133, perform positive dilation on the dark pixel points and negative dilation on the light pixel points to obtain the image of the ampoule to be detected.

[0118] It should be noted that in order to enhance the contrast between the defect target and the background, when the defect target in the original image is dark, if the pixel value of P(x, y) is greater than the mean value Q, the pixel point may be inside or on the boundary of the defect target; if the pixel value of P(x, y) is less than the mean value Q, the pixel point P(x, y) may be in the background; when the defect target in the original image is light, vice versa.

[0119] It should be noted that positive dilation can be used to expand the foreground (usually white or high-brightness areas). For each pixel point p, if there are foreground pixels in its neighborhood, then set p to the foreground value. It is applicable to filling small holes, connecting adjacent foreground areas, and expanding the object boundary.

[0120] Negative dilation is used to shrink the foreground (reduce the white or high-brightness areas). For each pixel point p, if all pixels in its neighborhood are foreground pixels, then keep the foreground value of p; otherwise, set it to the background value (usually black or low-brightness value). It is applicable to removing small noise, separating adhered objects, and shrinking the object boundary.

[0121] It should be emphasized that the dilation operation fills the small holes in the dark areas, making these areas more complete and continuous. At the same time, it removes the noise in the light areas and shrinks these areas, reducing artifacts. Through positive dilation and negative dilation processing, the edges and contours of the ampoule are clearer.

[0122] In the specific implementation, the dilation processing for dark pixel points and light pixel points can refer to the following formula:

[0123] In this embodiment, by performing positive dilation on the dark pixel points and negative dilation on the light pixel points of the filtered ampoule image respectively, we can effectively enhance the specific features in the image and reduce the influence of noise.

[0124] The above is only a feasible implementation manner of step S13 provided in this embodiment. This embodiment does not make specific limitations on the specific implementation manner of step S13.

[0125] This embodiment provides an ampoule defect detection method. By performing brightness equalization processing on the original ampoule image, the contrast and brightness distribution of the image are effectively improved, making the details in the image more clearly visible. Applying bilateral filtering processing further smooths the image and reduces noise, while retaining key edge and texture features to ensure the integrity of image information. By performing dilation correction processing on the filtered image, small holes in the dark areas are filled and noise points in the light areas are shrunk, making the contour of the ampoule more complete and clear, improving the effect and reliability of image processing, and enabling more accurate identification of ampoule defects.

[0126] Based on the first embodiment of this application, in the second embodiment of this application, for the same or similar content as the above-mentioned embodiment one, reference can be made to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 7 , step S20 further includes steps S21 to S23: Step S21, perform strided convolution on the image of the ampoule to be detected to obtain depth features.

[0127] It should be noted that performing strided convolution on the image of the ampoule to be detected to obtain depth features can be executed by the depth downsampling module. When the depth downsampling module processes an image or a feature map, the difference from non-strided convolution is that the stride of strided convolution is greater than 1, and while extracting features, it can achieve a downsampling effect and reduce the spatial dimension of the feature map.

[0128] It can be understood that strided structure convolution refers to the stride used in the convolution operation being greater than 1 (usually 2). This technique can achieve downsampling without using a pooling layer. The advantage of doing this is that it can maintain the end-to-end learning process and avoid the information loss problem that may be brought by traditional pooling methods.

[0129] In a feasible implementation manner, step S21 may include steps A211 to A213: Step A211, perform uniform cropping on the image of the ampoule to be detected to obtain a sub-feature map.

[0130] It should be noted that performing uniform cropping on the image of the ampoule to be detected can be to divide the input ampoule image into several small blocks or sub-regions of the same size. This segmentation method ensures that the entire image is evenly covered without omission or overlap (or with a specific strategy of overlap). The purpose of doing this is to enable each sub-region to independently receive subsequent processing, so as to better capture the local features in the image.

[0131] It should be understood that each local feature can be regarded as a sub-feature map, which contains the information of a certain part of the original image.

[0132] Step A212: Perform feature mapping and dimension connection on the sub-feature map to obtain the sub-feature to be output.

[0133] It can be understood that performing feature mapping and dimension connection on the feature map to obtain the sub-feature to be output may be to convert the sub-feature map into a format that is more easily understood and used by machine learning models.

[0134] It should be noted that feature mapping can be to apply convolution operations or other feature extraction techniques to each sub-feature map to identify key features therein, which may include information such as edges, textures, and color distributions. The result of feature mapping is a set of new representations that are more abstract and rich in semantic information.

[0135] It should be further noted that since each sub-feature map has undergone feature mapping, multi-dimensional data may be generated. At this time, it is necessary to integrate this information of different dimensions to form a comprehensive feature vector or matrix, ensuring that information from different sub-regions can be effectively combined.

[0136] In specific implementation, feature mapping can be carried out in ways such as using convolutional layers, non-linear functions, or multi-layer stacking; dimension connection can be carried out in ways such as cross-layer connection, multi-scale fusion, and channel concatenation.

[0137] Step A213: Perform strided structure convolution on the sub-feature to be output to obtain the deep feature.

[0138] It can be understood that the sub-feature to be output is further processed using strided structure convolution to reduce the data dimension while retaining key features, and finally obtain deep features suitable for high-level tasks (such as classification and detection).

[0139] It should be noted that using the strided convolution operation to slide the convolution kernel with a relatively large stride to process the sub-feature to be output can reduce the feature map size and the computational cost at the same time.

[0140] In this embodiment, the sub-feature map is obtained by uniformly cropping the image through a deep downsampling module; feature mapping and dimension connection are performed on the sub-feature map, high-level features are extracted and integrated into the sub-feature to be output, and the sub-feature to be output is further processed by strided structure convolution, realizing dimensionality reduction while retaining key feature information. Finally, deep features are generated, effectively reducing the data dimension and computational complexity, and also enhancing the feature representation ability and robustness, providing high-quality and high-dimensional feature representations for subsequent defect detection and analysis, and significantly improving the accuracy of defect detection.

[0141] The above is only a feasible implementation manner of step S21 provided in this embodiment, and this embodiment does not specifically limit the specific implementation manner of step S21.

[0142] Step S22: Perform wavelet transform processing on the image of the ampoule bottle to be detected to obtain wavelet features.

[0143] It can be understood that performing wavelet transform processing on the image of the ampoule bottle to be detected to obtain wavelet features can be carried out through the wavelet downsampling module.

[0144] It should be noted that wavelet transform is a signal processing method that decomposes the image of the ampoule bottle to be detected into representation forms of different scales and resolutions, thereby removing redundant information while retaining important features.

[0145] In specific implementation, the image of the ampoule bottle to be detected is decomposed into a series of wavelet coefficients. Each wavelet coefficient corresponds to information in a different frequency band. After decomposition, wavelet features are obtained. These wavelet features contain the key features of the original image and provide more detailed descriptions at different scales.

[0146] It should be noted that decomposing the image of the ampoule bottle to be detected through frequency domain analysis can not only achieve the downsampling operation, but also effectively retain the low-frequency information and detail features of the original image. At the same time, it has high computational efficiency and good versatility, can significantly improve the stability and accuracy of feature extraction, and enables the network to perform better when processing complex backgrounds and small targets.

[0147] In specific implementation, performing wavelet transform processing on the image of the ampoule bottle to be detected through the wavelet downsampling module to obtain wavelet features can be to select a suitable wavelet basis function and determine the number of decomposition layers, decompose the image into LL, LH, HL, and HH sub-bands, perform multi-level decomposition recursively layer by layer, select appropriate sub-bands for combination according to requirements, and perform normalization processing, and finally generate wavelet features containing multi-level features.

[0148] In a feasible implementation manner, step S22 may include steps A221 to A223: Step A221: Decompose the image of the ampoule bottle to be detected to obtain frequency information.

[0149] It should be noted that a suitable wavelet basis function and the number of decomposition layers can be determined in advance before decomposing the image; the number of layers for wavelet decomposition, the more layers, the finer details can be captured. Generally, it can be 4 layers, 5 layers, or can be selected according to the actual situation. This embodiment does not limit this.

[0150] Further, decomposing the image of the ampoule to be detected according to the wavelet basis function can be to perform one-level or multi-level two-dimensional discrete wavelet transform (2D-DWT) on the original image using the selected wavelet basis function. Each decomposition produces four sub-bands: LL (low-frequency approximation), LH (horizontal high-frequency detail), HL (vertical high-frequency detail), and HH (diagonal high-frequency detail). If more detailed analysis is needed, the next-level decomposition can be continued on the LL sub-band.

[0151] Step A222: Perform wavelet transform processing on the image of the ampoule to be detected based on the frequency information to obtain multi-frequency feature components.

[0152] It can be understood that the multi-frequency feature components can be information of different frequency bands extracted from the image of the ampoule to be detected through wavelet decomposition, which can reflect the characteristics of the image at different scales and directions.

[0153] It should be understood that the frequency information obtained after decomposition can be the sub-bands during each decomposition. Combine the selected sub-bands into a multi-frequency feature component set. The set can stack them together in time to form a multi-channel feature map, or weight and merge these sub-bands according to a certain strategy.

[0154] In this embodiment, in order to better detect whether there are defects (hook head defect, tilted head defect, flat head defect, bubble head defect, and foreign object defect) on the ampoule head and whether there are trace defects such as texture on the bottle body, the LH sub-band, HL sub-band, and HH sub-band are mainly selected.

[0155] In a specific implementation, the image of the ampoule to be detected with a resolution of H*W is decomposed by a low-pass filter H0 and a high-pass filter H1, which are respectively used to extract approximate and high-frequency information from the image. Based on the high-frequency information, wavelet transform is performed to generate four components: the approximate low-frequency component LL, and the high-frequency components in three directions: horizontal LH, vertical HL, and diagonal HH. Select the required components for feature merging, so as to comprehensively capture the multi-scale information of the feature map.

[0156] Step A223: Convolve the multi-frequency feature components to obtain the wavelet features after dimensionality reduction.

[0157] In a specific implementation, after extracting the feature of each group of components, the resolution of the feature map of each component can be reduced to (W / 2, W / 2), and the number of channels is increased from C to 4C. Then, discriminant features are extracted through the CBR (ConvolutionBatchReLU) conventional convolution module, and the required number of channels is restored to obtain the wavelet features.

[0158] In this embodiment, the information in different frequency bands (frequency information) is extracted by performing wavelet decomposition on the image of the ampoule bottle to be detected, and multi-frequency feature components are generated by using wavelet transform processing. By performing convolution operations on these multi-frequency feature components, key features are further extracted and integrated to generate wavelet features after dimensionality reduction, which can better capture subtle defects and structural features, reduce data dimensions and computational complexity, enhance the representational ability and robustness of features, provide high-quality and high-dimensional feature representations, and effectively improve the accuracy of subsequent defect detection and defect classification.

[0159] The above is only a feasible implementation manner of step S22 provided in this embodiment, and the specific implementation manner of step S22 in this embodiment is not specifically limited.

[0160] Step S23, using the depth feature and the wavelet feature as the initial feature map.

[0161] It can be understood that when sampling the ampoule bottle to be detected, depth downsampling and wavelet downsampling are performed simultaneously.

[0162] It should be noted that the depth downsampling and wavelet downsampling performed simultaneously obtain two sampling features. The depth feature can better retain detailed features, and the wavelet downsampling can filter out unimportant features. The two downsampling feature channels are connected to the convolutional layer, and the features of the two branches are fused to obtain a feature map with a size that meets the requirements of image output.

[0163] This embodiment provides a method for detecting ampoule bottle defects. By using a depth downsampling module and a wavelet downsampling module to extract depth features and wavelet features respectively and integrating them into an initial feature map, it can provide a richer and more comprehensive feature representation while maintaining key information, thus providing a good basis for subsequent feature detection and defect classification, and enabling more accurate detection of ampoule bottle defects.

[0164] It should be noted that the above examples are only for understanding this application and do not constitute a limitation to the method for detecting ampoule bottle defects in this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0165] This application also provides an ampoule bottle defect detection device. Please refer to Figure 8 , the ampoule bottle defect detection device includes: A preprocessing module 10, configured to obtain the original ampoule bottle image of the target ampoule bottle, and perform image correction on the original ampoule bottle image to obtain the ampoule bottle image to be detected. The image correction includes brightness equalization processing, bilateral filtering processing, and dilation correction processing; The sampling module 20 is configured to perform depth downsampling and wavelet downsampling on the image of the ampoule bottle to be detected synchronously, obtain depth features and wavelet features, and fuse the depth features and the wavelet features to obtain an initial feature map; The feature extraction module 30 is configured to divide the initial feature map into multiple feature convolution groups, and obtain a reference defect feature map according to the defect candidate regions of each feature convolution group; The defect recognition module 40 is configured to perform feature mapping on the reference defect feature map to obtain the defect image of the target ampoule bottle.

[0166] The ampoule bottle defect detection device provided by this application adopts the ampoule bottle defect detection method in the above embodiment, and can solve the technical problem that the current defect detection of ampoule bottles is not accurate enough. Compared with the prior art, the beneficial effects of the ampoule bottle defect detection device provided by this application are the same as those of the ampoule bottle defect detection method provided by the above embodiment, and other technical features in the ampoule bottle defect detection device are the same as those disclosed in the above embodiment method, which will not be elaborated here.

[0167] In one embodiment, the preprocessing module 10 is further configured to obtain the original ampoule bottle image of the target ampoule bottle, perform brightness equalization processing on the pixel points in the original ampoule bottle image to obtain the equalized ampoule bottle image; Perform bilateral filtering processing on the equalized ampoule bottle image to obtain the filtered ampoule bottle image; Perform dilation correction processing on each pixel point of the filtered ampoule bottle image to obtain the ampoule bottle image to be detected.

[0168] In one embodiment, the preprocessing module 10 is further configured to obtain the filtering range of each pixel point according to the positions of the pixel points in the equalized ampoule bottle image; Obtain the spatial proximity factor and pixel value similarity factor of each pixel point according to the filtering range and the pixel point position; Obtain a filtering factor according to the spatial proximity factor and the pixel value similarity factor; Perform bilateral filtering on each pixel point of the equalized ampoule bottle image according to the filtering factor to obtain the filtered ampoule bottle image.

[0169] In one embodiment, the preprocessing module 10 is further configured to obtain the pixel point mean value according to the filtered ampoule bottle image; Divide the pixel points in the filtered ampoule bottle image into dark pixel points and light pixel points according to the pixel point mean value; Perform positive dilation processing on the dark pixel points and negative dilation processing on the light pixel points to obtain the ampoule bottle image to be detected.

[0170] In one embodiment, the sampling module 20 is further configured to perform strided convolution on the image of the ampoule bottle to be detected to obtain depth features; perform wavelet transform processing on the image of the ampoule bottle to be detected to obtain wavelet features; Use the depth features and the wavelet features as initial feature maps.

[0171] In one embodiment, the sampling module 20 is further configured to perform image uniform cropping on the image of the ampoule bottle to be detected to obtain sub-feature maps; Perform feature mapping and dimension connection on the sub-feature maps to obtain sub-features to be output; Perform strided structure convolution on the sub-features to be output to obtain depth features.

[0172] In one embodiment, the feature extraction module 30 is further configured to generate preliminary defect candidate regions according to the initial feature maps; Perform regional boundary correction on the preliminary defect candidate regions to obtain selected defect candidate regions; Perform convolution on the selected defect candidate regions based on channel fixation to obtain reference defect feature maps.

[0173] The present application provides an ampoule bottle defect detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the ampoule bottle defect detection method in the first embodiment above.

[0174] Refer to the following Figure 9 , which shows a schematic structural diagram of an ampoule bottle defect detection device suitable for implementing the embodiments of the present application. The ampoule bottle defect detection device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 9 The ampoule bottle defect detection device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0175] As shown in Figure 9As shown, the ampoule defect detection device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the ampoule defect detection device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the ampoule defect detection device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an ampoule defect detection device having various systems, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems can be alternatively implemented or had.

[0176] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are executed.

[0177] The ampoule defect detection device provided by the present application adopts the ampoule defect detection method in the above embodiment, and can solve the technical problem that the current defect detection for ampoules is not accurate enough. Compared with the prior art, the beneficial effects of the ampoule defect detection device provided by the present application are the same as those of the ampoule defect detection method provided by the above embodiment, and other technical features in the ampoule defect detection device are the same as the features disclosed in the method of the previous embodiment, and will not be elaborated here.

[0178] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0179] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0180] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the ampoule bottle defect detection method in the above embodiments.

[0181] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0182] The above computer-readable storage medium can be included in the ampoule bottle defect detection device; it can also exist alone and not be assembled into the ampoule bottle defect detection device.

[0183] The above computer-readable storage medium carries one or more programs, which, when executed by an ampoule defect detection device, cause the ampoule defect detection device to: obtain an original ampoule image of a target ampoule, perform image correction on the original ampoule image to obtain an ampoule image to be detected; perform depth downsampling and wavelet downsampling on the ampoule image to be detected through the synchronous downsampling attention layer of the ampoule defect detection model to obtain an initial feature map; perform feature extraction on the initial feature map through the feature refinement and screening layer of the ampoule defect detection model to obtain a reference defect feature map; perform feature mapping on the reference defect feature map through the classification layer of the ampoule defect detection model to obtain a defect image of the target ampoule.

[0184] Computer program code for performing the operations of this application may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, execute as a stand-alone software package, execute partially on the user's computer and partially on a remote computer, or execute entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., by connecting through an Internet service provider using the Internet).

[0185] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0186] The modules involved in the embodiments of the present application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself under certain circumstances.

[0187] The readable storage medium provided by the present application is a computer-readable storage medium, and the computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned ampoule defect detection method, which can solve the technical problem that the current defect detection for ampoules is not accurate enough. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the ampoule defect detection method provided by the above embodiments, and will not be elaborated here.

[0188] The present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the ampoule defect detection method as described above are implemented.

[0189] The computer program product provided by the present application can solve the technical problem that the current defect detection for ampoules is not accurate enough. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the ampoule defect detection method provided by the above embodiments, and will not be elaborated here.

[0190] The above are only some embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural transformation made by using the content of the specification and drawings of the present application under the technical concept of the present application, or any direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A method for detecting defects in ampoules, characterized in that, The ampoule defect detection method includes: Obtain the original ampoule image of the target ampoule, perform image correction on the original ampoule image to obtain the ampoule image to be detected, and the image correction includes brightness equalization processing, bilateral filtering processing, and dilation correction processing; Simultaneously perform depth downsampling and wavelet downsampling on the ampoule image to be detected to obtain depth features and wavelet features, and fuse the depth features and the wavelet features to obtain an initial feature map; Divide the initial feature map into multiple feature convolution groups, and obtain a reference defect feature map according to the defect candidate regions of each feature convolution group; Perform feature mapping on the reference defect feature map to obtain the defect image of the target ampoule.

2. The ampoule defect detection method according to claim 1, wherein The steps of obtaining the original ampoule image of the target ampoule, performing image correction on the original ampoule image, and obtaining the ampoule image to be detected include: Obtain the original ampoule image of the target ampoule, perform brightness equalization processing on the pixel points in the original ampoule image to obtain the ampoule image after equalization; Perform bilateral filtering processing on the ampoule image after equalization to obtain the ampoule image after filtering; Perform dilation correction processing on each pixel point of the ampoule image after filtering to obtain the ampoule image to be detected.

3. The ampoule defect detection method according to claim 2, wherein The steps of performing bilateral filtering on the ampoule image after equalization to obtain the ampoule image after filtering include: Obtain the filtering range of each pixel point according to the position of each pixel point of the ampoule image after equalization; Obtain the spatial proximity factor and pixel value similarity factor of each pixel point according to the filtering range and the pixel point position; Obtain the filtering factor according to the spatial proximity factor and the pixel value similarity factor; Perform bilateral filtering on each pixel point of the ampoule image after equalization according to the filtering factor to obtain the ampoule image after filtering.

4. The ampoule defect detection method according to claim 2, characterized in that, The steps of performing dilation correction processing on each pixel point of the ampoule image after filtering to obtain the ampoule image to be detected include: Obtain the pixel point mean value according to the ampoule image after filtering; Divide the pixel points in the ampoule image after filtering into dark pixel points and light pixel points according to the pixel point mean value; Perform positive dilation processing on the dark pixel points and negative dilation processing on the light pixel points to obtain the ampoule image to be detected.

5. The ampoule defect detection method according to claim 1, characterized in that, The steps of simultaneously performing depth downsampling and wavelet downsampling on the ampoule image to be detected to obtain depth features and wavelet features, and fusing the depth features and the wavelet features to obtain an initial feature map include: Perform strided convolution on the ampoule image to be detected to obtain depth features; Perform wavelet transform processing on the ampoule image to be detected to obtain wavelet features; Use the depth features and the wavelet features as the initial feature map.

6. The ampoule defect detection method according to claim 5, wherein, The steps of performing strided convolution on the ampoule image to be detected through the depth downsampling module to obtain depth features include: Perform uniform image cropping on the ampoule image to be detected to obtain a sub-feature map; Perform feature mapping and dimension connection on the sub-feature map to obtain the sub-feature to be output; Perform strided structure convolution on the to-be-output sub-features to obtain depth features.

7. The ampoule defect detection method according to claim 1, wherein The step of dividing the initial feature map into multiple feature convolution groups and obtaining a reference defect feature map according to the defect candidate regions of each feature convolution group includes: Divide the initial feature map into multiple feature convolution groups, and generate preliminary defect candidate regions according to the feature convolution for defect localization; Perform regional boundary correction on the preliminary defect candidate regions to obtain selected defect candidate regions; Based on channel fixation, perform separable convolution on the selected defect candidate regions of each feature convolution group to obtain a reference defect feature map.

8. An ampoule defect detection device, characterized in that, The ampoule bottle defect detection device includes: A preprocessing module, configured to obtain the original ampoule bottle image of the target ampoule bottle, perform image correction on the original ampoule bottle image to obtain the to-be-detected ampoule bottle image, and the image correction includes brightness equalization processing, bilateral filtering processing, and dilation correction processing; A sampling module, configured to simultaneously perform depth downsampling and wavelet downsampling on the to-be-detected ampoule bottle image to obtain depth features and wavelet features, and fuse the depth features and the wavelet features to obtain an initial feature map; A feature extraction module, configured to divide the initial feature map into multiple feature convolution groups and obtain a reference defect feature map according to the defect candidate regions of each feature convolution group; A defect recognition module, configured to perform feature mapping on the reference defect feature map to obtain the defect image of the target ampoule bottle.

9. An ampoule defect detection device, characterized in that, The device includes: a memory, a processor, and an ampoule bottle defect detection program stored on the memory and executable on the processor, and the ampoule bottle defect detection program is configured to implement the ampoule bottle defect detection method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, An ampoule bottle defect detection program is stored on the storage medium, and when the ampoule bottle defect detection program is executed by the processor, it implements the ampoule bottle defect detection method according to any one of claims 1 to 7.

Images