Ampoule defect detection method, device, equipment and storage medium

Abstract

The application discloses an ampoule defect detection method, device, equipment and storage medium, including: acquiring an original ampoule image of a target ampoule, correcting the original ampoule image to obtain a to-be-detected ampoule image; depth down-sampling and wavelet down-sampling the to-be-detected ampoule image, fusing depth features and wavelet features to obtain an initial feature map; dividing the initial feature map into a plurality of feature convolution groups, obtaining a reference defect feature map according to defect candidate regions of the feature convolution groups; and performing feature mapping on the reference defect feature map to obtain a defect image of the target ampoule. The method highlights defects of the ampoule in the image by correcting, sampling and feature extraction on the original ampoule image, performs wavelet sampling and depth sampling on the processed image simultaneously, pays attention to detailed features, reduces the data volume while retaining key information, accurately identifies tiny defects on the ampoule, and more quickly and accurately identifies overall defects of the ampoule.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an ampoule defect detection method, device, equipment and storage medium. BACKGROUND

[0002] With the rapid development of the pharmaceutical industry, as an important link to ensure the quality and safety of drugs, the quality control in the production process of drug packaging is particularly critical. Ampoules, as one of the main packaging forms of liquid drugs, are widely used due to their good sealing performance, easy storage and transportation. The quality of ampoules not only relates to the safety of drugs, but also directly affects the credibility and market competitiveness of pharmaceutical enterprises. Therefore, it is extremely important to effectively detect and control the appearance defects of ampoules.

[0003] The traditional way of ampoule appearance defect detection mainly relies on manual visual inspection. Although this method can find some obvious appearance defects of ampoules to some extent, with the expansion of the scale of pharmaceutical enterprises, manual visual inspection is inefficient and difficult to meet the production demand of ampoules. In addition, different detection personnel may have different judgment standards, which can easily lead to inconsistent defect detection results and defect detection errors. In addition, long-term repetitive work can easily lead to fatigue of the detection personnel, thereby affecting the accuracy and efficiency of the detection.

[0004] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0005] The main purpose of the present application is to provide an ampoule defect detection method, device, equipment and storage medium, which aims to solve the technical problem that the current defect detection of ampoules is not accurate enough.

[0006] To achieve the above purpose, the present application provides an ampoule defect detection method, which comprises:

[0007] An original ampoule image of a target ampoule is obtained, and image correction is performed on the original ampoule image to obtain a to-be-detected ampoule image. The image correction includes brightness equalization processing, bilateral filtering processing and dilation correction processing.

[0008] The to-be-detected ampoule image is simultaneously subjected to depth downsampling and wavelet downsampling to obtain depth features and wavelet features, and the depth features and the wavelet features are fused to obtain an initial feature map.

[0009] The initial feature map is divided into a plurality of feature convolution groups, and a reference defect feature map is obtained according to the defect candidate regions of each feature convolution group.

[0010] Feature mapping is performed on the reference defect feature map to obtain a defect image of the target ampoule.

[0011] In an embodiment, the step of obtaining an original ampoule image of the target ampoule and performing image correction on the original ampoule image to obtain a to-be-detected ampoule image comprises:

[0012] An original ampoule image of the target ampoule is obtained, and brightness equalization processing is performed on the pixel points in the original ampoule image to obtain an equalized ampoule image.

[0013] Bilateral filtering processing is performed on the equalized ampoule image to obtain a filtered ampoule image.

[0014] Each pixel point of the filtered ampoule image is subjected to inflation correction processing to obtain a to-be-detected ampoule image.

[0015] In an embodiment, the step of performing bilateral filtering on the equalized ampoule image to obtain a filtered ampoule image comprises:

[0016] A filtering range of each pixel point is obtained according to the position of each pixel point of the equalized ampoule image.

[0017] A spatial proximity factor and a pixel value similarity factor of each pixel point are obtained according to the filtering range and the position of the pixel point.

[0018] A filtering factor is obtained according to the spatial proximity factor and the pixel value similarity factor.

[0019] Each pixel point of the equalized ampoule image is subjected to bilateral filtering according to the filtering factor to obtain a filtered ampoule image.

[0020] In an embodiment, the step of performing inflation correction processing on each pixel point of the filtered ampoule image to obtain a to-be-detected ampoule image comprises:

[0021] A pixel point mean value is obtained according to the filtered ampoule image.

[0022] The pixel points in the filtered ampoule image are divided into dark pixel points and light pixel points according to the pixel point mean value.

[0023] The dark pixel points are subjected to positive inflation processing, and the light pixel points are subjected to negative inflation processing to obtain a to-be-detected ampoule image.

[0024] In an embodiment, the step of synchronously performing depth down-sampling and wavelet down-sampling on the ampoule image to be detected to obtain a depth feature and a wavelet feature, and fusing the depth feature and the wavelet feature to obtain an initial feature map comprises:

[0025] performing cross-step convolution on the ampoule image to be detected to obtain a depth feature;

[0026] performing wavelet transform processing on the ampoule image to be detected to obtain a wavelet feature;

[0027] taking the depth feature and the wavelet feature as an initial feature map.

[0028] In an embodiment, the step of performing cross-step convolution on the ampoule image to be detected by the depth down-sampling module to obtain a depth feature comprises:

[0029] performing image uniform clipping on the ampoule image to be detected to obtain a sub-feature map;

[0030] performing feature mapping and dimension connection on the sub-feature map to obtain a sub-feature to be output;

[0031] performing cross-step structure convolution on the sub-feature to be output to obtain a depth feature.

[0032] In an embodiment, the step of dividing the initial feature map into a plurality of feature convolution groups and obtaining a reference defect feature map according to defect candidate regions of each feature convolution group comprises:

[0033] dividing the initial feature map into a plurality of feature convolution groups, and generating a preliminary defect candidate region according to defect positioning of the feature convolution;

[0034] performing region boundary correction on the preliminary defect candidate region to obtain a selected defect candidate region;

[0035] performing separable convolution on the selected defect candidate region of each feature convolution group based on channel fixing to obtain a reference defect feature map.

[0036] In addition, to achieve the above-mentioned purpose, the present application further provides an ampoule defect detection device, which comprises:

[0037] a preprocessing module configured to acquire an original ampoule image of a target ampoule, and perform image correction on the original ampoule image to obtain an ampoule image to be detected, wherein the image correction comprises brightness equalization processing, bilateral filtering processing and dilation correction processing;

[0038] a sampling module configured to perform depth subsampling and wavelet subsampling on the image of the ampoule to be detected synchronously to obtain a depth feature and a wavelet feature, and to fuse the depth feature and the wavelet feature to obtain an initial feature map;

[0039] a feature extraction module configured to divide the initial feature map into a plurality of feature convolution groups, and to obtain a reference defect feature map according to a defect candidate region of each feature convolution group;

[0040] a defect recognition module configured to perform feature mapping on the reference defect feature map to obtain a defect image of the target ampoule.

[0041] In addition, to achieve the above-mentioned purpose, the present application also provides an ampoule defect detection device, which comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the ampoule defect detection method as described above.

[0042] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the ampoule defect detection method as described above.

[0043] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the ampoule defect detection method as described above.

[0044] The one or more technical solutions provided by the present application have at least the following technical effects: by performing image processing such as correction, sampling, and feature extraction on the original image collected by the ampoule, the defects of the ampoule in the image are highlighted; according to the pre-processed image, depth sampling and wavelet sampling are performed synchronously, more detailed features are focused on, the data volume is effectively reduced while the key information is preserved, the micro defects on the ampoule are more accurately identified, and the overall defect problem of the ampoule is more quickly and accurately identified. BRIEF DESCRIPTION OF DRAWINGS

[0045] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the application.

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0047] Figure 1 The flowchart provided for the ampoule defect detection method embodiment one of the present application;

[0048] Figure 2 The ampoule defect detection device schematic diagram provided for the ampoule defect detection method embodiment one of the present application;

[0049] Figure 3 The ampoule image acquisition schematic diagram provided for the ampoule defect detection method embodiment one of the present application;

[0050] Figure 4 The lightweight convolutional network structure schematic diagram provided for the ampoule defect detection method embodiment one of the present application;

[0051] Figure 5 The region boundary correction neural network structure schematic diagram provided for the ampoule defect detection method embodiment one of the present application;

[0052] Figure 6 The flowchart provided for the ampoule defect detection method embodiment two of the present application;

[0053] Figure 7 The flowchart provided for the ampoule defect detection method embodiment three of the present application;

[0054] Figure 8 The module structure schematic diagram of the ampoule defect detection device of the present application embodiment;

[0055] Figure 9 The device structure schematic diagram of the hardware running environment involved in the ampoule defect detection method of the present application embodiment. DETAILED DESCRIPTION

[0056] It should be understood that the specific embodiments described herein are merely intended to explain the technical solutions of the present application, and are not intended to limit the present application.

[0057] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings of the specification and specific embodiments.

[0058] The main solution of the present application embodiment is: obtaining the original ampoule image of the target ampoule, correcting the original ampoule image to obtain the ampoule image to be detected, depth downsampling and wavelet downsampling the ampoule image to be detected, fusing the depth features and wavelet features to obtain the initial feature map, dividing the initial feature map into multiple feature convolution groups, obtaining the reference defect feature map according to the defect candidate region of the feature convolution group, and mapping the reference defect feature map to obtain the defect image of the target ampoule.

[0059] In the embodiment, for convenience of description, the following is described taking the ampoule defect detection device as the execution subject.

[0060] With the rapid development of the pharmaceutical industry, as an important link to ensure the quality and safety of drugs, the quality control of ampoules is particularly crucial. Ampoules, as one of the main packaging forms of liquid drugs, are widely used due to their good sealing performance, convenience of storage and transportation. The quality of ampoules not only relates to the safety of drugs, but also directly affects the credibility and market competitiveness of pharmaceutical enterprises. Therefore, it is extremely important to effectively detect and control the appearance defects of ampoules.

[0061] The traditional way of ampoule appearance defect detection mainly relies on manual visual inspection. Although this method can find some obvious appearance defects of ampoules to some extent, with the expansion of the scale of pharmaceutical enterprises, manual visual inspection is inefficient and cannot meet the production demand of ampoules. In addition, different detection personnel may have different judgment standards, which can easily lead to inconsistent defect detection results and defect detection errors. In addition, long-term repetitive work can easily lead to fatigue of the detection personnel, thereby affecting the accuracy and efficiency of the detection.

[0062] The present application provides a solution. By correcting, sampling and feature extraction of the original ampoule image, the defects of the ampoule in the image are highlighted. According to the processed image, wavelet sampling and deep sampling are simultaneously performed, the detailed features are focused on, the data volume is reduced while the key information is retained, the tiny defects on the ampoule are accurately identified, and the overall defects of the ampoule are more quickly and accurately identified.

[0063] From the above embodiment, it can be seen that the present application discloses an ampoule defect detection method, device, equipment and storage medium, relating to the technical field of image processing, comprising: acquiring an original ampoule image of a target ampoule, performing image correction on the original ampoule image to obtain a to-be-detected ampoule image; depth downsampling and wavelet downsampling the to-be-detected ampoule image, fusing the depth features and wavelet features to obtain an initial feature map; dividing the initial feature map into a plurality of feature convolution groups, obtaining a reference defect feature map according to the defect candidate regions of the feature convolution groups; and performing feature mapping on the reference defect feature map to obtain a defect image of the target ampoule. The method highlights the defects of the ampoule in the image by correcting, sampling and feature extraction of the original ampoule image, performs wavelet sampling and deep sampling simultaneously according to the processed image, focuses on the detailed features, reduces the data volume while retaining the key information, accurately identifies the tiny defects on the ampoule, and more quickly and accurately identifies the overall defects of the ampoule.

[0064] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, an ampoule defect detection device, etc. The following takes the ampoule defect detection device as an example to describe the embodiment and the following embodiments.

[0065] Based on this, the embodiment of the present application provides an ampoule defect detection method, which is described with reference to Figure 1 , Figure 1 The flowchart of the first embodiment of the ampoule defect detection method of the present application is shown in the figure.

[0066] In the embodiment, the ampoule defect detection method includes steps S10-S40:

[0067] Step S10, obtaining an original ampoule image of a target ampoule, performing image correction on the original ampoule image to obtain a to-be-detected ampoule image, the image correction including brightness equalization processing, bilateral filtering processing and dilation correction processing.

[0068] It can be understood that the target ampoule can be an ampoule that needs to be detected for defects, which can be a single ampoule image or multiple ampoule images. If it is a single ampoule image, a single ampoule image can be used for training when training the ampoule defect detection model.

[0069] It should be understood that the original ampoule image can be an image directly collected by an image collection device without image processing. In the embodiment, an ampoule defect detection device is also involved, which can include an ampoule detection point position frame, an industrial computer and a production line interaction module. In specific implementation, reference can be made to Figure 2 The ampoule detection point position frame in the schematic diagram of the ampoule defect detection device includes a detection point position frame body 1. In the embodiment, the detection point position frame body 1 is arranged on an ampoule production line 2. For details, reference can be made to Figure 3 The ampoule production line 2 conveys the ampoules to be detected. The detection point position frame body 1 is provided with a mounting plate 101 of an image collection unit 3. In the embodiment, the image collection unit 3 adopts a high-definition industrial camera, Figure 2 The camera body 301 and the lens 302 of the high-definition industrial camera are shown in

[0070] It should be noted that in the process of image acquisition of the target ampoule, because the acquisition angle may cause the light and the definition to be affected, the detection effect of directly using the acquired image for defect detection is not ideal, and the image can be corrected first, which is beneficial to the accuracy of the image defect detection in the later stage.

[0071] It should be emphasized that the detection of the original ampoule image can include brightness equalization processing, filtering processing, and expansion correction of the original ampoule image. It can be understood that the image obtained after the correction processing is the ampoule image to be detected.

[0072] Step S20, the depth feature and the wavelet feature are fused to obtain an initial feature map.

[0073] It can be understood that the ampoule defect detection model can be a model that has been trained to accurately detect defects.

[0074] It should be understood that the initial feature map can be a fusion map of the depth down-sampled feature map and the wavelet down-sampled feature map, and it can also be understood that the initial feature map includes the feature map obtained by depth down-sampling and the feature map obtained by wavelet down-sampling.

[0075] It should be noted that the synchronous down-sampling attention layer can be an image processing combined with an attention mechanism and a down-sampling technology, wherein the down-sampling is performed simultaneously using two down-sampling methods.

[0076] In a feasible implementation, step S20 further includes steps A201-A204 before step S20:

[0077] Step A201, a preset defect detection model and a training data set are obtained, the preset defect detection model includes a synchronous down-sampling attention layer, a feature refining and screening layer, a classification layer, and a generative adversarial layer.

[0078] It should be noted that the generative adversarial layer includes a generative adversarial network, which continuously generates new training data according to the training data set, and optimizes according to the detection result of the new training data.

[0079] It is worth noting that the training of the preset defect detection model has a high demand for a large amount of high-quality labeled data, and the traditional machine learning method relies on manually designed features, which increases the cost of data acquisition and processing, and it is difficult to obtain diverse and comprehensive defect sample data, which limits the generalization ability and application promotion of the model. Before training the multi-task defect detection model, the image sample set is formed by collecting multiple view images of ampoule bottles, and the image sample set is expanded by using data enhancement and transfer learning; in this embodiment, the generative adversarial network (GAN) is introduced for data enhancement to generate diverse and high-quality defect samples, enrich the training set, and further improve the generalization ability and accuracy of the detection model.

[0080] It should be understood that the defective ampoule bottles and the training data set generated by each defective ampoule bottle image can be acquired by the image acquisition device.

[0081] It can be understood that the synchronous down-sampling attention layer combines the attention mechanism and the down-sampling technology, aiming to focus on the most important part of the image and reduce the spatial resolution of the image to reduce the computational complexity; the feature refining and screening layer extracts useful features from the image after synchronous down-sampling processing, and further screens and optimizes these features; the classification layer maps the feature map after the feature refining and screening to a specific class label for classification tasks.

[0082] It should be emphasized that the generative adversarial layer is used to optimize the model data during the training process, which facilitates better training of the model. The ampoule defect detection model after the model is trained can not have the generative adversarial layer.

[0083] Step A202, based on the training data set and the generative adversarial layer, a target data set is obtained, and the synchronous down-sampling attention layer, the feature refining and screening layer, and the classification layer are trained according to the target data set to obtain a training result.

[0084] It should be noted that the target data set based on the training data set and the generative adversarial layer can be that the generative adversarial network (GAN) is first trained, the generator (Generator) in the generative adversarial network generates synthetic images similar to real defect images, starting from random noise or low-resolution images, and gradually generates high-resolution and realistic defect images. The discriminator distinguishes between generated images and real defect images, and through a feedback mechanism, the discriminator helps the generator to continuously improve the generation effect.

[0085] Further, a large number of synthetic defect images are generated using the trained generator. The synthetic images can supplement the deficiencies in the original training data set, increase the diversity and size of the data set, and mix the generated synthetic images with the original training data set to form a new and more abundant target data set that contains not only the actually collected defect images but also the high-quality defect images generated by the generator.

[0086] It should be emphasized that the training result can be the result of defect detection on the ampoule images in the target data set by the preset defect detection model, for example, the detection result can include the defects in the image, the accuracy of the defect category, etc.

[0087] Step A203, calculate the cross-entropy loss function according to the training result, and perform back propagation on the synchronous down-sampling attention layer, the feature refining screening layer, and the classification layer according to the cross-entropy loss function to obtain an optimized defect detection model.

[0088] It should be noted that the cross-entropy loss function can be calculated according to the following formula:

[0089]

[0090] where i represents different defect categories of the to-be-detected ampoule image, Y i represents the probability that the model predicts that the image has i-th defect, y i represents whether the image defect is the i-th defect, which can be represented by 1 or 0.

[0091] It should be noted that the cross-entropy loss function can be calculated according to the following formula:

[0092] In specific implementation, when training the preset ampoule defect detection model, the target data set is divided into a training set and a test set. Based on the back propagation method, the training set is used to train the preset ampoule defect detection model, and the model parameters of the preset ampoule defect detection model are adjusted by an optimizer. In the training loop, the training set is divided into several batches. For each batch of training set data, the gradient of the optimizer is first cleared, then the training set data is input into the preset ampoule defect detection model, the cross-entropy loss function is calculated, the cross-entropy loss function is back propagated, and the model parameters are updated according to the gradients calculated by the back propagation. When the cross-entropy loss function converges, a trained preset ampoule defect detection model is obtained; and the test set is used to test the preset ampoule defect detection model.

[0093] Step A203, pruning and quantizing the optimized defect detection model to obtain an ampoule defect detection model.

[0094] It should be noted that pruning and quantizing the optimized defect detection model can be pruning and quantizing the parameters in the model, wherein the model parameters can include the convolution kernel size, the number of convolution kernels, the step, the boundary mode, the bias weight, etc.

[0095] It should be understood that pruning refers to simplifying the model structure by removing unimportant weights or neurons in the model, thereby reducing the amount of calculation and memory occupation; quantization refers to converting the floating-point number weights and activation values in the model to low-precision representation (such as 8-bit integer), to reduce the storage requirement and computational complexity of the model.

[0096] It should be emphasized that after the multi-task defect detection model is trained, the model parameters of the multi-task defect detection model are pruned and quantized. The pruning is to reduce unnecessary weights and connections in the model to reduce the calculation cost; the quantization processing compresses or simplifies the model to reduce the calculation amount and storage space of the model, thereby improving the efficiency and speed of the model.

[0097] In this embodiment, the training data is enhanced by the generation of the adversarial layer, the enhanced images are mixed to train the model to obtain better training results, and after the model is trained, the calculation cost is reduced by pruning and quantization processing, the calculation amount and storage space of the model are reduced, and the detection efficiency and speed of the model are effectively improved under the condition of ensuring the detection accuracy of the model.

[0098] The above is only an implementation manner provided by the present embodiment before step S20, and the present embodiment does not specifically limit the specific implementation manner before step S20.

[0099] Step S30, dividing the initial feature map into a plurality of feature convolution groups, and obtaining a reference defect feature map according to the defect candidate regions of each feature convolution group.

[0100] It can be understood that the feature extraction of the initial feature map to obtain the reference defect feature map can be performed by the feature refining selection layer of the ampoule defect detection model, and the reference defect feature map can be a feature map with defects extracted from the initial feature map, which contains the key features that best represent the defects.

[0101] It should be noted that the feature refining selection layer can be a lightweight convolutional network, which can be referred to as Figure 4 , Figure 4The feature extraction network in the structure diagram of the light-weight convolutional network comprises a first convolutional layer Conv1, a maximum pooling layer MaxPooling1, a second convolutional layer Conv2 and a third convolutional layer Conv3 connected in sequence, and a ReLU activation function layer is connected behind each of the first convolutional layer Conv1, the second convolutional layer Conv2 and the third convolutional layer Conv3. After the image of the ampoule is subjected to convolutional operations in the first convolutional layer Conv1, the maximum pooling layer MaxPooling1, the second convolutional layer Conv2 and the third convolutional layer Conv3 in sequence, the image features are extracted to generate a feature map.

[0102] In an implementable embodiment, the step S30 can comprise steps A31-A33.

[0103] In step A31, the initial feature map is divided into a plurality of feature convolution groups, and a preliminary defect candidate region is generated according to defect positioning of the feature convolution.

[0104] It can be understood that the preliminary defect candidate region generated according to the initial feature map can be generated by a feature refining and screening layer of the ampoule defect detection model.

[0105] It should be noted that the initial feature map is subjected to grouping convolution, and when the convolution is divided into G groups, the parameter amount of the convolution kernel is 1 / G of the original amount. A plurality of preliminary candidate regions are obtained by respectively performing defect positioning according to the features extracted by each group, and further screening of the preliminary candidate regions is performed. The selected defect candidate regions of the plurality of feature groups are subjected to convolution to obtain the final output reference curve feature map.

[0106] It should be noted that the preliminary defect candidate region generated according to the fused image can refer to the output end of Figure 4 , Figure 4 The output end comprises three parts, the first part classification in the figure represents preliminary classification, the second part Bounding Box Regression represents preliminary bounding box regression, and the third part Localization represents preliminary positioning. The preliminary defect candidate region is generated by the feature extraction network shown in Figure 3 .

[0107] In the target detection task, preliminary classification refers to the process of class prediction for candidate regions or anchor boxes in an image, which can automatically extract features from the image and determine the class to which each candidate region belongs based on these features; bounding box regression aims to adjust the position and size of the candidate regions generated by the preliminary classification stage to make them more accurately match the actual position of the target object, thereby obtaining more accurate target positioning. This step is crucial to improve the accuracy of target detection; preliminary positioning refers to determining the approximate position of the target in the image to achieve more accurate target positioning.

[0108] In a specific implementation, the input image size is 12*12*3, the first convolutional layer Conv1 uses a 3x3 convolutional kernel with a step size of 1, outputs a feature map with 10 channels, the second convolutional layer Conv2 uses a 3x3 convolutional kernel with a step size of 1, outputs a feature map with 16 channels, the third convolutional layer Conv3 uses a 3x3 convolutional kernel with a step size of 1, outputs a feature map with 32 channels, the maximum pooling layer, the Max Pooling1 in the figure uses a 2x2 pooling window with a step size of 2, which can reduce the computational complexity of the upper layer, the feature map output by the convolutional layer often contains a lot of data, the maximum pooling layer selects the maximum value in its pooling window as the output, reducing the spatial dimension of the feature map, thereby reducing the computational load of the subsequent layer (upper layer), reducing the resources required for calculation, in addition, ensuring translational invariance, the maximum pooling layer is implemented on a 2x2 window, in the possible translational configuration, it can accurately produce the same result as the layer connected after the convolutional layer, meaning that when the target object is translated in the image within a certain range, the result obtained after the maximum pooling layer has similarity, this feature helps in image recognition tasks, when the target object has a small amount of translation in the image, the model can still recognize it well.

[0109] Step A32, performing region boundary correction on the preliminary defect candidate region to obtain a selected defect candidate region.

[0110] It can be understood that performing region boundary correction on the preliminary defect candidate region can be further processing the preliminary defect candidate region to adjust its boundary to obtain a more accurate defect position, thereby improving the accuracy of the defect candidate region and making it more accurately cover the actual defect. The candidate region after boundary correction has high positional accuracy.

[0111] It should be noted that region boundary correction can be applying geometric transformation (such as translation, scaling, rotation, etc.) to the preliminary defect candidate region through set transformation, which can adjust its shape and size to better match the actual defect position; it can also be fine-tuning the candidate region in combination with the surrounding context information; this embodiment does not limit it, and it can be selected or adjusted according to actual conditions.

[0112] In a specific implementation, the region boundary correction is a convolutional neural network with a complexity greater than that of the feature extraction network, and a region boundary correction neural network structure diagram can refer to Figure 5 , Figure 5 Conv4 represents the fourth convolutional layer, Max Pooling represents the maximum pooling layer, Conv5 represents the fifth convolutional layer, Max Pooling3 represents the maximum pooling layer, Conv6 represents the sixth convolutional layer, and the full connection layer. The fine selection network performs fine selection and preliminary candidate region boundary correction on the preliminary defect candidate region to generate a selected defect candidate region.

[0113] Step A33, based on the fixed channel, the selected defect candidate region is convolved to obtain a reference defect feature map.

[0114] It can be understood that the fixed number of channels can be in the convolution process, so that the channel (or feature dimension) is specially concerned or remains unchanged, which can better preserve the key features and reduce unnecessary calculations.

[0115] It should be understood that the convolution can further extract and optimize the features in the selected defect candidate region to generate the final reference defect feature map.

[0116] In this embodiment, the feature refinement and selection layer of the ampoule defect detection model generates a preliminary defect candidate region from the initial feature map, then performs boundary correction on these regions to obtain more accurate selected defect candidate regions, and finally extracts high-quality reference defect feature maps based on the fixed channel convolution operation. The accuracy and reliability of the defect detection are effectively improved, ensuring that the model can accurately identify and locate the subtle defects in the ampoule, while reducing false positives and computational complexity. The final generated reference defect feature map provides a basis for subsequent defect classification.

[0117] The above is only an implementation manner of step S30 provided by the present embodiment, and the present embodiment does not specifically limit the specific implementation manner of step S30.

[0118] Step S40, performing feature mapping on the reference defect feature map to obtain a defect image of the target ampoule.

[0119] It should be noted that the feature mapping of the reference defect feature map to obtain the defect image of the target ampoule can be performed by the classification layer of the ampoule defect detection model.

[0120] It should be further explained that the classification layer performs a nonlinear transformation on the input feature map through a series of learning parameters (such as weights and biases), and each output unit corresponds to a type of defect or background category. By comparing the activation values of each unit, the most likely defect type is determined. Based on the results of feature mapping, the location of the defect in the original ampoule image is located, and the spatial information of the feature map is reversely mapped back to the original image coordinate system to accurately identify the location of the defect.

[0121] The embodiment provides an ampoule defect detection method. By correcting, sampling and feature extraction on an original ampoule image, the defects of the ampoule in the image are highlighted. According to the processed image, wavelet sampling and deep sampling are simultaneously performed, the detailed features are focused on, the data amount is reduced while the key information is retained, the tiny defects on the ampoule are accurately identified, and the overall defects of the ampoule are more quickly and accurately identified.

[0122] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above embodiment one can be referred to the above introduction, and will not be described in detail. On this basis, please refer to Figure 6 , step S10 further includes steps S11-S13:

[0123] In step S11, an original ampoule image of a target ampoule is acquired, and a brightness equalization processing is performed on the pixel points in the original ampoule image to obtain an equalized ampoule image.

[0124] It can be understood that the brightness equalization processing is mainly performed on the pixel points in the original ampoule image to improve the contrast and brightness distribution of the image, so that the details in the image are more clearly visible.

[0125] It should be understood that the brightness equalization can re-distribute the gray values in the image to improve the contrast of the image, convert the gray histogram (i.e. the frequency of each gray level) of the original image into a uniformly distributed form, so that the overall brightness range of the image is expanded.

[0126] In a specific implementation, first, the number of occurrences of each gray level in the image is counted, and a gray histogram of the original image is drawn according to the data. Based on the original histogram, a cumulative distribution function is calculated, which represents the cumulative proportion of all pixel numbers less than or equal to a certain gray value. The cumulative distribution function is used as a transformation function; each pixel value in the original image is mapped to a new gray value, which is uniformly distributed in the entire gray range, which can increase the dynamic range of the image and thus improve the contrast; finally, a new image is generated using the above mapping relationship, in which the gray value of each pixel is replaced by the corresponding equalized gray value.

[0127] Step S12, bilateral filtering is performed on the equalized ampoule image to obtain a filtered ampoule image.

[0128] It can be understood that the bilateral filtering is a nonlinear filtering method which combines spatial proximity and pixel value similarity to smooth the image. This method can reduce noise while maintaining clear edges.

[0129] It should be understood that after the brightness equalization processing of the ampoule image, the bilateral filtering can help to further improve the image quality. Equalization may enhance the contrast of some regions, while it may also amplify the noise in the image. Bilateral filtering can effectively reduce these noises, making the image cleaner, and thus more accurately identify defects in the image.

[0130] In a possible implementation, step S12 can include steps A121-A124:

[0131] Step A121, obtaining a filtering range of each pixel point according to the position of each pixel point of the equalized ampoule image.

[0132] It should be noted that the filtering range can be determined according to a predetermined filtering radius. The filtering range is determined according to the filtering radius. In simple terms, the region around each pixel point with a distance of no more than a predetermined distance from the pixel point is taken as the filtering range.

[0133] Further, the predetermined distance can be calculated according to the filtering radius, specifically l=(2r+1)*(2r+1), where l represents the predetermined distance, and r represents the filtering radius; the range with a distance less than l from each pixel point is taken as the filtering range.

[0134] Step A122, obtaining a spatial proximity factor and a pixel value similarity factor of each pixel point according to the filtering range and the position of the pixel point.

[0135] It should be noted that the calculation of the spatial proximity factor can refer to the following formula:

[0136]

[0137] where i, j represent the coordinate points of each pixel point position; k, l represent the coordinate points of adjacent pixel point positions; σ s represents a spatial standard deviation parameter, which controls the spatial expansion range of the filter.

[0138]

[0139] wherein I(i, j) represents the gray value of each pixel point, and is a color vector if it is a color image; I(k, l) represents the gray value or color vector of the adjacent pixel point; σ r represents the standard deviation parameter of the pixel value, and determines the sensitivity of the filter to the difference of the pixel value.

[0140] Step A123, obtaining a filter factor according to the spatial proximity factor and the pixel value similarity factor.

[0141] It should be noted that the filter factor can be obtained by multiplying the spatial proximity factor and the pixel value similarity factor, and the specific formula can be referred to as follows:

[0142]

[0143] wherein H(i, j, k, l) represents the filter factor.

[0144] Step A124, performing bilateral filtering on each pixel point of the equalized ampoule image according to the filter factor to obtain a filtered ampoule image.

[0145] It should be noted that the calculation formula of the pixel point after performing bilateral filtering on each pixel point can be referred to as follows:

[0146]

[0147] wherein g(i, j) represents the filtered pixel point, S(i, j) represents the filtering range, and f(k, l) represents the adjacent pixel point.

[0148] It can be understood that the bilateral filtering on each pixel point in the equalized ampoule image obtains the filtered pixel to constitute the filtered ampoule image.

[0149] It should be noted that it is very important to keep the edge clear when detecting whether the ampoule has cracks, breakage or other defects. The bilateral filtering can smooth the image while ensuring that these key features are not affected. By removing unnecessary noise and keeping key features, the image processed by the bilateral filtering is more conducive to precise automated analysis, such as defect detection or content recognition.

[0150] In the embodiment, the bilateral filter adjusts the value of each pixel in the equalized ampoule image by considering the spatial proximity and the pixel value similarity at the same time. It not only considers the physical distance between the pixels, but also considers the color or brightness difference between them, which can effectively remove noise and smooth the image, and also can well keep important features in the image.

[0151] The above is only an implementation manner of step S12 provided by the embodiment, and the embodiment does not specifically limit the specific implementation manner of step S12.

[0152] Step S13, performing dilation correction processing on each pixel point of the filtered ampoule image to obtain a to-be-detected ampoule image.

[0153] It can be understood that the dilation correction is usually used to correct the image detail loss or deformation problem caused by noise or other factors, and the edge and contour of the filtered image are repaired, so that a more clear and accurate image can be obtained.

[0154] It should be noted that dilation is a basic operation in morphological operations, which is mainly used to expand the foreground region (usually white or high brightness region) in the image. The dilation operation is performed by sliding a structure element (also known as a kernel) over the image and increasing the pixel value according to the position of the structure element. The dilation operation can fill small holes, connect adjacent foreground regions, and expand object boundaries.

[0155] It should be understood that the to-be-detected ampoule image can be an image obtained by performing dilation correction on the filtered ampoule image.

[0156] In an implementation manner, step S13 can include steps A131-A133.

[0157] Step A131, obtaining a pixel point mean value according to the filtered ampoule image.

[0158] It can be understood that the pixel point mean value can be calculated according to the pixel value of each pixel point, and the specific calculation formula can refer to the following formula:

[0159]

[0160] Wherein, Q represents the pixel point mean value, P represents the pixel value of each pixel point, and n represents the total number of pixel points.

[0161] Step A132, dividing the pixel points in the filtered ampoule image into dark color pixel points and light color pixel points according to the pixel point mean value.

[0162] It can be understood that the pixel points with pixel values greater than the pixel point mean value are regarded as dark color pixel points, and the pixel points with pixel values less than or equal to the pixel point mean value are regarded as light color pixel points.

[0163] Step A133, performing positive dilation processing on the dark color pixel points and negative dilation processing on the light color pixel points to obtain a to-be-detected ampoule image.

[0164] It should be noted that, in order to enhance the contrast of the defect target and the background, when the defect target of the original image is dark, if the pixel value of P(x, y) is greater than the average value Q, the pixel point may exist in the defect target or the boundary; if the pixel value of P(x, y) is less than the average value Q, the pixel point P(x, y) may exist in the background; when the defect target of the original image is light, the opposite is also true.

[0165] It should be noted that positive expansion can be used to expand the foreground (usually white or high brightness area), and for each pixel point p, if there is a foreground pixel in its neighborhood, p is set to the foreground value, which is suitable for filling small holes, connecting adjacent foreground areas and expanding object boundaries.

[0166] Negative expansion is used to shrink the foreground (reduce white or high brightness area), and for each pixel point p, if all pixels in its neighborhood are foreground pixels, the foreground value of p is maintained; otherwise, it is set to the background value (usually black or low brightness value), which is suitable for removing small noise, separating adhered objects and reducing object boundaries.

[0167] It should be emphasized that the expansion operation fills small holes in dark areas, making these areas more complete and continuous, while removing noise in light areas and shrinking these areas to reduce artifacts. Through positive expansion and negative expansion processing, the edges and contours of the ampoule are more clear.

[0168] In specific implementation, the expansion processing of dark pixel points and light pixel points can refer to the following formulas:

[0169]

[0170] In this embodiment, by performing positive expansion processing on dark pixel points and negative expansion processing on light pixel points on the filtered ampoule image, we can effectively enhance specific features in the image and reduce the influence of noise.

[0171] The above is only an implementation manner of step S13 provided by the present embodiment, and the present embodiment does not specifically limit the specific implementation manner of step S13.

[0172] The present embodiment provides an ampoule defect detection method, which effectively improves the contrast and brightness distribution of the image by performing brightness equalization processing on the original ampoule image, so that the details in the image are more clear and visible. The application of bilateral filtering processing further smooths the image and reduces noise, while retaining key edge and texture features, ensuring the integrity of the image information. Through expansion correction processing on the filtered image, small holes in dark areas are filled and noise in light areas is shrunk, making the contour of the ampoule more complete and clear, improving the effect and reliability of image processing, and enabling more accurate identification of ampoule defects.

[0173] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above-mentioned embodiment one can refer to the above introduction, and the subsequent will not be described. On this basis, please refer to Figure 7 , step S20 further comprises steps S21-S23:

[0174] Step S21, the cross-step convolution is performed on the to-be-detected ampoule image to obtain a deep feature.

[0175] It should be noted that the cross-step convolution is performed on the to-be-detected ampoule image to obtain a deep feature can be executed by the depth downsampling module. When the image or feature map is processed by the depth downsampling module, the difference between non-cross-step convolution is that the stride of cross-step convolution is greater than 1, which can realize the downsampling effect while extracting the feature, and reduce the spatial dimension of the feature map.

[0176] It can be understood that the cross-step structure convolution refers to the stride used in the convolution operation is greater than 1 (usually 2), and this technology can realize the downsampling without using the pooling layer. The advantage of this is that it can maintain the end-to-end learning process, while avoiding the information loss problem that may be caused by the traditional pooling method.

[0177] In a possible implementation, step S21 can include steps A211-A213:

[0178] Step A211, the image uniform cropping is performed on the to-be-detected ampoule image to obtain a sub-feature map.

[0179] It should be noted that the image uniform cropping is performed on the to-be-detected ampoule image can divide the input ampoule image into several small blocks or sub-regions with the same size. This segmentation method ensures that the entire image is evenly covered without omission or overlap (or with specific strategy overlap). The purpose of this is to make each sub-region independently accept subsequent processing, so as to better capture the local features in the image.

[0180] It should be understood that each local feature can be regarded as a sub-feature map, which contains the information of a part of the original image.

[0181] Step A212, the feature mapping and dimension connection are performed on the sub-feature map to obtain a to-be-output sub-feature.

[0182] It can be understood that the feature mapping and dimension connection are performed on the feature map to obtain a to-be-output sub-feature can be to convert the sub-feature map into a format that is easier for the machine learning model to understand and use.

[0183] It should be noted that the feature mapping can be to apply a convolution operation or other feature extraction techniques to each sub-feature map to identify key features therein, which can include edge, texture, color distribution, etc. The result of the feature mapping is a set of new representations, which are more abstract and rich in semantic information.

[0184] It should be further noted that since each sub-feature map has been subjected to feature mapping, multi-dimensional data can be generated. At this time, it is necessary to integrate these different dimensional information to form a comprehensive feature vector or matrix, ensuring that information from different sub-regions can be effectively combined together.

[0185] In specific implementation, the feature mapping can be performed in the form of convolution layer, nonlinear function or multi-layer stacking; the dimension connection can be performed in the form of cross-layer connection, multi-scale fusion and channel concatenation, etc.

[0186] Step A213, performing strided structure convolution on the to-be-output sub-feature to obtain a deep feature.

[0187] It can be understood that the to-be-output sub-feature is further processed by using the strided structure convolution to reduce the data dimension while retaining the key features, and finally a deep feature suitable for high-level tasks (such as classification and detection) is obtained.

[0188] It should be noted that the to-be-output sub-feature is processed by using the strided convolution operation with a large step size to slide the convolution kernel, which can reduce the feature map size while reducing the calculation cost.

[0189] In this embodiment, the sub-feature map is obtained by uniformly cropping the image through the deep downsampling module; the high-level features are extracted and integrated into the to-be-output sub-feature by performing feature mapping and dimension connection on the sub-feature map; the to-be-output sub-feature is further processed by using the strided structure convolution to reduce the dimension while retaining the key feature information, finally the deep feature is generated, which effectively reduces the data dimension and calculation complexity, and also enhances the representation ability and robustness of the feature, provides high-quality and high-dimensional feature representation for subsequent defect detection and analysis, and significantly improves the accuracy of defect detection.

[0190] The above is only an implementation manner of step S21 provided by the embodiment, and the embodiment does not specifically limit the specific implementation manner of step S21.

[0191] Step S22, performing wavelet transform processing on the to-be-detected ampoule image to obtain a wavelet feature.

[0192] It can be understood that the wavelet transform processing on the to-be-detected ampoule image to obtain a wavelet feature can be performed by using the wavelet downsampling module.

[0193] It should be noted that the wavelet transform is a signal processing method, which decomposes the ampoule image to be detected into different scales and resolutions, thereby removing redundant information while retaining important features.

[0194] In a specific implementation, the ampoule image to be detected is decomposed into a series of wavelet coefficients, each corresponding to information of different frequency bands. After decomposition, a wavelet feature is obtained, which contains the key features of the original image and provides more detailed descriptions at different scales.

[0195] It should be noted that by decomposing the ampoule image to be detected through frequency domain analysis, not only can the downsampling operation be realized, but also the low-frequency information and detailed features of the original image can be effectively retained, while having high computational efficiency and good universality, which can significantly improve the stability and accuracy of feature extraction, making the network perform better when processing complex backgrounds and small targets.

[0196] In a specific implementation, the wavelet transform processing of the ampoule image to be detected by the wavelet downsampling module can be performed by selecting a suitable wavelet basis function and determining the number of decomposition layers. The image is decomposed into LL, LH, HL and HH subbands, and multi-level decomposition is performed recursively layer by layer. The appropriate subbands are selected for combination and normalization processing, and finally the wavelet feature containing multi-level features is generated.

[0197] In a feasible implementation, step S22 can include steps A221-A223:

[0198] Step A221, decompose the ampoule image to be detected to obtain frequency information.

[0199] It should be noted that the appropriate wavelet basis function and decomposition layers can be determined in advance before the image is decomposed. The number of wavelet decomposition layers, the more layers, the more detailed the details that can be captured. Generally, it can be 4 layers, 5 layers, or selected according to actual conditions, and this embodiment does not limit it.

[0200] Further, the decomposition of the ampoule image to be detected according to the wavelet basis function can be a one-level or multi-level two-dimensional discrete wavelet transform (2D-DWT) of the original image using the selected wavelet basis function. Each decomposition produces four subbands: LL (low-frequency approximation), LH (horizontal high-frequency detail), HL (vertical high-frequency detail), and HH (diagonal high-frequency detail). If more detailed analysis is required, the next level of decomposition can be performed on the LL subband.

[0201] Step A222, wavelet transform processing is performed on the ampoule image to be detected based on the frequency information, to obtain a multi-frequency feature component.

[0202] It can be understood that the multi-frequency feature component can be information of different frequency bands extracted from the ampoule image to be detected through wavelet decomposition, and can reflect characteristics of the image in different scales and directions.

[0203] It should be understood that the frequency information obtained after decomposition can be a subband at each decomposition, and selected subbands are combined into a multi-frequency feature component set, which can stack them together in time to form a multi-channel feature map, or combine the subbands according to a certain strategy.

[0204] In this embodiment, in order to better detect whether the ampoule head has defects (hook head defect, skew head defect, flat head defect, bubble head defect and foreign matter defect) and whether there are texture and other trace defects on the bottle body, the LH subband, the HL subband and the HH subband are mainly selected.

[0205] In specific implementation, the ampoule image to be detected with a resolution of H*W is decomposed by a low-pass filter H0 and a high-pass filter H1, which are respectively used to extract approximate and high-frequency information from the image, and wavelet transform is performed based on the high-frequency information, to generate four components: an approximate low-frequency component LL and high-frequency components in three directions of horizontal LH, vertical HL and diagonal HH, from which required components are selected for feature merging, so as to comprehensively capture multi-scale information of the feature map.

[0206] Step A223, convolution is performed on the multi-frequency feature component, to obtain a wavelet feature after dimension reduction.

[0207] In specific implementation, after each group of component features is extracted, the resolution of the feature map of each component can be reduced to (W / 2, W / 2), and the number of channels is increased from C to 4C. Then, a CBR (Convolution Batch ReLU) regular convolution module is used to extract discriminative features, to restore the required number of channels, and to obtain a wavelet feature.

[0208] In this embodiment, by performing wavelet decomposition on the ampoule image to be detected to extract information of different frequency bands (frequency information), and generating a multi-frequency feature component by wavelet transform processing, and by further extracting and integrating key features through convolution operation on the multi-frequency feature component, a wavelet feature after dimension reduction is generated, which can better capture subtle defects and structural characteristics, reduce data dimension and computational complexity, enhance the representation ability and robustness of the features, provide high-quality and high-dimensional feature representation, and effectively improve the accuracy of subsequent defect detection and defect classification.

[0209] The above is only an implementation manner in which step S22 provided in the embodiment can be implemented, and the embodiment does not specifically limit the specific implementation of step S22.

[0210] In step S23, the deep feature and the wavelet feature are taken as an initial feature map.

[0211] It can be understood that the deep subsampling and the wavelet subsampling are simultaneously performed when the ampoule to be detected is sampled.

[0212] It should be noted that the deep subsampling and the wavelet subsampling are simultaneously performed to obtain two sampling features. The deep feature can better retain the detail feature, and the wavelet subsampling can filter out unimportant features. The two subsampling feature channels are connected to a convolution layer, and the features of the two branches are fused to obtain a feature map with a size meeting the image output requirement.

[0213] The embodiment provides an ampoule defect detection method. Deep features and wavelet features are extracted by a deep subsampling module and a wavelet subsampling module respectively, and the deep features and the wavelet features are integrated into an initial feature map. The initial feature map can provide more rich and comprehensive feature representation while retaining key information, thereby providing a good basis for subsequent feature detection and defect classification, and thus the ampoule defect detection can be more accurately performed.

[0214] It should be noted that the above examples are only used for understanding the present application and do not constitute a limitation on the ampoule defect detection method of the present application. More forms of simple transformation based on the technical concept are within the protection scope of the present application.

[0215] The present application also provides an ampoule defect detection device, which is described with reference to Figure 8 The ampoule defect detection device comprises:

[0216] A preprocessing module 10 is configured to acquire an original ampoule image of a target ampoule, perform image correction on the original ampoule image, and obtain an ampoule image to be detected. The image correction comprises brightness equalization processing, bilateral filtering processing, and dilation correction processing.

[0217] A sampling module 20 is configured to simultaneously perform deep subsampling and wavelet subsampling on the ampoule image to be detected, obtain deep features and wavelet features, and fuse the deep features and the wavelet features to obtain an initial feature map.

[0218] A feature extraction module 30 is configured to divide the initial feature map into a plurality of feature convolution groups, and obtain a reference defect feature map according to a defect candidate region of each feature convolution group.

[0219] A defect recognition module 40 is configured to perform feature mapping on the reference defect feature map to obtain a defect image of the target ampoule.

[0220] The ampoule defect detection device provided by the present application adopts the ampoule defect detection method in the above embodiments, and can solve the technical problem that the current defect detection of the ampoule is not accurate enough. Compared with the prior art, the beneficial effects of the ampoule defect detection device provided by the present application are the same as those of the ampoule defect detection method provided by the above embodiments, and the other technical features of the ampoule defect detection device are the same as those disclosed in the above embodiment method, which will not be repeated here.

[0221] In an embodiment, the preprocessing module 10 is further configured to obtain an original ampoule image of a target ampoule, perform brightness equalization processing on pixel points in the original ampoule image to obtain an equalized ampoule image.

[0222] Perform bilateral filtering processing on the equalized ampoule image to obtain a filtered ampoule image.

[0223] Perform inflation correction processing on each pixel point of the filtered ampoule image to obtain a to-be-detected ampoule image.

[0224] In an embodiment, the preprocessing module 10 is further configured to obtain a filtering range of each pixel point according to the position of each pixel point of the equalized ampoule image.

[0225] Obtain a spatial proximity factor and a pixel value similarity factor of each pixel point according to the filtering range and the position of the pixel point.

[0226] Obtain a filtering factor according to the spatial proximity factor and the pixel value similarity factor.

[0227] Perform bilateral filtering on each pixel point of the equalized ampoule image according to the filtering factor to obtain a filtered ampoule image.

[0228] In an embodiment, the preprocessing module 10 is further configured to obtain a pixel point mean value according to the filtered ampoule image.

[0229] Divide the pixel points in the filtered ampoule image into dark color pixel points and light color pixel points according to the pixel point mean value.

[0230] Perform positive inflation processing on the dark color pixel points and negative inflation processing on the light color pixel points to obtain a to-be-detected ampoule image.

[0231] In an embodiment, the sampling module 20 is further configured to perform cross-step convolution on the to-be-detected ampoule image to obtain a depth feature.

[0232] The ampoule image to be detected is subjected to wavelet transform processing to obtain wavelet features.

[0233] The deep features and the wavelet features are used as initial feature maps.

[0234] In an embodiment, the sampling module 20 is further configured to perform image uniform clipping on the ampoule image to be detected to obtain a sub-feature map.

[0235] The sub-feature map is subjected to feature mapping and dimension connection to obtain a sub-feature to be output.

[0236] The sub-feature to be output is subjected to cross-step structure convolution to obtain deep features.

[0237] In an embodiment, the feature extraction module 30 is further configured to generate a preliminary defect candidate region based on the initial feature map.

[0238] The preliminary defect candidate region is subjected to region boundary correction to obtain a selected defect candidate region.

[0239] The selected defect candidate region is subjected to convolution based on channel fixing to obtain a reference defect feature map.

[0240] The present application provides an ampoule defect detection device, which comprises at least one processor and a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the ampoule defect detection method in the above-mentioned embodiment one.

[0241] Reference will now be made to the drawings, and specific examples thereof will be illustrated. Figure 9 The ampoule defect detection device in the embodiments of the present application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 9 The illustrated ampoule defect detection device is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0242] As Figure 9As shown, the ampoule defect detection device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. Various programs and data required for operation of the ampoule defect detection device are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the ampoule defect detection device to communicate wirelessly or by wire with other devices to exchange data. Although the ampoule defect detection device with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0243] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0244] The ampoule defect detection device provided by the present disclosure adopts the ampoule defect detection method in the above embodiments, and can solve the technical problem that the current defect detection for ampoules is not accurate enough. Compared with the prior art, the ampoule defect detection device provided by the present disclosure has the same beneficial effects as the ampoule defect detection method provided by the above embodiments, and other technical features in the ampoule defect detection device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0245] It should be understood that various aspects of the disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any appropriate manner in any one or more embodiments or examples.

[0246] The above description is merely illustrative of the application and is not intended to limit the scope of the application. Any variations and modifications that can be made by any person skilled in the art within the spirit and scope of the application are intended to be encompassed by the application. The scope of the application is defined by the appended claims.

[0247] The application provides a computer readable storage medium having computer readable program instructions (i.e. computer programs) stored thereon, the computer readable program instructions being used to perform the ampoule defect detection method in the above embodiments.

[0248] The computer readable storage medium provided by the application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, system or device, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection having one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer readable storage medium can be transmitted by any appropriate medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), etc., or any appropriate combination thereof.

[0249] The above computer readable storage medium can be contained in the ampoule defect detection device; or can exist separately and not be assembled into the ampoule defect detection device.

[0250] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by the ampoule defect detection device, the ampoule defect detection device: obtains an original ampoule image of a target ampoule, performs image correction on the original ampoule image to obtain a to-be-detected ampoule image; performs deep downsampling and wavelet downsampling on the to-be-detected ampoule image through a synchronous downsampling attention layer of an ampoule defect detection model to obtain an initial feature map; performs feature extraction on the initial feature map through a feature refining and screening layer of the ampoule defect detection model to obtain a reference defect feature map; and performs feature mapping on the reference defect feature map through a classification layer of the ampoule defect detection model to obtain a defect image of the target ampoule.

[0251] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0252] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0253] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the names of the modules do not constitute a limitation on the modules themselves.

[0254] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the ampoule defect detection method described above, and can solve the technical problem that the current defect detection for ampoules is not accurate enough. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the ampoule defect detection method provided by the above embodiments, and will not be described here.

[0255] The present application also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the ampoule defect detection method as described above.

[0256] The computer program product provided by the present application can solve the technical problem that the current defect detection for ampoules is not accurate enough. Compared with the prior art, the computer program product provided by the present application has the same beneficial effects as the ampoule defect detection method provided by the above embodiments, and will not be described here.

[0257] The above only describes some embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structural transformation, direct / indirect application in other related technical fields based on the technical concept of the present application, and the content of the present application specification and drawings are included in the patent protection scope of the present application.

Claims

1. An ampoule defect detection method characterized by comprising: The ampoule defect detection method comprises: An original ampoule image of a target ampoule is acquired, and image correction is performed on the original ampoule image to obtain a to-be-detected ampoule image, wherein the image correction comprises brightness equalization processing, bilateral filtering processing and dilation correction processing; Depth feature and wavelet feature are obtained by simultaneously performing depth downsampling and wavelet downsampling on the to-be-detected ampoule image, and initial feature map is obtained by fusing the depth feature and the wavelet feature; The initial feature map is divided into a plurality of feature convolution groups, and reference defect feature map is obtained according to defect candidate regions of each feature convolution group; Feature mapping is performed on the reference defect feature map to obtain a defect image of the target ampoule; The step of dividing the initial feature map into a plurality of feature convolution groups and obtaining reference defect feature map according to defect candidate regions of each feature convolution group comprises: The initial feature map is divided into a plurality of feature convolution groups, and preliminary defect candidate regions are generated by performing defect positioning on the feature convolution; The preliminary defect candidate regions are subjected to region boundary correction to obtain selected defect candidate regions; The selected defect candidate regions of each feature convolution group are subjected to separable convolution based on channel fixing to obtain reference defect feature map.

2. The ampoule defect detection method according to claim 1, wherein The step of acquiring an original ampoule image of a target ampoule and performing image correction on the original ampoule image to obtain a to-be-detected ampoule image comprises: An original ampoule image of a target ampoule is acquired, and brightness equalization processing is performed on pixel points in the original ampoule image to obtain an equalized ampoule image; Bilateral filtering processing is performed on the equalized ampoule image to obtain a filtered ampoule image; Each pixel point of the filtered ampoule image is subjected to dilation correction processing to obtain a to-be-detected ampoule image.

3. The ampoule defect detection method according to claim 2, wherein The step of performing bilateral filtering on the equalized ampoule image to obtain a filtered ampoule image comprises: Filtering ranges of each pixel point are obtained according to positions of each pixel point of the equalized ampoule image; Spatial proximity factors and pixel value similarity factors of each pixel point are obtained according to the filtering ranges and the pixel point positions; Filtering factors are obtained according to the spatial proximity factors and the pixel value similarity factors; Each pixel point of the equalized ampoule image is subjected to bilateral filtering according to the filtering factors to obtain a filtered ampoule image.

4. The ampoule defect detection method according to claim 2, wherein The step of performing dilation correction processing on each pixel point of the filtered ampoule image to obtain a to-be-detected ampoule image comprises: Pixel point mean values are obtained according to the filtered ampoule image; Pixel points in the filtered ampoule image are divided into dark color pixel points and light color pixel points according to the pixel point mean values; The dark color pixel points are subjected to positive dilation processing, and the light color pixel points are subjected to negative dilation processing to obtain a to-be-detected ampoule image.

5. The ampoule defect detection method according to claim 1, wherein The step of simultaneously performing depth downsampling and wavelet downsampling on the to-be-detected ampoule image to obtain depth feature and wavelet feature, and fusing the depth feature and the wavelet feature to obtain initial feature map comprises: The ampoule image to be detected is subjected to stride convolution to obtain a deep feature; The ampoule image to be detected is subjected to wavelet transform processing to obtain a wavelet feature; The deep feature and the wavelet feature are taken as an initial feature map.

6. The ampoule defect detection method according to claim 5, wherein The step of performing stride convolution on the ampoule image to be detected to obtain a deep feature comprises: The ampoule image to be detected is subjected to image uniform clipping to obtain a sub-feature map; The sub-feature map is subjected to feature mapping and dimension connection to obtain a sub-feature to be output; The sub-feature to be output is subjected to stride structure convolution to obtain a deep feature.

7. An ampoule defect detection apparatus characterized by comprising: The ampoule defect detection device executes the ampoule defect detection method of any one of claims 1 to 6, and the ampoule defect detection device comprises: A preprocessing module is configured to obtain an original ampoule image of a target ampoule, and perform image correction on the original ampoule image to obtain an ampoule image to be detected, wherein the image correction comprises brightness equalization processing, bilateral filtering processing and dilation correction processing; A sampling module is configured to simultaneously perform deep downsampling and wavelet downsampling on the ampoule image to be detected to obtain a deep feature and a wavelet feature, and fuse the deep feature and the wavelet feature to obtain an initial feature map; A feature extraction module is configured to divide the initial feature map into a plurality of feature convolution groups, and obtain a reference defect feature map according to a defect candidate region of each feature convolution group; A defect recognition module is configured to perform feature mapping on the reference defect feature map to obtain a defect image of the target ampoule.

8. An ampoule defect detection apparatus characterized by comprising: The device comprises a memory, a processor, and an ampoule defect detection program stored on the memory and executable on the processor, wherein the ampoule defect detection program is configured to implement the ampoule defect detection method of any one of claims 1 to 6.

9. A storage medium, characterized by The storage medium stores an ampoule defect detection program, and the ampoule defect detection program is executed by the processor to implement the ampoule defect detection method of any one of claims 1 to 6.

Images