Production line image-based pharmaceutical raw material foreign matter detection method and system

Abstract

The invention relates to the technical field of image detection, and discloses a pharmaceutical raw material foreign matter detection method and system based on a production line image. The method comprises the following steps: firstly, carrying out edge extraction on an initial pharmaceutical raw material production line image, and carrying out division based on an adaptive threshold to obtain a processed pharmaceutical raw material production line image; secondly, performing background and foreground segmentation on the processed pharmaceutical raw material production line image by using a KNN background difference method, and extracting and filling a foreground image region to obtain a final pharmaceutical raw material production line image; constructing a MaskR-CNN network, and optimizing an initial learning rate in the network by using a termite life cycle optimization algorithm to obtain a global optimal solution; and finally, obtaining an improved MaskR-CNN network model according to the globally optimal solution, outputting an identification result, and completing the detection of the foreign matters in the pharmaceutical raw materials. The pharmaceutical raw material foreign matter detection purpose is achieved by processing and analyzing the pharmaceutical raw material production line image, and the method is accurate and objective.

Description

Technical Field

[0001] The present invention relates to the technical field of image detection, and in particular to a method and system for detecting foreign matter in pharmaceutical raw materials based on production line images. Background Art

[0002] Chinese patent CN114723678B discloses a method and system for detecting foreign matter in high-voltage wires based on video images. The method specifically includes: obtaining a video image for detecting foreign matter in high-voltage wires, obtaining the next frame of image at intervals of m minutes, and obtaining a set of video images for detecting foreign matter in high-voltage wires; performing equal slicing processing on the video image for detecting foreign matter in high-voltage wires in the set of video images for detecting foreign matter in high-voltage wires to form a grid diagram, and obtaining an image of the area framed by a coordinate frame; after the grid diagram is enlarged, it is sequentially superimposed along the X-axis and the Y-axis to form a batch of image groups, and the image groups are divided into positive samples and negative samples according to whether the foreign matter stays on the high-voltage wires, and are input into a deep learning network model for training to obtain a recognition model of a binary classification result, and the image of the area framed by the coordinate frame is input into the recognition model of the binary classification result, and the binary classification result is output to judge the foreign matter on the high-voltage wires and the foreign matter not on the high-voltage wires. This invention does not optimize the deep learning network model, and the convergence speed is slow.

[0003] The traditional method of detecting foreign matter in pharmaceutical raw materials relies on a camera group to perform detection on a conveyor belt. However, high-tech technologies such as neural networks are not used in the detection process, and the detection results often have accuracy errors, which can cause small foreign matter to mix into the raw materials. At the same time, the traditional method of detecting foreign matter in pharmaceutical raw materials has a slow detection speed and a high missed detection rate, which poses a great threat to the health of patients. Summary of the invention

[0004] In view of the problems in the related art, the present invention provides a method and system for detecting foreign matter in pharmaceutical raw materials based on production line images to overcome the above-mentioned technical problems existing in the existing related art.

[0005] In order to solve the above technical problems, the present invention is achieved through the following technical solutions: The present invention is a method for detecting foreign matter in pharmaceutical raw materials based on production line images, comprising the following steps: S1. Acquire pharmaceutical raw material production line images to obtain an initial pharmaceutical raw material production line image set, use an improved Sobel operator edge detection algorithm to perform edge extraction on the initial pharmaceutical raw material production line images in the initial pharmaceutical raw material production line image set, and obtain a processed pharmaceutical raw material production line image set; S2. Based on the KNN background difference method, the processed pharmaceutical raw material production line images in the processed pharmaceutical raw material production line image set are segmented into background and foreground, the foreground image area is extracted, and then the hole filling is performed to obtain a final pharmaceutical raw material production line image set; S3. Build the MaskR-CNN network, establish the stochastic gradient descent optimizer iteration formula, use the termite life cycle optimization algorithm to optimize the initial learning rate in the stochastic gradient descent optimizer iteration formula, and obtain the global optimal solution of the stochastic gradient descent optimizer iteration formula; S4. An improved MaskR-CNN network model is obtained according to the global optimal solution of the iterative formula of the stochastic gradient descent optimizer, and a recognition result is output according to the final pharmaceutical raw material production line image set to complete the foreign matter detection of pharmaceutical raw materials.

[0006] The invention uses an improved Sobel operator edge detection algorithm to extract the edges of the initial pharmaceutical raw material production line image, introduces the K-means algorithm to cluster the initial pharmaceutical raw material production line image, obtains an adaptive threshold, and then divides it according to the adaptive threshold to obtain a processed pharmaceutical raw material production line image; compared with the traditional edge detection algorithm, the algorithm effectively retains the edge information and has better robustness and accuracy; secondly, the KNN background difference method is used to segment the background and foreground of the processed pharmaceutical raw material production line image, and image segmentation is achieved by establishing a probability distribution model, and then the image processed by the KNN background difference method is filled with holes to make the foreground area more complete and obtain a better segmentation effect. The KNN background difference method is used to distinguish pharmaceutical raw materials from foreign matter in pharmaceutical raw materials, laying the foundation for subsequent processing and improving the accuracy of foreign matter judgment; then the MaskR-CNN network is constructed, and the termite life cycle optimization algorithm is used to optimize the initial learning rate in the neural network to obtain the global optimal solution in the iterative process of the neural network. The algorithm simulates the initial learning rate iteration process into the behavior of termite colonies to solve the optimization problem. Levy flight effectively improves the convergence speed, and different types of termites such as worker ants expand the spatial search ability, which makes it easier to find the global optimal value with the smallest error, further improving the detection speed and accuracy; finally, the improved MaskR-CNN network model is obtained according to the global optimal solution, and the recognition results are output to complete the detection of foreign matter in pharmaceutical raw materials.

[0007] Preferably, the S1 comprises the following steps: S11, acquiring each frame of the pharmaceutical raw material production line image to obtain an initial pharmaceutical raw material production line image set, and quantizing the pixel points on the initial pharmaceutical raw material production line image in the initial pharmaceutical raw material production line image set; setting the size to An operator template is placed on the initial pharmaceutical raw material production line image, and a Cartesian grid is established to obtain the city distance of the pixel points in the operator template, thereby obtaining a distance matrix; Calculate the sum of the gradient vectors of the central pixel of the operator template in the vertical direction, the horizontal direction, the 45-degree diagonal direction, and the 135-degree diagonal direction, and calculate the average gradient component of the central pixel of the operator template according to the gradient vector and the distance matrix; calculate the average gradient components of other pixel points in the operator template in turn to obtain an average gradient component matrix, and perform a convolution operation on the average gradient component matrix to obtain an average gradient value matrix; S12, traversing the initial pharmaceutical raw material production line image, calculating the average gradient value of the pixels in the initial pharmaceutical raw material production line image, and forming an image average gradient value matrix; according to the image average gradient value matrix, using the K-means algorithm to cluster the initial pharmaceutical raw material production line image, and obtaining an adaptive threshold, the specific steps are as follows: S121, select any two pixels in the initial pharmaceutical raw material production line image, record them as the first cluster center pixel and the second cluster center pixel, and use them as the initial cluster center; calculate the average gradient difference between other pixels in the initial pharmaceutical raw material production line image and the first cluster center pixel and the second cluster center pixel according to the image average gradient value matrix, and classify the pixels to the nearest initial cluster center according to the minimum average gradient difference to form a class; S122, after all pixels in the initial pharmaceutical raw material production line image are classified, the number of pixels in the class is set to a , the pixel coordinates are marked as , divide the average gradient value of the class again to get a new average gradient value, the calculation formula is as follows: ; in, Indicates horizontal i , vertical j Pixel coordinates, Represents pixel coordinates The average gradient value at Represents a class, A Represents the new average gradient value; A new cluster center is obtained according to the new average gradient value, and the classification is re-performed according to the new cluster center until the cluster center no longer changes, and the classification is stopped to obtain the final class; the minimum average gradient value is found in the final class, and the minimum average gradient value is the adaptive threshold; S13. The initial pharmaceutical raw material production line images in the initial pharmaceutical raw material production line image set are marked according to the adaptive threshold value. When the grayscale value of a pixel in the initial pharmaceutical raw material production line image is greater than the adaptive threshold value, the grayscale value of the pixel is set to 255, otherwise the grayscale value of the pixel is set to 0, and the edge extraction of the initial pharmaceutical raw material production line image is completed. The images outside the production line are eliminated to obtain the processed pharmaceutical raw material production line images, and a processed pharmaceutical raw material production line image set is generated.

[0008] This invention uses an improved Sobel operator edge detection algorithm to extract edges from the initial pharmaceutical raw material production line image, and clusters the initial pharmaceutical raw material production line image by introducing the K-means algorithm to obtain an adaptive threshold. Compared with the traditional edge detection algorithm, it effectively retains the edge information and has better robustness and accuracy.

[0009] Preferably, S2 comprises the following steps: S21, select any processed pharmaceutical raw material production line image from the processed pharmaceutical raw material production line image set, record it as the current frame pharmaceutical raw material production line image, and then select the previous frame pharmaceutical raw material production line image of the current frame pharmaceutical raw material production line image; collect k nearest pixel points on the current frame pharmaceutical raw material production line image, calculate the norm difference between the nearest pixel point and the previous frame pharmaceutical raw material production line image, and establish a pixel point probability distribution model; set a regional indicator set ,in Indicates m A region index, where a region index of 0 represents a foreground image region, and a region index of 1 represents a background image region, and the background probability is calculated according to the pixel probability distribution model and the region index set; Set a foreground threshold and a background threshold, assign a gray value of 255 to the foreground image area where the pixel point corresponding to the background probability being less than the foreground threshold is located, assign a gray value of 0 to the background image area where the pixel point corresponding to the background probability being greater than the background threshold is located, and assign a gray value of 128 to the image area where the pixel point corresponding to the background probability being greater than or equal to the foreground threshold and less than or equal to the background threshold is located; take the pharmaceutical raw material foreign matter as the foreground image area and the pharmaceutical raw material as the background image area, complete the background and foreground segmentation, obtain the segmented pharmaceutical raw material production line image, and form a segmented pharmaceutical raw material production line image set; S22, performing an opening operation on the segmented pharmaceutical raw material production line images in the segmented pharmaceutical raw material production line image set, and then performing a closing operation to obtain a processed pharmaceutical raw material production line image set, and performing hole filling on the processed pharmaceutical raw material production line image set to obtain a final pharmaceutical raw material production line image set, the specific steps are as follows: S221, for the foreground image area in the processed pharmaceutical raw material production line image, select any pixel point in the foreground image area, record it as a seed pixel point, traverse other pixel points in the foreground image area with the seed pixel point as the center, set the grayscale value of other pixel points with the same grayscale value as the seed pixel point to 255, until traversing to the edge of the foreground image area, and obtain a preliminarily extracted pharmaceutical raw material production line image; S222. Reverse the grayscale values ​​of the pixels in the initially extracted pharmaceutical raw material production line image to obtain an inverted pharmaceutical raw material production line image, perform a bitwise AND operation on the initially extracted pharmaceutical raw material production line image and the inverted pharmaceutical raw material production line image to fill in the missing pixel grayscale values ​​in the initially extracted pharmaceutical raw material production line image, complete the hole filling, obtain the final pharmaceutical raw material production line image, and form a final pharmaceutical raw material production line image set.

[0010] The invention uses the KNN background difference method to segment the background and foreground of the processed pharmaceutical raw material production line image, distinguishes the pharmaceutical raw materials from foreign matter in the pharmaceutical raw materials, lays the foundation for subsequent processing, and improves the accuracy of foreign matter judgment; then fills the holes in the image processed by the KNN background difference method to make the foreground area more complete and obtain a better segmentation effect.

[0011] Preferably, S3 comprises the following steps: S31. Set the backbone network of the Mask R-CNN network to the ResNet 50 network, which includes several networks with sizes Two-dimensional convolutional layer, maximum pooling layer, average pooling layer, residual module and fully connected layer; introduce the attention mechanism, compress the input image in the average pooling layer, and then obtain the channel attention feature map after convolution, and output it to the fully connected layer; use transfer learning to transfer the training weights of the input image to the Mask R-CNN network to construct the Mask R-CNN network; S32, obtaining a COCO dataset on the Internet, selecting pharmaceutical raw material foreign body images such as fibers, metal shavings, and glass shavings from the COCO dataset to form a pharmaceutical raw material foreign body image sample set; extracting the foreground image area of ​​the pharmaceutical raw material foreign body image sample in the pharmaceutical raw material foreign body image sample set, and annotating them to obtain an annotated pharmaceutical raw material foreign body image sample set; and then performing brightness transformation, rotation, and noise addition on the annotated pharmaceutical raw material foreign body image sample set to achieve data enhancement, and obtain a final pharmaceutical raw material foreign body image sample set; The MaskR-CNN network is set to use a stochastic gradient descent optimizer, and the number of final pharmaceutical raw material foreign body image sample sets is b , the weight parameter is The initial learning rate is , the final pharmaceutical raw material foreign body image sample set b The labels of the samples are , the final pharmaceutical raw material foreign body image sample set b The predicted label of the sample is , then the stochastic gradient descent optimizer iteration formula is as follows: ; in, Represents the updated weight parameter; S33, inputting the final pharmaceutical raw material foreign body image sample set into the MaskR-CNN network, updating the initial learning rate with continuous iteration, optimizing the initial learning rate using the termite life cycle optimization algorithm, and obtaining the global optimal solution of the stochastic gradient descent optimizer iteration formula, the specific steps are as follows: S331, using the stochastic gradient descent optimizer iteration formula as the fitness function, assuming that there is a termite population in the search space, the initial size of the termite population is F, and the termite individuals in the termite population represent the initial learning rate in the stochastic gradient descent optimizer iteration formula; in the process of determining the initial step size of the termite individual, the Levy flight is introduced, and the Levy flight index is set to , the calculation formula of the initial step length of termites is as follows: ; in, E represents the initial step length of the termite individual and obeys the Levy flight distribution. and represents the normal distribution coefficient; When the Levy flight index is between 1 and 2, the initial step length of the termite individual is controlled by the Levy flight index, otherwise the initial step length of the termite individual is improved and the current number of iterations is set to n , the maximum number of iterations is N , in n The individual termite step length at the iteration is recorded as , then the individual step length of termites satisfies the following formula: ; S332: The termite population moves with the termite individual step length, and a moving vector set is set. ,in , , and Represents the moving vector. The termite population is controlled by the moving vector set when moving. n The position of the termite individual at the iteration is , No. n The optimal position of termite individuals in the iteration is , thenn+ Individual termite positions at 1 iteration The calculation formula is as follows: ; Further, n At the first iteration, the position of the termite individuals corresponds to the Mask R-CNN network at the n Iterative learning rate, update the initial learning rate to n Iteration learning rate; S333, as the position of individual termites is continuously updated, worker ant soldiers are introduced to expand the search space, setting the n At the iteration, the position of the worker soldier is , then n+ At the 1st iteration h The individual positions of worker soldiers The calculation formula is as follows: ; In the process of updating the position of individual worker soldiers, calculate the n+ At the 1st iteration h The best fitness function value corresponding to the individual position of the worker ant soldier is compared with the n+ The best fitness function value corresponding to the individual position of the termite at the first iteration is selected. n+ The best fitness function value of 1 iteration is assigned to the n Iteration learning rate; when the current number of iterations reaches the maximum number of iterations, stop the iteration and get the final h Only the individual positions of worker ant soldiers are determined, and the global optimal solution of the iterative formula of the stochastic gradient descent optimizer is obtained.

[0012] The invention constructs a MaskR-CNN network, uses the termite life cycle optimization algorithm to optimize the initial learning rate in the neural network, and obtains an improved MaskR-CNN network model. The initial learning rate iteration process is simulated into termite colony behavior to solve the optimization problem. Among them, Levy flight effectively improves the convergence speed, and different types of termites such as worker ants expand the spatial search capability, making it easier to find the global optimal value with the smallest error, further improving the detection speed and accuracy.

[0013] Preferably, S4 comprises the following steps: S41, obtaining an improved MaskR-CNN network model according to the global optimal solution of the iterative formula of the stochastic gradient descent optimizer, annotating, brightness transforming, rotating and adding noise to the final pharmaceutical raw material production line image set, inputting the image into the improved MaskR-CNN network model, and outputting the recognition result; S42. Obtain the type of foreign matter in the pharmaceutical raw material according to the identification result, and complete the foreign matter detection in the pharmaceutical raw material.

[0014] This embodiment also discloses a system for a pharmaceutical raw material foreign body detection method based on production line images, which specifically includes: an image edge extraction module, an image background and foreground segmentation and filling module, a neural network parameter optimization module, and a pharmaceutical raw material foreign body detection module; The image edge extraction module is used to extract the edge of the initial pharmaceutical raw material production line image using an improved sobel operator edge detection algorithm; The image background and foreground segmentation and filling module is used to perform background and foreground segmentation on the processed pharmaceutical raw material production line image based on the KNN background difference method, and then perform hole filling; The neural network parameter optimization module is used to optimize the initial learning rate using a termite life cycle optimization algorithm; The pharmaceutical raw material foreign body detection module is used to output recognition results using an improved MaskR-CNN network model.

[0015] The present invention has the following beneficial effects: 1. This invention uses an improved Sobel operator edge detection algorithm to extract edges from the initial pharmaceutical raw material production line image, and clusters the initial pharmaceutical raw material production line image by introducing a K-means algorithm to obtain an adaptive threshold. Compared with traditional edge detection algorithms, it effectively retains edge information and has better robustness and accuracy.

[0016] 2. The invention uses the KNN background difference method to segment the background and foreground of the processed pharmaceutical raw material production line image, distinguishes the pharmaceutical raw materials from foreign matter in the pharmaceutical raw materials, lays the foundation for subsequent processing, and improves the accuracy of foreign matter judgment; then fills the holes in the image processed by the KNN background difference method to make the foreground area more complete and obtain a better segmentation effect.

[0017] 3. The invention constructs a MaskR-CNN network, uses a termite life cycle optimization algorithm to optimize the initial learning rate in the neural network, and obtains an improved MaskR-CNN network model. The initial learning rate iteration process is simulated into termite colony behavior to solve the optimization problem. Among them, Levy flight effectively improves the convergence speed, and different types of termites such as worker ants expand the spatial search capability, making it easier to find the global optimal value with the smallest error, further improving the detection speed and accuracy.

[0018] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the invention, the drawings required for describing the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For ordinary technicians in this field, they can also obtain drawings based on these drawings without paying any creative work.

[0020] Figure 1 A schematic diagram of the process of foreign matter detection in pharmaceutical raw materials by a pharmaceutical raw material foreign matter detection system based on production line images provided by the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] In the description of the present invention, it is necessary to understand that the terms "opening", "upper", "lower", "top", "middle", "inside" and the like indicating orientation or positional relationship are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the components or elements referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the invention.

[0023] Example 1 See also Figure 1 This embodiment discloses a method for detecting foreign matter in pharmaceutical raw materials based on production line images, which specifically includes the following contents: S1. Acquire pharmaceutical raw material production line images to obtain an initial pharmaceutical raw material production line image set, use an improved Sobel operator edge detection algorithm to perform edge extraction on the initial pharmaceutical raw material production line images in the initial pharmaceutical raw material production line image set, and obtain a processed pharmaceutical raw material production line image set; The S1 comprises the following steps: S11, acquiring each frame of the pharmaceutical raw material production line image to obtain an initial pharmaceutical raw material production line image set, and quantizing the pixel points on the initial pharmaceutical raw material production line image in the initial pharmaceutical raw material production line image set; setting the size to An operator template is placed on the initial pharmaceutical raw material production line image, and a Cartesian grid is established to obtain the city distance of the pixel points in the operator template, thereby obtaining a distance matrix; Calculate the sum of the gradient vectors of the central pixel of the operator template in the vertical direction, the horizontal direction, the 45-degree diagonal direction, and the 135-degree diagonal direction, and calculate the average gradient component of the central pixel of the operator template according to the gradient vector sum and the distance matrix; calculate the average gradient components of other pixel points in the operator template in turn to obtain an average gradient component matrix, and perform a convolution operation on the average gradient component matrix to obtain an average gradient value matrix; S12, traversing the initial pharmaceutical raw material production line image, calculating the average gradient value of the pixels in the initial pharmaceutical raw material production line image, and forming an image average gradient value matrix; according to the image average gradient value matrix, using the K-means algorithm to cluster the initial pharmaceutical raw material production line image, and obtaining an adaptive threshold, the specific steps are as follows: S121, select any two pixels in the initial pharmaceutical raw material production line image, record them as the first cluster center pixel and the second cluster center pixel, and use them as the initial cluster center; calculate the average gradient difference between other pixels in the initial pharmaceutical raw material production line image and the first cluster center pixel and the second cluster center pixel according to the image average gradient value matrix, and classify the pixels to the nearest initial cluster center according to the minimum average gradient difference to form a class; S122, after all pixels in the initial pharmaceutical raw material production line image are classified, the number of pixels in the class is set to a , the pixel coordinates are marked as , divide the average gradient value of the class again to get a new average gradient value, the calculation formula is as follows: ; in, Indicates horizontal i , vertical j Pixel coordinates, Represents pixel coordinates The average gradient value at Represents a class, A Represents the new average gradient value; A new cluster center is obtained according to the new average gradient value, and the classification is re-performed according to the new cluster center until the cluster center no longer changes, and the classification is stopped to obtain the final class; the minimum average gradient value is found in the final class, and the minimum average gradient value is the adaptive threshold; S13, the initial pharmaceutical raw material production line images in the initial pharmaceutical raw material production line image set are marked according to the adaptive threshold value, when the gray value of the pixel in the initial pharmaceutical raw material production line image is greater than the adaptive threshold value, the gray value of the pixel is set to 255, otherwise the gray value of the pixel is set to 0, the edge extraction of the initial pharmaceutical raw material production line image is completed, the images outside the production line are eliminated, the processed pharmaceutical raw material production line images are obtained, and the processed pharmaceutical raw material production line image set is generated; S2. Based on the KNN background difference method, the processed pharmaceutical raw material production line images in the processed pharmaceutical raw material production line image set are segmented into background and foreground, the foreground image area is extracted, and then the hole filling is performed to obtain a final pharmaceutical raw material production line image set; The S2 comprises the following steps: S21, select any processed pharmaceutical raw material production line image from the processed pharmaceutical raw material production line image set, record it as the current frame pharmaceutical raw material production line image, and then select the previous frame pharmaceutical raw material production line image of the current frame pharmaceutical raw material production line image; collect k nearest pixel points on the current frame pharmaceutical raw material production line image, calculate the norm difference between the nearest pixel point and the previous frame pharmaceutical raw material production line image, and establish a pixel point probability distribution model; set a regional indicator set ,in Indicates m A region index, where a region index of 0 represents a foreground image region, and a region index of 1 represents a background image region, and the background probability is calculated according to the pixel probability distribution model and the region index set; Set a foreground threshold and a background threshold, assign a gray value of 255 to the foreground image area where the pixel point corresponding to the background probability being less than the foreground threshold is located, assign a gray value of 0 to the background image area where the pixel point corresponding to the background probability being greater than the background threshold is located, and assign a gray value of 128 to the image area where the pixel point corresponding to the background probability being greater than or equal to the foreground threshold and less than or equal to the background threshold is located; take the pharmaceutical raw material foreign matter as the foreground image area and the pharmaceutical raw material as the background image area, complete the background and foreground segmentation, obtain the segmented pharmaceutical raw material production line image, and form a segmented pharmaceutical raw material production line image set; S22, performing an opening operation on the segmented pharmaceutical raw material production line images in the segmented pharmaceutical raw material production line image set, and then performing a closing operation to obtain a processed pharmaceutical raw material production line image set, and performing hole filling on the processed pharmaceutical raw material production line image set to obtain a final pharmaceutical raw material production line image set, the specific steps are as follows: S221, for the foreground image area in the processed pharmaceutical raw material production line image, select any pixel point in the foreground image area, record it as a seed pixel point, traverse other pixel points in the foreground image area with the seed pixel point as the center, set the grayscale value of other pixel points with the same grayscale value as the seed pixel point to 255, until traversing to the edge of the foreground image area, and obtain a preliminarily extracted pharmaceutical raw material production line image; S222, inverting the grayscale values ​​of pixels in the initially extracted pharmaceutical raw material production line image to obtain an inverted pharmaceutical raw material production line image, performing a bitwise AND operation on the initially extracted pharmaceutical raw material production line image and the inverted pharmaceutical raw material production line image to fill in the missing pixel grayscale values ​​in the initially extracted pharmaceutical raw material production line image, completing the hole filling, obtaining a final pharmaceutical raw material production line image, and forming a final pharmaceutical raw material production line image set; S3. Build the MaskR-CNN network, establish the stochastic gradient descent optimizer iteration formula, use the termite life cycle optimization algorithm to optimize the initial learning rate in the stochastic gradient descent optimizer iteration formula, and obtain the global optimal solution of the stochastic gradient descent optimizer iteration formula; The S3 comprises the following steps: S31. Set the backbone network of the Mask R-CNN network to the ResNet 50 network, which includes several networks with sizes Two-dimensional convolutional layer, maximum pooling layer, average pooling layer, residual module and fully connected layer; introduce the attention mechanism, compress the input image in the average pooling layer, and then obtain the channel attention feature map after convolution, and output it to the fully connected layer; use transfer learning to transfer the training weights of the input image to the Mask R-CNN network to construct the Mask R-CNN network; S32, obtaining a COCO dataset on the Internet, selecting pharmaceutical raw material foreign body images such as fibers, metal shavings, and glass shavings from the COCO dataset to form a pharmaceutical raw material foreign body image sample set; extracting the foreground image area of ​​the pharmaceutical raw material foreign body image sample in the pharmaceutical raw material foreign body image sample set, and annotating them to obtain an annotated pharmaceutical raw material foreign body image sample set; and then performing brightness transformation, rotation, and noise addition on the annotated pharmaceutical raw material foreign body image sample set to achieve data enhancement, and obtain a final pharmaceutical raw material foreign body image sample set; The MaskR-CNN network is set to use a stochastic gradient descent optimizer, and the number of final pharmaceutical raw material foreign body image sample sets is b , the weight parameter is The initial learning rate is , the final pharmaceutical raw material foreign body image sample set b The labels of the samples are , the final pharmaceutical raw material foreign body image sample set b The predicted label of the sample is , then the stochastic gradient descent optimizer iteration formula is as follows: ; in, Represents the updated weight parameter; S33, inputting the final pharmaceutical raw material foreign body image sample set into the MaskR-CNN network, updating the initial learning rate with continuous iteration, optimizing the initial learning rate using the termite life cycle optimization algorithm, and obtaining the global optimal solution of the stochastic gradient descent optimizer iteration formula, the specific steps are as follows: S331, using the stochastic gradient descent optimizer iteration formula as the fitness function, assuming that there is a termite population in the search space, the initial size of the termite population is F, and the termite individuals in the termite population represent the initial learning rate in the stochastic gradient descent optimizer iteration formula; in the process of determining the initial step size of the termite individual, the Levy flight is introduced, and the Levy flight index is set to , the calculation formula of the initial step length of termites is as follows: ; in, E represents the initial step length of the termite individual and obeys the Levy flight distribution. and represents the normal distribution coefficient; When the Levy flight index is between 1 and 2, the initial step length of the termite individual is controlled by the Levy flight index, otherwise the initial step length of the termite individual is improved and the current number of iterations is set to n , the maximum number of iterations is N , in n The individual step length of termites in the iteration is recorded as , then the individual step length of termites satisfies the following formula: ; S332: The termite population moves with the termite individual step length, and a moving vector set is set. ,in , , and Represents the moving vector. The termite population is controlled by the moving vector set when moving. n The position of the termite individual at the iteration is , No. n The optimal position of termite individuals in the iteration is , then n+ Individual termite positions at 1 iteration The calculation formula is as follows: ; Further, n At the first iteration, the position of the termite individuals corresponds to the Mask R-CNN network at the n Iterative learning rate, update the initial learning rate to n Iteration learning rate; S333, as the position of individual termites is continuously updated, worker ant soldiers are introduced to expand the search space, setting the n At the iteration, the position of the worker soldier is , then n+ At the 1st iteration h The individual positions of worker soldiers The calculation formula is as follows: ; In the process of updating the position of individual worker soldiers, calculate the n+ At the 1st iteration h The best fitness function value corresponding to the individual position of the worker ant soldier is compared with the n+ The best fitness function value corresponding to the individual position of the termite at the first iteration is selected. n+ The best fitness function value of 1 iteration is assigned to the n Iteration learning rate; when the current number of iterations reaches the maximum number of iterations, stop the iteration and get the final h Only the individual positions of worker ants and soldiers are used to obtain the global optimal solution of the iterative formula of the stochastic gradient descent optimizer; S4, obtaining an improved MaskR-CNN network model according to the global optimal solution of the stochastic gradient descent optimizer iterative formula, outputting the recognition result according to the final pharmaceutical raw material production line image set, and completing the pharmaceutical raw material foreign body detection; The S4 comprises the following steps: S41, obtaining an improved MaskR-CNN network model according to the global optimal solution of the iterative formula of the stochastic gradient descent optimizer, annotating, brightness transforming, rotating and adding noise to the final pharmaceutical raw material production line image set, inputting the image into the improved MaskR-CNN network model, and outputting the recognition result; S42. Obtain the type of foreign matter in the pharmaceutical raw material according to the identification result, and complete the foreign matter detection in the pharmaceutical raw material.

[0024] Example 2 This embodiment also discloses a system for a pharmaceutical raw material foreign body detection method based on production line images, which specifically includes: an image edge extraction module, an image background and foreground segmentation and filling module, a neural network parameter optimization module, and a pharmaceutical raw material foreign body detection module; The image edge extraction module is used to extract the edge of the initial pharmaceutical raw material production line image using an improved sobel operator edge detection algorithm; The image background and foreground segmentation and filling module is used to perform background and foreground segmentation on the processed pharmaceutical raw material production line image based on the KNN background difference method, and then perform hole filling; The neural network parameter optimization module is used to optimize the initial learning rate using a termite life cycle optimization algorithm; The pharmaceutical raw material foreign body detection module is used to output recognition results using an improved MaskR-CNN network model.

[0025] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0026] The preferred embodiments of the invention disclosed above are only used to help explain the invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the invention, so that those skilled in the art can understand and use the invention well.

Claims

1. A method for detecting foreign matter in pharmaceutical raw materials based on production line images, characterized in that: The steps include: S1. Acquire pharmaceutical raw material production line images to obtain an initial pharmaceutical raw material production line image set, use an improved Sobel operator edge detection algorithm to perform edge extraction on the initial pharmaceutical raw material production line images in the initial pharmaceutical raw material production line image set, and obtain a processed pharmaceutical raw material production line image set; S2. Based on the KNN background difference method, the processed pharmaceutical raw material production line images in the processed pharmaceutical raw material production line image set are segmented into background and foreground, the foreground image area is extracted, and then the hole filling is performed to obtain a final pharmaceutical raw material production line image set; S3. Build the MaskR-CNN network, establish the stochastic gradient descent optimizer iteration formula, optimize the initial learning rate in the stochastic gradient descent optimizer iteration formula, and obtain the global optimal solution of the stochastic gradient descent optimizer iteration formula; S4. An improved MaskR-CNN network model is obtained according to the global optimal solution of the iterative formula of the stochastic gradient descent optimizer, and a recognition result is output according to the final pharmaceutical raw material production line image set to complete the foreign matter detection of pharmaceutical raw materials.

2. The method for detecting foreign matter in pharmaceutical raw materials based on production line images according to claim 1, characterized in that: The S1 comprises the following steps: S11, acquiring a pharmaceutical raw material production line image to obtain an initial pharmaceutical raw material production line image set, setting an operator template on the initial pharmaceutical raw material production line image, calculating the average gradient component of the pixel points in the operator template, and obtaining an average gradient value matrix; S12, traversing the initial pharmaceutical raw material production line image to obtain an image average gradient value matrix, and clustering the initial pharmaceutical raw material production line image using a K-means algorithm according to the image average gradient value matrix to obtain an adaptive threshold; S13, using the adaptive threshold to perform edge extraction on the initial pharmaceutical raw material production line image to obtain a processed pharmaceutical raw material production line image, and generate a processed pharmaceutical raw material production line image set.

3. The method for detecting foreign matter in pharmaceutical raw materials based on production line images according to claim 2, characterized in that: The clustering of the initial pharmaceutical raw material production line images using the K-means algorithm comprises the following steps: An initial cluster center is selected from the initial pharmaceutical raw material production line image, and the average gradient difference is calculated according to the image average gradient value matrix, and the pixel points are classified to the nearest initial cluster center according to the minimum average gradient difference to form a class; the average gradient value of the class is divided again to obtain a new average gradient value, and a new cluster center is obtained according to the new average gradient value, and reclassification is performed according to the new cluster center until the cluster center no longer changes, and classification is stopped to obtain a final class; the minimum average gradient value is found in the final class, and the minimum average gradient value is the adaptive threshold.

4. The method for detecting foreign matter in pharmaceutical raw materials based on production line images according to claim 3, characterized in that: The S2 comprises the following steps: S21, selecting a current frame pharmaceutical raw material production line image and a previous frame pharmaceutical raw material production line image from the processed pharmaceutical raw material production line image set, collecting k nearest pixel points on the current frame pharmaceutical raw material production line image, calculating the norm difference between the nearest pixel point and the previous frame pharmaceutical raw material production line image, establishing a pixel point probability distribution model, and calculating the background probability; Setting a foreground threshold and a background threshold, and performing background and foreground segmentation by comparing with the background threshold, thereby obtaining a segmented pharmaceutical raw material production line image and forming a segmented pharmaceutical raw material production line image set; S22, performing opening and closing operations on the segmented pharmaceutical raw material production line image set to obtain a processed pharmaceutical raw material production line image set, and performing hole filling on the processed pharmaceutical raw material production line image set to obtain a final pharmaceutical raw material production line image set.

5. The method for detecting foreign matter in pharmaceutical raw materials based on production line images according to claim 4, characterized in that: The hole filling of the processed pharmaceutical raw material production line image set comprises the following steps: S221, selecting a seed pixel in the foreground image area of ​​the processed pharmaceutical raw material production line image, setting the grayscale values ​​of other pixels having the same grayscale value as the seed pixel to 255, and obtaining a preliminarily extracted pharmaceutical raw material production line image; S222. After inverting the grayscale values ​​of the pixels in the initially extracted pharmaceutical raw material production line image, perform a bitwise AND operation with the initially extracted pharmaceutical raw material production line image to complete the hole filling, obtain the final pharmaceutical raw material production line image, and form a final pharmaceutical raw material production line image set.

6. The method for detecting foreign matter in pharmaceutical raw materials based on production line images according to claim 5, characterized in that: The S3 comprises the following steps: S31. Set the backbone network of the Mask R-CNN network to the ResNet 50 network, which includes several networks with sizes Two-dimensional convolutional layer, maximum pooling layer, average pooling layer, residual module and fully connected layer; introduce attention mechanism and use transfer learning to transfer the training weights of the input image to the Mask R-CNN network to construct the Mask R-CNN network; S32, obtaining a COCO dataset on the Internet, selecting fiber, metal shavings, and glass shavings pharmaceutical raw material foreign body images from the COCO dataset to form a pharmaceutical raw material foreign body image sample set; extracting the foreground image area of ​​the pharmaceutical raw material foreign body image samples in the pharmaceutical raw material foreign body image sample set, and performing annotation, brightness transformation, rotation, and noise addition to obtain a final pharmaceutical raw material foreign body image sample set; The MaskR-CNN network is set to use a stochastic gradient descent optimizer, and the number of final pharmaceutical raw material foreign body image sample sets is b , the weight parameter is The initial learning rate is , the final pharmaceutical raw material foreign body image sample set b The labels of the samples are , the final pharmaceutical raw material foreign body image sample set b The predicted label of the sample is , then the stochastic gradient descent optimizer iteration formula is as follows: ； in, Represents the updated weight parameter; S33. Input the final pharmaceutical raw material foreign body image sample set into the Mask R-CNN network. With continuous iterations, the initial learning rate is updated, and the initial learning rate is optimized using the termite life cycle optimization algorithm to obtain the global optimal solution of the stochastic gradient descent optimizer iterative formula.

7. A method for detecting foreign matter in pharmaceutical raw materials based on production line images according to claim 6, characterized in that: The S33 comprises the following steps: S331, using the stochastic gradient descent optimizer iteration formula as the fitness function, assuming that there is a termite population in the search space, the initial size of the termite population is F, and the termite individuals in the termite population represent the initial learning rate in the stochastic gradient descent optimizer iteration formula; in the process of determining the initial step size of the termite individual, the Levy flight is introduced, and the Levy flight index is set to , get the initial step length of the termite individual; When the Levy flight index is between 1 and 2, the initial step length of the termite individual is controlled by the Levy flight index, otherwise the initial step length of the termite individual is improved and the current number of iterations is set to n , the maximum number of iterations is N , in n The individual termite step length at the iteration is recorded as , then the individual step length of termites satisfies the following formula: ； S332: The termite population moves with the termite individual step length, and a moving vector set is set. ,in , , and Represents the moving vector. The termite population is controlled by the moving vector set when moving. n The position of the termite individual at the iteration is , No. n The optimal position of termite individuals in the iteration is , then n+ Individual termite positions at 1 iteration The calculation formula is as follows: ； Further, n At the nth iteration, the position of the termite individuals corresponds to the learning rate of the Mask R-CNN network at the nth iteration, and the initial learning rate is updated to the nth iteration. n Iteration learning rate; S333, as the position of individual termites is continuously updated, worker ant soldiers are introduced to expand the search space, setting the n The individual positions of worker ant soldiers at the iteration are calculated. n+ At the 1st iteration h Only the individual positions of worker soldiers; In the process of updating the position of individual worker soldiers, calculate the n+ At the 1st iteration h The best fitness function value corresponding to the individual position of the worker ant soldier is compared with the n+ The best fitness function value corresponding to the individual position of the termite at the first iteration is selected. n+ The best fitness function value of 1 iteration is assigned to the n Iteration learning rate; when the current number of iterations reaches the maximum number of iterations, stop the iteration and get the final h Only the individual positions of worker ant soldiers are determined, and the global optimal solution of the iterative formula of the stochastic gradient descent optimizer is obtained.

8. The method for detecting foreign matter in pharmaceutical raw materials based on production line images according to claim 7, characterized in that: The S4 comprises the following steps: S41, obtaining an improved MaskR-CNN network model according to the global optimal solution of the iterative formula of the stochastic gradient descent optimizer, annotating, brightness transforming, rotating and adding noise to the final pharmaceutical raw material production line image set, inputting the image into the improved MaskR-CNN network model, and outputting the recognition result; S42. Obtain the type of foreign matter in the pharmaceutical raw material according to the identification result, and complete the foreign matter detection in the pharmaceutical raw material.

9. A system for implementing the method for detecting foreign matter in pharmaceutical raw materials based on production line images as claimed in any one of claims 1 to 8, characterized in that: Specifically include: Image edge extraction module, image background and foreground segmentation and filling module, neural network parameter optimization module and pharmaceutical raw material foreign body detection module; The image edge extraction module is used to extract the edge of the initial pharmaceutical raw material production line image using an improved sobel operator edge detection algorithm; The image background and foreground segmentation and filling module is used to perform background and foreground segmentation on the processed pharmaceutical raw material production line image based on the KNN background difference method, and then perform hole filling; The neural network parameter optimization module is used to optimize the initial learning rate using a termite life cycle optimization algorithm; The pharmaceutical raw material foreign body detection module is used to output recognition results using an improved MaskR-CNN network model.

Images