A medicine quality detection method and system based on boundary distributed attention

Abstract

The present invention discloses a pharmaceutical quality inspection method and system based on boundary distribution attention, which includes: constructing a data set, that is, calibrating the position and category of visible foreign matter on the target liquid image; building a neural network model including a feature extraction network, a feature fusion network, and a boundary distribution attention detection head network; wherein the output of the boundary distribution attention detection head network is the category to which each pixel point belongs and the distance from the four boundaries of the regression box with each pixel point as the center point to the center point, and the category is the visible foreign matter category or background; using the data set to train the neural network model; inputting the target liquid image to be inspected into the trained neural network model to obtain the detection result of the visible foreign matter. The technical solution of the present invention creatively designs a neural network including a boundary distribution attention detection head network, which can analyze the importance of each boundary distribution of the regression box, thereby guiding the classification of pixel points and reducing the false detection rate.

Description

Technical Field

[0001] The present invention belongs to the technical field of visible foreign matter detection based on image technology, and specifically relates to a medicine quality detection method and system based on boundary distribution attention. Background Art

[0002] The pharmaceutical industry is a major livelihood industry in my country, and the quality of medicine is directly related to national security. Currently, liquid medicines used in my country, such as vaccines, oral liquids, eye drops, and intravenous drips, are typically bottled and stored in transparent plastic or glass bottles. However, during the production and packaging process, visible foreign matter may be introduced into the liquid medicine due to chemical reactions during raw material processing, impurities produced, incomplete cleaning of the medicine bottles, aging and damage of the filter material, and glass shards falling off during capping, thus affecting the quality of the medicine. Most of these liquid medicines are used directly on the human body, and their quality is directly related to the health of the patient. If visible foreign matter is present in the liquid medicine, the patient may experience adverse reactions, which may affect the effectiveness of the medicine at the mildest, or even lead to serious diseases such as thrombosis, pulmonary fibrosis, and even organ failure at the worst. Therefore, the detection of visible foreign matter in the liquid medicine is an essential production process.

[0003] In order to provide unified standards and guidance for drug quality, the following regulations have been formulated: Liquid drugs must not contain any visible foreign matter, including metal shavings, glass shavings, fibers longer than 2mm, lumps with a maximum particle size exceeding 2mm, smoke-like particle deposits visible to the naked eye when rotated after standing for a period of time, and protein-like substances that are difficult to count within the specified time. Therefore, the detection of visible foreign matter in drug liquids presents the following technical difficulties:

[0004] 1. There are many types of visible foreign matter in the medicine liquid. Different types of visible foreign matter have different sizes and properties, and their posture changes with the movement of the medicine liquid will also be different, making classification difficult.

[0005] 2. The background is complex and noisy. The bottle body will reflect the light during light inspection, and bubbles will be generated when the liquid medicine rotates. These may lead to misjudgment, and the bubbles may block foreign objects, increasing the difficulty of positioning.

[0006] 3. Visible foreign matter is tiny. Among them, visible insoluble matter larger than 50 microns is considered visible foreign matter, which means that the resolution of the algorithm needs to reach 50 microns.

[0007] 4. High detection speed is required. The detection speed of high-end pharmaceutical production lines is usually very fast, and the detection algorithm needs to meet real-time requirements.

[0008] Currently, visible foreign matter detection in pharmaceutical liquids involves both traditional manual inspection and machine vision. Traditional manual inspection typically uses a visual inspection method, whereby the bottle is manually flipped or shaken to cause visible foreign matter at the bottom of the bottle to move with the liquid. If visible foreign matter is present, the liquid is classified as defective. However, manual inspection suffers from low efficiency, susceptibility to subjective judgment, poor stability, short continuous operation time, and high false detection rates. Therefore, research is needed on machine learning visual inspection algorithms for visible foreign matter detection in pharmaceutical liquids. Traditional visible foreign matter detection algorithms process images of pharmaceutical liquids using manually designed feature extractors, resulting in poor generalization performance and the difficulty of manually designing feature extractors. To address the shortcomings of traditional visible foreign matter detection algorithms, research on deep learning-based visible foreign matter detection algorithms for pharmaceutical liquids is essential. Currently, there is limited research in my country on deep learning for visible foreign matter detection in pharmaceuticals, resulting in a significant gap between China and other countries in the field of fully automated visual inspection machines. Therefore, developing a deep learning-based visible foreign matter detection algorithm for pharmaceutical liquids with independent intellectual property rights is of great significance. Summary of the Invention

[0009] The purpose of the present invention is to solve the technical problem of the accuracy of detecting visible foreign matter in liquid medicine, and further provide a method and system for pharmaceutical quality inspection based on boundary distribution attention, which realizes the detection of visible foreign matter in liquid medicine based on deep learning with the help of deep learning, and the method is also applicable to the detection of visible foreign matter in other liquids on the production line. Among them, the pharmaceutical quality inspection method realizes the detection of visible foreign matter based on deep learning through a neural network model including a feature extraction network, a feature fusion network and a boundary distribution attention detection head network. In particular, the creative design of the neural network including the boundary distribution attention detection head network can analyze the importance of each boundary distribution of the regression frame, thereby guiding the classification of pixel points, accurately identifying the visible foreign matter area, improving the recognition accuracy of visible foreign matter detection, and reducing the false detection rate.

[0010] In one aspect, the present invention provides a method for detecting pharmaceutical quality based on boundary distributed attention, comprising the following steps:

[0011] S1: Constructing a dataset: Acquire target liquid images and calibrate the locations and categories of visible foreign matter on the target liquid images;

[0012] S2: Construct a neural network model that includes a feature extraction network, a feature fusion network, and a boundary distribution attention detection head network connected in sequence;

[0013] The input of the feature extraction network is the target liquid image, the output of the feature extraction network serves as the input of the feature fusion network, the output of the feature fusion network serves as the input of the boundary distribution attention detection head network, and the output of the boundary distribution attention detection head network is the category to which each pixel belongs and the distance from the four boundaries of the regression box with each pixel as the center point to the center point, and the category is the visible foreign matter category or background;

[0014] S3: training the neural network model using the data set;

[0015] S4: Inputting the target liquid image to be detected into the trained neural network model to obtain the detection result of visible foreign matter.

[0016] Further optionally, the boundary distribution attention detection head network is provided with a boundary prediction branch for predicting the distance from the boundary to the center point of the regression box and a classification prediction branch for predicting the category to which the pixel point belongs;

[0017] For each boundary of the regression box with the pixel as the center point, the boundary prediction branch predicts N1 discretely distributed prediction values, and then combines the N1 prediction values ​​to obtain the distance from the boundary to the center point, where N1 is a positive integer;

[0018] For each pixel point, the classification prediction branch determines the category to which each pixel point belongs under the guidance of the prediction value obtained by the boundary prediction branch;

[0019] Specifically, the predicted values ​​of each boundary on the regression box of the same pixel point are used to obtain the total guidance score, and then the total guidance score is multiplied by the classification prediction branch to predict the initial value of each category probability of the same pixel point to obtain the final value of each category probability, among which the category corresponding to the largest category probability final value is the category to which the pixel point belongs.

[0020] Further optionally, the process of obtaining the total guidance score is as follows:

[0021] Score=sigmoid(K1*S t +K2*S b +K3*S l +K4*S r +K5*S m )

[0022] In the formula, Score is the total guidance score, sigmoid is a nonlinear action function of a neuron, S t 、S b 、S l 、S r are the guidance scores corresponding to the four boundaries of the regression box, S mis the guidance score corresponding to the mean, K1, K2, K3, K4, K5 represent S t 、S b 、S l 、S r and S m The percentage of the total guidance score;

[0023] Among them, the boundary prediction branch predicts N1 prediction values ​​corresponding to each boundary on the regression box where the pixel point is located, and then selects q prediction values ​​with the highest probability from the N1 prediction values ​​of each boundary, and then calculates the mean Xm corresponding to each boundary, and then concatenates the q prediction values ​​with the highest probability corresponding to each boundary with the mean Xm, recorded as Xc, and uses convolution to adjust the number of channels and a sigmoid function for Xc to obtain the proportion values ​​K1, K2, K3, K4, and K5.

[0024] Then divide Xc into 4 parts according to the boundary, record them as Xt, Xb, Xl, Xr, and then use the convolution with shared weights to adjust the number of channels of Xt, Xb, Xl, Xr, and Xm, and then pass a sigmoid function to obtain the guidance scores S corresponding to the four boundaries. t 、S b 、S l 、S r 、S m .

[0025] Further optionally, the feature extraction network is provided with a conv1 block, a conv2 block, a conv3 block, a conv4 block, and a conv5 block;

[0026] The target liquid image is first input into the conv1 block, then the maxpooling operation is performed and then input into the conv2 block. The conv2 block repeats the z1 round operation and outputs the feature map. Figure 1 ; Then the features Figure 1 Input the conv3 block, the conv3 block repeats the z2 rounds of operation and outputs feature map 2; input the feature map 2 to the conv4 block, the conv4 block repeats the z3 rounds of operation and outputs feature map 3; input the feature map 3 to the conv5 block, the conv5 block repeats the z4 rounds of operation and outputs feature map 3. Figure 4 ;

[0027] Among them, the ratio of the feature map size input to the output feature map size of the conv2 block, conv3 block, conv4 block, and conv5 block is equal to 2:1, and the number of channels corresponding to the feature maps output by the conv2 block, conv3 block, conv4 block, and conv5 block increases in sequence by a factor of 2; z1, z2, z3, and z4 are all positive integers and are empirical values, which can be adaptively adjusted according to the model accuracy.

[0028] Further optionally, the input of the feature fusion network is the four feature maps output by the feature extraction network, which are respectively denoted as: C2, C3, C4, and C5;

[0029] First, the feature maps C2, C3, C4, and C5 are converted into feature maps D2, D3, D4, and D5 respectively through convolution in the feature fusion network to unify the number of channels.

[0030] Then, after upsampling the feature map D5, the feature map D5 is added to the feature map D4 to obtain the feature map E4; after upsampling the feature map E4 and the feature map D5, the three are added to the feature map D3 to obtain the feature map E3; after upsampling the feature map E3, the feature map D2 is added to obtain the feature map P2; after downsampling the feature map P2, the feature map E3 is added to the feature map E3 to obtain the feature map P3; after downsampling the feature map P3 and the feature map P2, the feature map E4 is added to obtain the feature map P4; after downsampling the feature map P4, the feature map D5 is added to obtain the feature map P5, and finally the feature map P5 is subjected to the maxpooling operation to obtain the feature map P6. Among them, the size of the feature maps when superimposed on each other is consistent, and the number of channels matches; the feature fusion network outputs feature maps P2, P3, P4, P5, and P6.

[0031] Further optionally, the process of constructing the data set in step S1 is as follows:

[0032] Image acquisition: collect target liquid images and process them into a uniform size;

[0033] Calibration and cropping of visible foreign matter: calibrating the position and category of the visible foreign matter on the target liquid image, and cropping the visible foreign matter image on the target liquid image;

[0034] Threshold processing: Threshold processing is performed on the cropped visible foreign body image; the RGB channel values ​​of the pixels in the background area are set to 0 or 255, while the RGB channel values ​​of the pixels in the visible foreign body area remain unchanged;

[0035] Deformation: Randomly rotate and randomly mirror the thresholded visible foreign body image;

[0036] Data augmentation: Randomly add visible foreign objects after random rotation and random mirroring to the liquid area of ​​the target body fluid image to form a data set.

[0037] Further optionally, in step S3, when training the neural network model using the data set, a stochastic gradient descent method is used to perform network optimization;

[0038] Among them, the loss function for:

[0039]

[0040] Among them, N pos is the number of positive samples, which are visible foreign objects; is the GIoU loss, is the QFL loss, is the DFL loss, λ0 and λ1 are constant coefficients used to set and f(z) is the indicator function, that is, the value is 1 when there is visible foreign matter in the regression frame, and the value is 0 otherwise.

[0041] In a second aspect, the present invention further provides a system based on the pharmaceutical quality detection method, comprising:

[0042] The data set construction module is used to construct the data set, that is, to collect the target liquid image and calibrate the position and category of the visible foreign matter on the target liquid image;

[0043] A neural network model building module is used to build a neural network model that connects the feature extraction network, the feature fusion network, and the boundary distribution attention detection head network in sequence;

[0044] The input of the feature extraction network is the target liquid image, the output of the feature extraction network serves as the input of the feature fusion network, the output of the feature fusion network serves as the input of the boundary distribution attention detection head network, and the output of the boundary distribution attention detection head network is the category to which each pixel belongs and the distance from the four boundaries of the regression box with each pixel as the center point to the center point, and the category is the visible foreign matter category or background;

[0045] A training module, configured to train a neural network model using the data set;

[0046] The detection module is used to input the target liquid image to be detected into the trained neural network model to obtain the detection result of visible foreign matter.

[0047] In a third aspect, the present invention further provides an electronic terminal, comprising:

[0048] one or more processors;

[0049] a memory storing one or more computer programs;

[0050] The processor calls the computer program to implement:

[0051] A method for detecting pharmaceutical quality based on boundary-distributed attention.

[0052] In a fourth aspect, the present invention further provides a readable storage medium storing a computer program, wherein the computer program is called by a processor to implement:

[0053] A method for detecting pharmaceutical quality based on boundary-distributed attention.

[0054] Beneficial effects

[0055] The pharmaceutical quality detection method provided by the present invention implements a deep learning-based visible foreign matter detection algorithm for liquid medicine by constructing a neural network model including a feature extraction network, a feature fusion network, and a boundary distribution attention detection head network. This provides another new technical approach and means to achieve visible foreign matter detection, effectively solving various problems of traditional manual detection. Furthermore, the technical solution of the present invention creatively designs a neural network including a boundary distribution attention detection head network, which can analyze the importance of each boundary distribution of the regression frame, thereby guiding pixel classification, accurately identifying the visible foreign matter area, improving the recognition accuracy of visible foreign matter detection, and reducing the false detection rate. Among them, the method proposed by the present invention can guide the classification result based on the distribution of the boundaries of the four edges after attention weighting. The attention weighting module can automatically focus on the edges with larger distribution variance during network training optimization, and then reduce the variance of the distribution through backpropagation, which can accelerate the convergence speed and increase the detection accuracy of the network. At the same time, the detection head classification branch and regression branch are connected, solving the problem that the training phase and prediction phase of the traditional detection head classification branch and regression branch are inconsistent, resulting in higher scores for negative sample frames than positive samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a schematic diagram of the visible foreign matter extraction results;

[0057] Figure 2a-2c It is a schematic diagram of the comparison of randomly adding 3 visible foreign objects, 5 visible foreign objects and the original liquid image; Figure 2a is the original solution image, Figure 2b This is the image of the liquid medicine after 3 visible foreign objects are randomly added. Figure 2c This is the image of the drug solution after 5 visible foreign objects are randomly added;

[0058] Figure 3a-Figure 3b is a comparison of visible foreign matter before and after threshold processing, where Figure 3a is the image before threshold processing, Figure 3b is the image after threshold processing;

[0059] Figure 4 It is the feature extraction network structure diagram;

[0060] Figure 5 It is the feature fusion network structure diagram;

[0061] Figure 6 This is the network structure diagram of the boundary distribution attention detection head;

[0062] Figure 7a-7b This is a comparison of the original drug solution image, the label image, and the neural network prediction result, where: Figure 7a is the original solution image, Figure 7b is the label image, Figure 7c Predict results for the neural network;

[0063] Figure 8 It is the AP50 curve of each round in the validation set during the training process. DETAILED DESCRIPTION

[0064] The present invention provides a pharmaceutical quality inspection method based on boundary distributed attention for identifying visible foreign matter in liquids. Specifically, it is applied to identifying visible foreign matter in liquid medicines on a production line. This method will be described in detail below using this example. It should be understood that the technical solution of the present invention is also applicable to the detection of visible foreign matter in other liquids. The present invention will be further illustrated below with reference to examples.

[0065] Example 1:

[0066] The pharmaceutical quality inspection method based on boundary distributed attention, provided by an embodiment of the present invention, is a deep learning-based algorithm for detecting visible foreign matter in pharmaceutical solutions. Therefore, one of its core components is a constructed neural network model. The neural network model provided by the technical solution of the present invention primarily comprises three components: a feature extraction network, a feature fusion network, and a boundary attention detection head network.

[0067] like Figure 4As shown, the feature extraction network is used to extract high-level features of liquid images. Therefore, the input of the feature extraction network is the liquid image and label in the data set. The label is a calibration of whether the pixel is a visible foreign matter and the type and position of the visible foreign matter. The feature extraction network and feature fusion network of this embodiment are constructed based on the Resnet50 network, and the fully connected layer in the original Resnet50 network is deleted. The output of the feature extraction network in this embodiment is four feature maps, with sizes of 96*75, 48*37, 24*18, and 12*9, respectively, and the unit is the number of pixels. The feature extraction network is provided with conv1 block, conv2 block, conv3 block, conv4 block, and conv5 block. The specific details are as follows: the conv1 block is a convolution kernel of size 7*7 and stride 2, with a total of 64. After passing through the conv1 block, the target liquid image becomes a feature map of size 385*300 and the number of channels is 64. After a 3*3 maxpooling operation with a step size of 2, it becomes a feature map of size 192*150 and the number of channels is 64, which is sent to the conv2 block. The conv2 block repeats three rounds of operations, in which 64 1*1 convolutions, 64 3*3 convolutions, and 256 1*1 convolutions are stacked in sequence, and there is a shortcut connection between the input and output. After passing through the conv2 block, the feature map becomes a feature map of size 96*75 and the number of channels is 256, which is used as the first output of the feature extraction network (feature map). Figure 1 ); and then the feature Figure 1 The conv3 block repeats four rounds of operations, in which 128 1*1 convolutions, 128 3*3 convolutions, and 512 1*1 convolutions are stacked in sequence, and there is a shortcut connection between the input and output. Figure 1 After passing through the conv3 block, it becomes a feature map of size 48*37 and number of channels 512, which is used as the second output of the feature extraction network (feature map 2); then the feature map 2 is sent to the conv4 block, and the conv4 block repeats six rounds of operations, in which 256 1*1 convolutions, 256 3*3 convolutions, and 1024 1*1 convolutions are stacked in sequence, and there is a shortcut connection between the input and output. After passing through the conv4 block, the feature map becomes 24*18 in size and number of channels 1 024 feature map, as the third output of the feature extraction network (feature map 3), and then send the feature map 3 to the conv5 block, which repeats three rounds of operations, in which 512 1*1 convolutions, 512 3*3 convolutions, and 2048 1*1 convolutions are stacked in sequence, and there is a shortcut connection between the input and output. After passing through the conv5 block, the feature map 3 becomes a feature map of size 12*9 and the number of channels is 2048, which is used as the fourth output of the feature extraction network (feature map 3). Figure 4 ), at this point, the feature extraction network is constructed.

[0068] It should be noted that when setting up the feature extraction network of this embodiment, it is necessary to set the convolution kernel size, step size, and number of conv1 blocks, conv2 blocks, conv3 blocks, conv4 blocks, and conv5 blocks, and the settings of the above parameters are all adjusted based on the model accuracy obtained from the experiment (for example, continuous adjustment when using a data set for training). Among them, the above parameters are preferably set so that the ratio of the feature map size of the input of the conv2 block, conv3 block, conv4 block, and conv5 block to the output feature map size is equal to or close to 2:1 (such as 75 / 2=37.5, the number of pixels cannot be 0.5, so the convolution operation will automatically change the number of convolutions according to the image size, that is, directly round down. Therefore, the present invention will also regard approximately 2:1 as 2:1), and the number of channels increases by 2 times. The purpose is to facilitate subsequent feature fusion and detection of foreign objects by the detection head.

[0069] like Figure 5 As shown in the figure, the feature fusion network is used to fuse the four feature maps output by the feature extraction network. Therefore, the four feature maps of the feature extraction network are the input of the feature fusion network, that is, the feature maps of the feature extraction network output sizes of 96*75, 48*37, 24*18, and 12*9 Figure 1-4 They are named C2, C3, C4, and C5 in sequence. Feature maps C2, C3, C4, and C5 are all subjected to 256 1*1 convolutions to unify the number of channels to 256 (the original channels were 256, 512, 1024, and 2048, respectively). The feature maps after channel unification are also named D2, D3, D4, and D5 in sequence. Feature map D5 is upsampled twice using the CARAFE method and then added to feature map D4 to obtain feature map E4. Feature map E4 is upsampled twice using the CARAFE method and then quadrupled using the CARAFE method. The two are added together and then added to feature map D3 to obtain feature map E3. Feature map E3 is upsampled twice using the CARAFE method and then added to feature map D2 to obtain feature map P2. Next, feature map P2 is downsampled by a factor of two and added to feature map E3 to obtain feature map P3. Feature map P3 is downsampled by a factor of two, and then feature map P2 is downsampled by a factor of four. After adding, feature map P2 is added to feature map E4 to obtain feature map P4. Feature map P4 is downsampled by a factor of two and added to feature map D5 to obtain feature map P5. Finally, feature map P5 is subjected to a maxpooling operation with a size of 3*3 and a step size of 2 to obtain feature map P6 with a size of 6*4 and a number of channels of 256. The outputs of the final feature fusion network are feature maps P2, P3, P4, P5, and P6. At this point, the feature fusion network is constructed.

[0070] It should be understood that the configuration of the above-mentioned feature fusion network is related to the parameter setting of the feature extraction network, and the rule it should satisfy is: the parameter setting of the fusion network should make the superimposed feature maps of the same size and the number of channels match. Because the feature map is a matrix, it cannot be added if the size is inconsistent. The fusion logic of the fusion network in this embodiment is: after upsampling the feature map D5, it is added to the feature map D4 to obtain the feature map E4; after upsampling the feature map E4 and the feature map D5, it is added to the feature map D3 to obtain the feature map E3; after upsampling the feature map E3, it is added to the feature map D2 to obtain the feature map P2; after downsampling the feature map P2, it is added to the feature map E3 to obtain the feature map P3; after downsampling the feature map P3 and the feature map P2, it is added to the feature map E4 to obtain the feature map P4; after downsampling the feature map P4, it is added to the feature map D5 to obtain the feature map P5, and finally, after the feature map P5 is subjected to the maxpooling operation, the feature map P6 is obtained. Among them, the fusion logic set up in the present invention can effectively reduce the distance between the bottom and top layer features, reduce the feature loss caused by intermediate operations, effectively retain the features of tiny foreign objects, and improve the detection accuracy of small foreign objects.

[0071] like Figure 6 As shown in the figure, the input of the boundary distributed attention detection head network is the five feature maps P2, P3, P4, P5, and P6 output by the feature fusion network. The output is the category to which each pixel in the feature map belongs (visible foreign object category or background) and the distance from the four edges of the regression box to the center point of each pixel. The boundary distributed attention detection head network mainly includes a boundary prediction branch and a classification prediction branch.

[0072] In this embodiment, the distance from the four boundaries of the predicted regression box to the center point is calculated by the expected value of the discrete distribution. In this embodiment, the distance distribution from each boundary to the center point is represented by 16 discrete prediction values, where the horizontal coordinate of the distance distribution is distance and the vertical coordinate is probability; there are 64 prediction values ​​for the four boundaries, so the feature maps P2, P3, P4, P5, and P6 are first copied into four copies, each of which is used to predict one edge. First, 16 1*1 convolutions are set on the boundary prediction branch, and then the number of channels of the 20 copied feature maps is changed to 16, and each channel is used to predict the distance from the corresponding edge to the center point. The following is an example to illustrate the distance:

[0073] Taking a pixel point in the feature map as the center point, the boundary prediction branch predicts one side of its regression frame and obtains a prediction result (prediction value) as a series: [0, 0, 0.02, 0.03, 0.06, 0.78, 0.11, 0], where the first number in the series represents the probability that the distance to the center point is 0; the second number in the series represents the probability that the distance to the center point is 1; the third number in the series represents the probability that the distance to the center point is 2, and so on. The above is a numerical representation using 8 discrete distributions. The embodiment of the present invention uses a numerical representation of 16 discrete distributions, and finally calculates the expected value of the discrete distribution as the distance from the boundary to the center point. If the same regression frame has four boundaries, the above calculation is performed four times to determine the regression frame. It should be understood that the setting of 16 discrete distribution values ​​in this embodiment is one of the best values ​​obtained through experiments. In other feasible embodiments, adaptive adjustment of the number according to the network accuracy requirements also falls within the scope of protection of the present invention, that is, each boundary corresponds to N1 predicted values, then the four boundaries correspond to 4N1 predicted values, and the number of channels is N1.

[0074] In the present embodiment, the pixel category prediction process is also to first copy the feature maps P2, P3, P4, P5, and P6 into four copies. Assuming that there are C categories of visible foreign objects and the size of the feature map is H*W, where H is height and W is width, the classification prediction branch first uses C 1*1 convolutions to change the number of channels of the copied feature map to the number of categories C, and then performs a sigmoid operation on it to ensure that each channel outputs the probability value of the corresponding category.

[0075] The prediction value corresponding to each boundary obtained by the boundary prediction branch is used to guide the determination of the category to which each pixel point on the classification prediction branch belongs. The specific operations of this embodiment are as follows: first, the q most probable values ​​in the discrete distribution of each edge of the boundary prediction branch are taken out. In this embodiment, the 4 most probable values ​​are set to be taken out. The taken out values ​​can more comprehensively represent the distribution of boundary prediction. A feature map has a total of H*W pixels, and the four edges have a total of 16 values ​​(or expressed as 4q). Therefore, the number of taken out values ​​is H*W*16 (or expressed as H*W*4q). Then calculate the mean of the distribution of the four edges, a total of H*W*4 values, denoted as Xm, and perform concat operation on the extracted value and the mean to form a tensor value of size H*W*20 (or expressed as H*W*(4+4q)), denoted as Xc; divide the extracted value into four parts according to the boundary, each part is H*W*4 (or expressed as H*W*q), and each part corresponds to a boundary, where the extracted values ​​of the four edges of up, down, left and right (tblr) are denoted as Xt, Xb, Xl, and Xr respectively. Use 64 shared weights of 1*1 convolution to change the number of channels of Xt, Xb, Xl, Xr, and Xm to 64, and then use 1*1 shared weight convolution to change the number of channels to 1, and then pass a sigmoid to obtain the guidance score S corresponding to the four edges of up, down, left and right. t 、S b 、S l 、S r , and the guidance score S corresponding to the mean m Then use 128 1*1 convolutions to change the number of channels of Xc to 128, and then use 5 1*1 convolutions to change the number of channels to 5, and then pass a sigmoid to get S t 、S b 、S l 、S r and S m The proportion values ​​K1, K2, K3, K4, and K5 in the total guidance score are added to draw attention to the boundaries. Edges with unclear boundary distribution will have a larger proportion value. The final total guidance score is calculated as follows:

[0076] Score=sigmoid(K1*S t +K2*S b +K3*S l +K4*S r +K5*S m )

[0077] Finally, the score is multiplied by the result obtained by the classification prediction branch as the final output of the classification prediction branch. At this point, the boundary distribution attention detection head network is completed.

[0078] Based on the above neural network model, an embodiment of the present invention provides a medicine quality detection method based on boundary distributed attention, which includes the following steps:

[0079] S1: Construct a data set, i.e., collect target liquid images and calibrate the positions and categories of visible foreign matter on the target liquid images.

[0080] In order to improve recognition accuracy and increase the diversity of visible foreign matter in samples, this embodiment constructs a data set according to the following steps:

[0081] Image acquisition: Capture target liquid images and process them into a uniform size. In this embodiment, images of liquid medicine with visible foreign matter on a pharmaceutical inspection production line are captured and the sizes of the liquid medicine images are processed into a uniform size.

[0082] Calibration and cropping of visible foreign matter: Calibrate the position and category of visible foreign matter on the target liquid image, and crop the visible foreign matter image on the target liquid image. In this embodiment, a production software (such as labelImg) is used to calibrate the position of visible foreign matter on the image and produce a dataset of visible foreign matter in the liquid medicine; then, according to the recorded position of the visible foreign matter, the corresponding visible foreign matter image is cropped out from the liquid medicine image. Figure 1 As shown in FIG, the cropped visible foreign body image generally has redundant background.

[0083] Threshold processing: threshold processing is performed on the cropped visible foreign body image; Figure 3a-Figure 3b As shown, the RGB channel values ​​of the pixels in the background area are set to 0 or 255, and the RGB channel values ​​of the pixels in the visible foreign matter area remain unchanged. Specifically, by observing the specific values ​​of the three RGB channels of the image, since the visible foreign matter and the light inspection background are different in color, a suitable RGB threshold can be selected to distinguish the visible foreign matter from the redundant background. After selecting the threshold, the visible foreign matter image is threshold processed - if the background is darker than the visible foreign matter, the pixels with RGB channel values ​​greater than the threshold are classified as background, and the values ​​of the three RGB channels are all set to 0. The pixels with values ​​of the three RGB channels less than or equal to the threshold are classified as visible foreign matter. The RGB channel values ​​of the pixels before and after processing remain unchanged. If the background color is lighter than the visible foreign matter, the pixels with values ​​of the three RGB channels less than the threshold are classified as background, and the values ​​of the three RGB channels are all set to 255. The pixels with values ​​of the three RGB channels greater than or equal to the threshold are classified as visible foreign matter. The RGB channel values ​​of the pixels before and after processing remain unchanged.

[0084] It should be noted that liquid medicine foreign body detection is divided into bright field and dark field. Generally, once the scene is determined, the scene will not be changed when the dataset is created. Therefore, it is only necessary to make a judgment once - whether the foreign body is darker or lighter than the background color. For example, in a dark field, the color of the foreign body must be lighter than the background. The foreign body is white (255) and the background is black (0). The threshold is set by randomly selecting several liquid medicine images, then observing the RGB channel values ​​of the foreign body images, and then manually selecting a threshold that can distinguish the foreign body from the background, and then applying it to all images.

[0085] Deformation: Randomly rotate and randomly mirror the thresholded visible foreign matter image. In this embodiment, the thresholded visible foreign matter image is randomly rotated and randomly mirrored to generate a new visible foreign matter image. The rotation angle is generally set to 0°, 90°, 180°, and 270°.

[0086] Data augmentation: Randomly add visible foreign matter in the liquid area of ​​the target body fluid image after random rotation and random mirroring to form a data set. The purpose of this step is to enrich the visible foreign matter in the original liquid image. Specifically:

[0087] First determine the range of the liquid medicine in the image. Visible foreign matter can only be added within this range. Since the visible foreign matter image itself has a certain size, when randomly adding the visible foreign matter image to the original liquid medicine image, it is necessary to limit the random coordinate value range of the upper left corner of the visible foreign matter addition position. The coordinates of the upper left corner of the liquid medicine range are marked as (xmin, ymin), and the coordinates of the lower right corner are marked as

[0088] (xmax, ymax); the width of the image to be added with visible foreign matter is recorded as w, and the height is recorded as h; the horizontal coordinate value range of the random coordinate value of the upper left corner of the visible foreign matter to be added is recorded as [xl, xr], and the vertical coordinate value range is recorded as [yt, yb]. The random coordinate value range of the upper left corner of the visible foreign matter addition position can be calculated based on the range of the liquid in the image according to the following formula:

[0089]

[0090] Then, we randomly add visible foreign matter into the range of the liquid medicine. That is, we randomly determine the upper left corner coordinate value of the position where the visible foreign matter is added within the range of the upper left corner coordinate value. Starting from the upper left corner coordinate value, we replace the pixels corresponding to the original liquid medicine image with the pixels whose visible foreign matter image value is not 0 or 255. The number of visible foreign matter added to each image can be determined by ourselves. At this point, the data enhancement is completed. The comparison before and after data enhancement is shown in the figure below. Figure 2a-2c shown.

[0091] In this embodiment, a dataset of drug solution images was constructed and randomly divided into a training set, a test set, and a validation set in a ratio of 7:2:1. The training set was used to train the neural network model, and the test set and validation set were used to test and verify the accuracy of the neural network model.

[0092] It should be noted that, in other feasible embodiments, without departing from the technical concept of the present invention, not performing data enhancement on the data also falls within the scope of protection of the present invention.

[0093] S2: Construct a neural network model that connects the feature extraction network, feature fusion network and boundary distribution attention detection head network in sequence. It should be understood that according to the above content, Figure 4-6 The feature extraction network, feature fusion network, and boundary distribution detection head network shown are used to construct a neural network model. As mentioned above, the input of the neural network model of this embodiment is the drug liquid image, and the output is the position and category of the foreign matter on the target drug liquid image, that is, the position of the four sides of the regression frame of the foreign matter and the category of the foreign matter. That is, each pixel point is used as the center point to predict the distance from the four sides of the regression frame of the visible foreign matter to the center point. Each edge will have 16 prediction results, and the expected value of these 16 prediction results (distance distribution) is calculated as the final prediction result of the corresponding edge.

[0094] S3: Using the data set to train the neural network model. In this embodiment, the training set, test set, and validation set are used to train, test, and validate the neural network model. Preferably, the stochastic gradient descent method is used for optimization during training, and the loss function is:

[0095]

[0096] Among them, N pos is the number of positive samples (visible foreign objects), is the GIoU loss, is the QFL loss, is the DFL loss, λ0 and λ1 are constant coefficients used to set and The proportion of λ0 is usually 0.25, and λ1 is usually 2. f(z) is an indicator function, that is, the value is 1 when there is a visible foreign object in the regression frame, and the value is 0 otherwise. z corresponds to the number of regression frames, that is, the losses of all regression frames need to be superimposed.

[0097] The QFL loss is expressed as follows:

[0098]

[0099] Among them, y is the label value and σ is the regression box classification score predicted by the neural network model.

[0100] The DFL loss is expressed as follows:

[0101]

[0102] Among them, y i+1 and y i are the i-th and i+1-th prediction results of the distance from one edge of the regression box to the center point, and y is the label value.

[0103] The GIoU loss is shown in the following formula:

[0104]

[0105] Among them, A is the regression box (regression box), B is the label real box, C is the minimum bounding box that can surround A and B, and IoU is:

[0106]

[0107] It should be understood that the updated weights after training (corresponding to the convolution kernel parameters in the neural network, and the convolution kernel parameters are continuously optimized through training of the network with data sets) are imported into the neural network model to complete the actual construction of the neural network model. Subsequently, the image output of the camera that takes pictures of the light inspection liquid is connected to the input of the neural network to realize the detection of visible foreign matter in the liquid.

[0108] S4: Inputting the target liquid image to be detected into the trained neural network model to obtain the detection result of visible foreign matter.

[0109] In the present embodiment, taking Hunan University's pharmaceutical assembly production line as an example, the graphics card of the experimental platform is 3080ti, the system used is ubuntu18.04, and the python version is 3.8. First, a total of 600 pictures with visible foreign matter are taken, and each visible foreign matter of each picture is marked. The specific details of the marking are: all images are cut to 770*600 size, wherein 770 is the width of the image, and 600 is the height of the image, and the unit is the number of pixels. Then the labelImg software is used to open the picture taken, and after clicking the create box button, the visible foreign matter is selected with the mouse frame and the category is selected. When the visible foreign matter is selected, the regression frame is made as close to the edge of the visible foreign matter as possible to avoid framing unnecessary background, resulting in poor neural network training effect. After completing the frame selection of all visible foreign matter according to the operation, a .xml file corresponding to the picture name is generated, which contains the position and category information of the visible foreign matter corresponding to each picture. The dataset was then divided into a training set: test set: validation set ratio of 7:3:1, with 420 images in the training set, 120 images in the test set, and 60 images in the validation set. After creating the dataset, it was formatted according to the VOC format. The images were then processed according to the previous steps. The data-augmented dataset was then trained on the neural network model with a learning rate of 0.001 and 110 training epochs. The learning rate was then reduced to 0.0001 between epochs 70 and 100, and then to 0.00001 between epochs 100 and 110. The stochastic gradient descent method is used to optimize the above loss function and the convergence value of the loss function is obtained. After the training round reaches 110, the weight corresponding to the round with the highest accuracy in the test set is taken out and imported into the neural network (the so-called weight corresponds to the convolution kernel parameter in the neural network. The convolution kernel parameter is continuously optimized through the training of the network with the data set to reduce the loss value). The input of the neural network is then connected to the image output of the light inspection camera. After the neural network receives the image, it predicts visible foreign matter and filters out the results with a predicted classification score lower than 0.3. At the same time, the opencv library is used to display the prediction results of the neural network and the liquid medicine picture on the screen, thereby realizing the function of visible foreign matter detection and positioning. In actual measurements in this experimental environment, the algorithm can reach 19FPS and AP50 can reach 95.3%, which can meet the requirements of high-speed liquid medicine visible foreign matter detection.

[0110] In summary, the technical solution of the present invention creatively designs a neural network including a boundary distribution attention detection head network, which can analyze the importance of each boundary distribution of the regression box, thereby guiding pixel classification, accurately identifying visible foreign object areas, improving the recognition accuracy of visible foreign object detection, and reducing the false detection rate. Specifically:

[0111] Detecting foreign matter in liquids, especially pharmaceuticals, requires rotating the liquid to move any foreign matter at the bottom of the bottle, facilitating detection. However, moving foreign matter can be easily obscured by bubbles or other foreign matter, leading to uncertainty in the regression frame for the foreign matter. Conventional regression frames cannot resolve this uncertainty. The boundary distribution proposed in the pharmaceutical quality testing method provided by this invention can represent this boundary uncertainty and eliminate it through backpropagation, thereby improving accuracy.

[0112] 2. In traditional detection heads, the classification branch and regression branch are trained separately, and the two are not connected to each other. However, during prediction, the classification score and the regression score are multiplied to represent the final quality score of the regression frame. This may lead to the problem that the negative sample frame may have a higher quality score than the positive sample frame, that is, the regression score and the classification score are not connected during training, resulting in the regression frame with poor quality having a higher score than the regression frame with good quality. It is obviously unreasonable to use this method to evaluate the quality of the regression frame, which will affect the detection effect of the network. The boundary distribution attention-guided detection head proposed in the present invention can guide the classification results according to the distribution of the boundaries of the four edges after attention weighting, and establish a connection between the two to solve the above problem; in addition, an attention mechanism is added to the boundary distribution-guided classification. Through the attention module, the network pays more attention to the uncertain distribution edges in the regression frame when guiding the classification, thereby improving the contribution of the distribution of uncertain edges in the regression frame score, thereby better guiding the classification results.

[0113] 3. This paper proposes a new feature fusion structure lightweight upsampling multi-feature fusion network, such as Figure 5 As shown in the figure, in order to obtain more information about tiny foreign objects, the feature maps of layers C2 to C5 are obtained for fusion. In addition to introducing the top-down and bottom-up feature fusion branches, downsameple shortcut and upsameple shortcut connections are added to the separated feature layers, which reduces the distance between the bottom and top features and reduces the feature loss caused by intermediate operations. It can effectively retain the features of tiny foreign objects and improve the detection accuracy of tiny foreign objects, while only introducing a small amount of computation.

[0114] In summary, in order to solve the problem that there are many types of foreign objects that are difficult to classify, the present invention has built a deep learning framework. Compared with traditional machine learning methods, it can effectively extract the deep feature information of foreign objects, and then perform classification and regression based on the feature information, which greatly improves the accuracy of foreign object recognition. In order to solve the problem that there are many interferences in the detection process and foreign objects are blocked, which can easily lead to misjudgment, a boundary distribution attention detection head is proposed. The boundary distribution guides the classification result after attention weighting, establishes the connection between regression and classification, can reduce misjudgment, and increase positioning accuracy. In order to solve the problem that foreign objects are small and difficult to detect, a lightweight upsampling multi-feature fusion network is proposed, which solves the problem of foreign object information loss caused by multiple downsampling. It can fuse the deep feature map with high-level semantics with the low-level feature map with tiny foreign object information to obtain a feature map that has both high-level semantic information and retains tiny foreign objects. At the same time, the CARAFE upsampling method is introduced for the tiny characteristics of foreign objects, which can greatly increase the semantic information between feature map pixels and improve the detection accuracy of tiny foreign objects. Figure 7a-7c as well as Figure 8 It can also be seen that the method shown in the present invention has good performance and accuracy.

[0115] Example 2:

[0116] This embodiment provides a system based on the pharmaceutical quality detection method, which includes: a data set construction module, a neural network model construction module, a training module and a detection module.

[0117] Among them, the data set construction module is used to construct the data set, that is, to collect the target liquid image and calibrate the position and category of the visible foreign matter on the target liquid image.

[0118] A neural network model construction module is used to construct a neural network model in which a feature extraction network, a feature fusion network and a boundary distribution attention detection head network are connected in sequence; wherein, the input of the feature extraction network is a liquid image, the output of the feature extraction network is used as the input of the feature fusion network, the output of the feature fusion network is used as the input of the boundary distribution attention detection head network, and the output of the boundary distribution attention detection head network is the category to which each pixel point belongs and the distance from the four boundaries of the regression box with each pixel point as the center point to the center point, and the category is a visible foreign body category or background.

[0119] The training module is used to train the neural network model using the data set. The detection module is used to input the liquid image to be detected into the trained neural network model to obtain the detection result of visible foreign matter.

[0120] Please refer to the above-mentioned methods for the specific implementation process of each module, and will not be elaborated here. It should be understood that the above-mentioned division of functional modules is merely a division of logical functions. In actual implementation, other division methods can be used. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not implemented. At the same time, the above-mentioned integrated units can be implemented in the form of hardware or software functional units.

[0121] Example 3:

[0122] This embodiment provides an electronic terminal comprising one or more processors and a memory storing one or more computer programs, wherein the processor calls the computer program to implement the steps of a medicine quality detection method based on boundary distributed attention, specifically performing:

[0123] S1: Construct a data set, i.e., collect target liquid images and calibrate the positions and categories of visible foreign matter on the target liquid images.

[0124] S2: Construct a neural network model that connects the feature extraction network, feature fusion network, and boundary distribution attention detection head network in sequence.

[0125] Among them, the input of the feature extraction network is a liquid image, the output of the feature extraction network serves as the input of the feature fusion network, the output of the feature fusion network serves as the input of the boundary distribution attention detection head network, and the output of the boundary distribution attention detection head network is the category to which each pixel point belongs and the distance from the four boundaries of the regression box with each pixel point as the center point to the center point. The category is a visible foreign body category or background.

[0126] S3: Using the data set to train a neural network model.

[0127] S4: Inputting the liquid image to be detected into the trained neural network model to obtain the detection result of visible foreign matter.

[0128] The terminal also includes a communication interface for communicating with external devices and performing data exchange transmission.

[0129] The memory may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0130] If the memory, processor, and communication interface are implemented independently, they can be interconnected via a bus and communicate with each other. The bus can be an industrial standard architecture bus, a peripheral device interconnect bus, or an extended industrial standard architecture bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0131] Optionally, in a specific implementation, if the memory, processor, and communication interface are integrated on a chip, the memory, processor, and communication interface can communicate with each other through an internal interface.

[0132] For the specific implementation process of each step, please refer to the description of the above method.

[0133] It should be understood that in the embodiments of the present invention, the processor referred to may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may include a read-only memory and a random access memory, and provides instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0134] Example 4:

[0135] This embodiment provides a readable storage medium storing a computer program, which is called by a processor to implement the steps of a medicine quality detection method based on boundary distributed attention, specifically performing:

[0136] S1: Construct a data set, i.e., collect target liquid images and calibrate the positions and categories of visible foreign matter on the target liquid images.

[0137] S2: Construct a neural network model that connects the feature extraction network, feature fusion network, and boundary distribution attention detection head network in sequence.

[0138] Among them, the input of the feature extraction network is a liquid image, the output of the feature extraction network serves as the input of the feature fusion network, the output of the feature fusion network serves as the input of the boundary distribution attention detection head network, and the output of the boundary distribution attention detection head network is the category to which each pixel point belongs and the distance from the four boundaries of the regression box with each pixel point as the center point to the center point. The category is a visible foreign body category or background.

[0139] S3: Using the data set to train a neural network model.

[0140] S4: Inputting the liquid image to be detected into the trained neural network model to obtain the detection result of visible foreign matter.

[0141] The readable storage medium is a computer-readable storage medium, which may be an internal storage unit of the controller described in any of the aforementioned embodiments, such as a hard disk or memory of the controller. The readable storage medium may also be an external storage device of the controller, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the controller. Furthermore, the readable storage medium may also include both an internal storage unit of the controller and an external storage device. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0142] Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0143] It should be emphasized that the examples described in the present invention are illustrative rather than restrictive. Therefore, the present invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solution of the present invention that do not depart from the purpose and scope of the present invention, whether modified or replaced, also fall within the scope of protection of the present invention.

Claims

1. A medicine quality detection method based on boundary distributed attention, characterized by: The following steps are involved: S1: Constructing a dataset: Acquire target liquid images and calibrate the locations and categories of visible foreign matter on the target liquid images; S2: Construct a neural network model that connects the feature extraction network, feature fusion network, and boundary distribution attention detection head network in sequence; The input of the feature extraction network is the target liquid image, the output of the feature extraction network serves as the input of the feature fusion network, the output of the feature fusion network serves as the input of the boundary distribution attention detection head network, and the output of the boundary distribution attention detection head network is the category to which each pixel belongs and the distance from the four boundaries of the regression box with each pixel as the center point to the center point. The category is a visible foreign body category or background; S3: training the neural network model using the data set; S4: inputting the target liquid image to be detected into the trained neural network model to obtain a detection result of visible foreign matter; The boundary distribution attention detection head network is provided with a boundary prediction branch for predicting the distance from the boundary to the center point of the regression box and a classification prediction branch for predicting the category to which the pixel point belongs; For each boundary of the regression box with the pixel as the center point, the boundary prediction branch predicts N1 discretely distributed prediction values, and then combines the N1 prediction values ​​to obtain the distance from the boundary to the center point, where N1 is a positive integer; For each pixel point, the classification prediction branch determines the category to which each pixel point belongs under the guidance of the prediction value obtained by the boundary prediction branch; Specifically, the predicted values ​​of each boundary on the regression box of the same pixel point are used to obtain the total guidance score, and then the total guidance score is multiplied by the classification prediction branch to predict the initial value of each category probability of the same pixel point to obtain the final value of each category probability, among which the category corresponding to the largest category probability final value is the category to which the pixel point belongs.

2. The pharmaceutical quality testing method according to claim 1, wherein: The process of obtaining the total guidance score is as follows: Score=sigmoid(K1*S t +K2*S b +K3*S l +K4*S r +K5*S m ) In the formula, Score is the total guidance score, sigmoid is a nonlinear action function of a neuron, S t 、S b 、S l 、S r are the guidance scores corresponding to the four boundaries of the regression box, S m is the guidance score corresponding to the mean, K1, K2, K3, K4, K5 represent S t 、S b 、S l 、S r and S m The percentage of the total guidance score; Among them, the boundary prediction branch predicts N1 prediction values ​​corresponding to each boundary on the regression box where the pixel point is located, and then selects q prediction values ​​with the highest probability from the N1 prediction values ​​of each boundary, and then calculates the mean Xm corresponding to each boundary, and then concatenates the q prediction values ​​with the highest probability corresponding to each boundary with the mean Xm, recorded as Xc, and uses convolution to adjust the number of channels and a sigmoid function for Xc to obtain the proportion values ​​K1, K2, K3, K4, and K5. Divide Xc into four parts according to the boundary, record them as Xt, Xb, Xl, Xr, and then use the convolution with shared weights to adjust the number of channels of Xt, Xb, Xl, Xr, and Xm, and then pass a sigmoid function to obtain the guidance scores S corresponding to the four boundaries. t 、S b 、S l 、S r 、S m .

3. The pharmaceutical quality testing method according to claim 1, wherein: The feature extraction network is provided with conv1 block, conv2 block, conv3 block, conv4 block and conv5 block; Among them, the target liquid image is first input into the conv1 block, and then the maxpooling operation is performed and then input into the conv2 block. The conv2 block repeats the z1 round operation and outputs feature map 1; then the feature map 1 is input into the conv3 block, and the conv3 block repeats the z2 round operation and outputs feature map 2; the feature map 2 is input into the conv4 block, and the conv4 block repeats the z3 round operation and outputs feature map 3; the feature map 3 is input into the conv5 block, and the conv5 block repeats the z4 round operation and outputs feature map 4; Among them, the ratio of the feature map size input to the conv2 block, conv3 block, conv4 block, and conv5 block to the output feature map size is equal to or approximately equal to 2:1, and the number of channels corresponding to the feature maps output by the conv2 block, conv3 block, conv4 block, and conv5 block increases in sequence by a factor of 2, and z1, z2, z3, and z4 are all positive integers.

4. The method for testing the quality of medicines according to claim 1, wherein: The input of the feature fusion network is the four feature maps output by the feature extraction network, which are denoted as: C2, C3, C4, and C5; First, the feature maps C2, C3, C4, and C5 are converted into feature maps D2, D3, D4, and D5 respectively through convolution in the feature fusion network to unify the number of channels. Then, after upsampling the feature map D5, it is added to the feature map D4 to obtain the feature map E4; after upsampling the feature map E4 and the feature map D5, the three are added to the feature map D3 to obtain the feature map E3; after upsampling the feature map E3, it is added to the feature map D2 to obtain the feature map P2; after downsampling the feature map P2, it is added to the feature map E3 to obtain the feature map P3; after downsampling the feature map P3 and the feature map P2, the three are added to the feature map E4 to obtain the feature map P4; after downsampling the feature map P4, it is added to the feature map D5 to obtain the feature map P5, and finally, the feature map P5 is subjected to the maxpooling operation to obtain the feature map P6; Among them, the sizes of the feature maps when superimposed in pairs are consistent and the number of channels matches; the feature fusion network outputs feature maps P2, P3, P4, P5, and P6.

5. The method for testing the quality of medicines according to claim 1, wherein: The process of constructing the data set in step S1 is as follows: Image acquisition: collect target liquid images and process them into a uniform size; Calibration and cropping of visible foreign matter: calibrating the position and category of the visible foreign matter on the target liquid image, and cropping the visible foreign matter image on the target liquid image; Threshold processing: Threshold processing is performed on the cropped visible foreign body image; the RGB channel values ​​of the pixels in the background area are set to 0 or 255, while the RGB channel values ​​of the pixels in the visible foreign body area remain unchanged; Deformation: Randomly rotate and randomly mirror the thresholded visible foreign body image; Data augmentation: Randomly add visible foreign objects after random rotation and random mirroring to the liquid area of ​​the target body fluid image to form a data set.

6. The method for testing the quality of medicines according to claim 1, wherein: In step S3, the stochastic gradient descent method is used to optimize the network when training the neural network model using the data set; Among them, the loss function for: Among them, N pos is the number of positive samples, which are visible foreign objects; is the GIoU loss, is the QFL loss, is the DFL loss, λ0 and λ1 are constant coefficients used to set and f(z) is the indicator function, that is, the value is 1 when there is visible foreign matter in the regression frame, and the value is 0 otherwise.

7. A system based on the pharmaceutical quality testing method according to any one of claims 1 to 6, characterized in that: include: The data set construction module is used to construct the data set, that is, to collect the target liquid image and calibrate the position and category of the visible foreign matter on the target liquid image; A neural network model building module is used to build a neural network model that connects the feature extraction network, the feature fusion network, and the boundary distribution attention detection head network in sequence; The input of the feature extraction network is the target liquid image, the output of the feature extraction network serves as the input of the feature fusion network, the output of the feature fusion network serves as the input of the boundary distribution attention detection head network, and the output of the boundary distribution attention detection head network is the category to which each pixel belongs and the distance from the four boundaries of the regression box with each pixel as the center point to the center point, and the category is the visible foreign matter category or background; A training module, configured to train a neural network model using the data set; The detection module is used to input the target liquid image to be detected into the trained neural network model to obtain the detection result of visible foreign matter.

8. An electronic terminal, characterized in that: include: one or more processors; a memory storing one or more computer programs; The processor calls the computer program to implement: The steps of the method for testing the quality of medicines according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that: A computer program is stored, which is called by a processor to implement: The steps of the method for testing the quality of medicines according to any one of claims 1 to 6.

Images