Machine vision-based online detection method and system for soluble microneedle quality

Abstract

The present application belongs to machine vision, in particular to a soluble microneedle quality online detection method and system based on machine vision, comprising: collecting a digital image of microneedles, extracting a multi-scale pyramid feature map using a backbone network, generating defect candidate region anchor point coordinates and size parameters through a region proposal network, determining a sparse query guide position, for each anchor point, extracting a structure tensor and a statistical texture feature in the corresponding scale feature layer, fusing and encoding to construct a feature descriptor, then combining a basic query embedding vector, obtaining an enhanced sparse query vector through Hadamard product and residual connection, carrying out cross attention operation with the vector as the query and multi-scale features as the key value, generating a modulation mask through a spatial weight network, optimizing the attention score to realize spatial focusing, predicting the scale coefficient to complete multi-scale feature fusion, inputting the fused features into a feedforward network for decoding, and outputting microneedle defect categories and positioning coordinates to realize accurate online quality defect detection of soluble microneedles.

Description

Technical Field

[0001] This application belongs to the field of machine vision, and in particular relates to a machine vision-based online detection method and system for the quality of soluble microneedles. Background Technology

[0002] As a novel transdermal drug delivery system, soluble microneedles face significant challenges in quality control during production, especially with their large-scale industrialization. The morphology, integrity, and surface defects of the microneedles affect the accuracy of dosage and the reliability of skin penetration. Due to the extremely small size and dense arrangement of the microneedle array, and the fact that the material is typically semi-transparent, traditional quality inspection methods largely rely on manual sampling or simple optical inspection. Manual sampling is not only inefficient and subjective but also prone to visual fatigue, leading to extremely high rates of missed and false detections, making it unsuitable for the real-time monitoring requirements of modern production lines. Conventional machine vision methods, on the other hand, are unable to extract distinguishable features due to the reflective interference of semi-transparent materials and the obstruction caused by the dense microstructures, resulting in frequent missed and false alarms, thus hindering the high-quality, large-scale production of soluble microneedles.

[0003] This paper utilizes a backbone network to extract multi-scale features and generates candidate boxes through a globally dense attention mechanism or simple region proposal to identify defects, thus addressing complex backgrounds and multi-scale targets. Traditional cross-attention mechanisms perform dense query computations on the global feature map, leading to computational redundancy and memory overhead, which cannot meet the real-time requirements of pipelined online detection. Existing sparse query mechanisms often rely only on location coordinates to generate basic embeddings, resulting in insufficient query vector content and easy target loss in complex backgrounds with semi-transparent and low-contrast microneedles. When aggregating multi-scale features, the model lacks the ability to focus on different scales and spatial locations, making it unable to suppress background noise interference when dealing with minute defects with large surface size spans and highly irregular distributions of microneedle arrays. This results in insufficient defect localization accuracy and a high misclassification rate in quality assessment. Summary of the Invention

[0004] To improve the accuracy of detection, in the first aspect, a machine vision-based online detection method for the quality of soluble microneedles is proposed, including the following steps:

[0005] A digital image of the soluble microneedles to be detected is acquired, and a multi-scale pyramid feature map is extracted using a backbone network. The multi-scale pyramid feature map is analyzed by a region proposal network to generate the anchor point coordinates and reference width and height of the initial defect candidate region, which serve as a guide for sparse query locations. For each anchor point coordinate, the structural tensor and statistical texture features of the local neighborhood are extracted at the corresponding scale level of the multi-scale pyramid feature map and fused and encoded into a feature descriptor; the feature descriptor is fused with the learnable basic query embedding vector through Hadamard product and residual connection to generate a content-enhanced sparse query vector; the content-enhanced sparse query vector is used as the query and the multi-scale pyramid feature map is used as the key and value to perform cross-attention calculation; The query generates a spatial modulation mask through a spatial weight generation network, and performs element-wise multiplication modulation on the attention scores of the query and the key to achieve spatial focusing. The query is also used to predict the scale coefficient vector. When weighting and aggregating the values, the aggregation results of feature maps at different scales are weighted twice to achieve scale fusion. The feature vectors output by the cross-attention calculation are fed into the feedforward network, decoded to obtain the defect category and coordinates, and complete the quality detection.

[0006] Optionally, the step of acquiring a digital image of the soluble microneedles to be detected and extracting a multi-scale pyramid feature map using a backbone network includes: The digital image is input into a ResNet-50 convolutional neural network containing five concatenated stages; Three corresponding to the original were extracted sequentially through the third, fourth, and fifth stages. Figure 1 Single-layer feature maps at resolution scales of 8, 1 / 16 and 1 / 32; The extracted single-layer feature map is input into a top-down feature pyramid network with lateral additive connections for feature dimension alignment and weighted fusion to generate a multi-scale pyramid feature map with three consecutive scales, each with a uniform number of 256 feature channels.

[0007] Optionally, the step of analyzing the multi-scale pyramid feature map through a region proposal network to generate anchor point coordinates and baseline width and height of initial defect candidate regions as guidance for sparse query locations includes: A fully convolutional neural network with three stacked 3x3 convolutional layers was used to construct classification and coordinate regression branches, respectively, and to traverse each spatial location point in the multi-scale pyramid feature map at each scale. The classification branch is used to predict the probability value of the foreground having surface defect features at each pixel using the Sigmoid function; The coordinate regression branch is used to predict the x and y offsets of the reference center anchor point corresponding to each pixel. All pixels are sorted in descending order according to the predicted foreground probability values. The bias values ​​of the top 300 pixels are extracted, and the bias values ​​are multiplied by the downsampling step size of their respective scales and accumulated to their respective reference center coordinates. Combined with the preset width and height dimensions of the corresponding scale, the coordinates of 300 two-dimensional anchor points and the reference width and height of the initial defect candidate region are formed.

[0008] Optionally, for each anchor point coordinate, extracting the structural tensor and statistical texture features of the local neighborhood at the corresponding scale level of the multi-scale pyramid feature map and fusing and encoding them into a feature descriptor includes: The center of each extracted two-dimensional anchor point coordinate is obtained by dividing it by the downsampling step size of the scale level and mapping it back to the feature map coordinate system. A fixed 7x7 size spatial neighborhood matrix is ​​then cropped from the multi-scale pyramid feature map of the corresponding scale. Calculate the first-order spatial gradient derivative matrix of the spatial neighborhood matrix along the horizontal axis and vertical axis, calculate the independent square matrix and dot product matrix of the two, and perform spatial summation and aggregation on the three sets of matrices in the spatial neighborhood. Then, concatenate the three sets of feature matrices obtained after aggregation and reshape them into a structure tensor vector representing the local boundary gradient. The average value, statistical variance, and distribution skewness statistical vector of all 256-dimensional feature response pixels within the spatial neighborhood matrix are obtained along the spatial dimension to represent the image coarseness and homogeneity attributes. These are used as statistical texture features and concatenated to the tail of the corresponding structural tensor vector to form a unified feature vector. The unified feature vector is transformed to 256 dimensions using a global linear mapping matrix, and used as a feature descriptor.

[0009] Optionally, the step of fusing the feature descriptor with the learnable base query embedding vector through a Hadamard product and residual connection to generate a content-enhanced sparse query vector includes: Extract the initialized 256-dimensional random learnable basic query embedding vector from the model weight matrix; Perform element-wise multiplication between the feature descriptor and the basic query embedding vector to obtain a product multiplication result vector; The product-multiplication result vector is added to the basic query embedding vector to establish a residual channel structure. The summation output is then fed into a layer normalization network for normalization processing to generate a content-enhanced sparse query vector.

[0010] Optionally, the query generates a spatial modulation mask using spatial weights to predict the spatial modulation mask, including: The content-enhanced sparse query vector with a size of 256 dimensions is fed separately into a gating mechanism network consisting of two multilayer perceptrons. The first fully connected linear layer after the input layer reduces the dimensionality of the input layer to a 128-dimensional hidden intermediate state vector and performs activation operations using modified linear units; The second fully connected linear layer restores and expands the number of neurons to 256 dimensions, generates a projection query vector, and performs an inner product calculation on the projection query vector and the feature vector of each spatial position in the multi-scale feature sequence formed by flattening and splicing the multi-scale pyramid feature maps of all scales along the spatial dimension while retaining the feature channel dimension, to generate a one-dimensional spatial weight sequence. Each component of the one-dimensional spatial weight sequence is input into the Sigmoid activation function, and the values ​​are smoothly mapped to the range of 0 to 1 to generate the spatial modulation mask composed of spatial modulation multiplicative factors.

[0011] Optionally, the step of using the content-enhanced sparse query vector as the query and the multi-scale pyramid feature map as the key and value, and performing cross-attention calculation, includes: The inner product of the content-enhanced sparse query vector and the multi-scale pyramid feature map of all input scales, flattened and stitched along the spatial dimension as the key in the multi-scale feature sequence, is calculated to obtain the global attention score matrix. The global attention score matrix is ​​scaled by division according to the built-in scale parameter and normalized using the Softmax function to obtain the initial attention probability feature tensor. Match the spatial modulation mask of the corresponding size to the initial attention probability feature tensor, perform element-wise multiplication calculation, realize hard product attenuation modulation, and obtain the attention probability feature tensor with spatial weight redistribution; The attention probability feature tensor with the spatial weight redistribution is multiplied by the multi-scale feature sequence as the value according to scale segmentation to obtain the weighted aggregation result of each independent scale. The weighted aggregation result of the feature maps of different scales is then weighted and summed twice using the predicted scale coefficient vector to achieve scale fusion and obtain the feature vector output by cross-attention calculation.

[0012] Optionally, the step of feeding the feature vector output by the cross-attention calculation into the feedforward network, decoding it to obtain the defect category and coordinates, and completing the quality detection includes: The 256-dimensional latent state vector output by the cross-attention calculation is passed into the terminal parallel defect classification branch and bounding box regression branch. The defect classification branch is used to calculate the multi-class discrete probability of the current latent vector through a fully connected layer and a Softmax function, and the class corresponding to the highest probability is taken as the defect class of the soluble microneedle; The center coordinate offset of the bounding box is predicted using the bounding box regression branch and the logarithmic scaling rate of change based on the reference width and height. The reference width and height are then decoded and converted into bounding box coordinates in an absolute rectangular coordinate system, and the coordinates of the defect are output.

[0013] On the other hand, a machine vision-based online quality detection system for soluble microneedles is proposed, comprising: The analysis module is used to acquire digital images of the soluble microneedles to be detected, extract multi-scale pyramid feature maps using a backbone network, and analyze the multi-scale pyramid feature maps through a region proposal network to generate anchor point coordinates and reference width and height of the initial defect candidate regions as sparse query location guides. The attention calculation module is used to extract the structural tensor and statistical texture features of the local neighborhood at the corresponding scale level of the multi-scale pyramid feature map for each anchor point coordinate and fuse and encode them into a feature descriptor; the feature descriptor is fused with the learnable basic query embedding vector through Hadamard product and residual connection to generate a content-enhanced sparse query vector; the content-enhanced sparse query vector is used as the query and the multi-scale pyramid feature map is used as the key and value to perform cross-attention calculation; The detection module is used to predict a spatial modulation mask through a spatial weight generation network for the query, and to perform element-wise multiplication modulation on the attention scores of the query and the key to achieve spatial focusing; the query is also used to predict a scale coefficient vector, and when the values ​​are weighted and aggregated, the aggregation results of feature maps at different scales are weighted twice to achieve scale fusion; the feature vector output by the cross-attention calculation is sent to the feedforward network, decoded to obtain the defect category and coordinates, and completes the quality detection.

[0014] Preferably, the step of acquiring the digital image of the soluble microneedles to be detected and extracting multi-scale pyramid feature maps using a backbone network includes: The digital image is input into a ResNet-50 convolutional neural network containing five concatenated stages; Three corresponding to the original were extracted sequentially through the third, fourth, and fifth stages. Figure 1 Single-layer feature maps at resolution scales of 8, 1 / 16 and 1 / 32; The extracted single-layer feature map is input into a top-down feature pyramid network with lateral additive connections for feature dimension alignment and weighted fusion to generate a multi-scale pyramid feature map with three consecutive scales, each with a uniform number of 256 feature channels.

[0015] Preferably, the step of analyzing the multi-scale pyramid feature map through a region proposal network to generate anchor point coordinates and baseline width and height of initial defect candidate regions as guidance for sparse query locations includes: A fully convolutional neural network with three stacked 3x3 convolutional layers was used to construct classification and coordinate regression branches, respectively, and to traverse each spatial location point in the multi-scale pyramid feature map at each scale. The classification branch is used to predict the probability value of the foreground having surface defect features at each pixel using the Sigmoid function; The coordinate regression branch is used to predict the x and y offsets of the reference center anchor point corresponding to each pixel. All pixels are sorted in descending order according to the predicted foreground probability values. The bias values ​​of the top 300 pixels are extracted, and the bias values ​​are multiplied by the downsampling step size of their respective scales and accumulated to their respective reference center coordinates. Combined with the preset width and height dimensions of the corresponding scale, the coordinates of 300 two-dimensional anchor points and the reference width and height of the initial defect candidate region are formed.

[0016] Preferably, for each anchor point coordinate, extracting the structural tensor and statistical texture features of the local neighborhood at the corresponding scale level of the multi-scale pyramid feature map and fusing and encoding them into a feature descriptor includes: The center of each extracted two-dimensional anchor point coordinate is obtained by dividing it by the downsampling step size of the scale level and mapping it back to the feature map coordinate system. A fixed 7x7 size spatial neighborhood matrix is ​​then cropped from the multi-scale pyramid feature map of the corresponding scale. Calculate the first-order spatial gradient derivative matrix of the spatial neighborhood matrix along the horizontal axis and vertical axis, calculate the independent square matrix and dot product matrix of the two, and perform spatial summation and aggregation on the three sets of matrices in the spatial neighborhood. Then, concatenate the three sets of feature matrices obtained after aggregation and reshape them into a structure tensor vector representing the local boundary gradient. The average value, statistical variance, and distribution skewness statistical vector of all 256-dimensional feature response pixels within the spatial neighborhood matrix are obtained along the spatial dimension to represent the image coarseness and homogeneity attributes. These are used as statistical texture features and concatenated to the tail of the corresponding structural tensor vector to form a unified feature vector. The unified feature vector is transformed to 256 dimensions using a global linear mapping matrix, and used as a feature descriptor.

[0017] Preferably, the step of fusing the feature descriptor with the learnable base query embedding vector through a Hadamard product and residual connection to generate a content-enhanced sparse query vector includes: Extract the initialized 256-dimensional random learnable basic query embedding vector from the model weight matrix; Perform element-wise multiplication between the feature descriptor and the basic query embedding vector to obtain a product multiplication result vector; The product-multiplication result vector is added to the basic query embedding vector to establish a residual channel structure. The summation output is then fed into a layer normalization network for normalization processing to generate a content-enhanced sparse query vector.

[0018] Preferably, the query generates a spatial modulation mask for network prediction using spatial weights, including: The content-enhanced sparse query vector with a size of 256 dimensions is fed separately into a gating mechanism network consisting of two multilayer perceptrons. The first fully connected linear layer after the input layer reduces the dimensionality of the input layer to a 128-dimensional hidden intermediate state vector and performs activation operations using modified linear units; The second fully connected linear layer restores and expands the number of neurons to 256 dimensions, generates a projection query vector, and performs an inner product calculation on the projection query vector and the feature vector of each spatial position in the multi-scale feature sequence formed by flattening and splicing the multi-scale pyramid feature maps of all scales along the spatial dimension while retaining the feature channel dimension, to generate a one-dimensional spatial weight sequence. Each component of the one-dimensional spatial weight sequence is input into the Sigmoid activation function, and the values ​​are smoothly mapped to the range of 0 to 1 to generate the spatial modulation mask composed of spatial modulation multiplicative factors.

[0019] Preferably, the step of using the content-enhanced sparse query vector as the query and the multi-scale pyramid feature map as the key and value, and performing cross-attention calculation, includes: The inner product of the content-enhanced sparse query vector and the multi-scale pyramid feature map of all input scales, flattened and stitched along the spatial dimension as the key in the multi-scale feature sequence, is calculated to obtain the global attention score matrix. The global attention score matrix is ​​scaled by division according to the built-in scale parameter and normalized using the Softmax function to obtain the initial attention probability feature tensor. Match the spatial modulation mask of the corresponding size to the initial attention probability feature tensor, perform element-wise multiplication calculation, realize hard product attenuation modulation, and obtain the attention probability feature tensor with spatial weight redistribution; The attention probability feature tensor with the spatial weight redistribution is multiplied by the multi-scale feature sequence as the value according to scale segmentation to obtain the weighted aggregation result of each independent scale. The weighted aggregation result of the feature maps of different scales is then weighted and summed twice using the predicted scale coefficient vector to achieve scale fusion and obtain the feature vector output by cross-attention calculation.

[0020] Preferably, the step of feeding the feature vector output by the cross-attention calculation into the feedforward network, decoding it to obtain the defect category and coordinates, and completing the quality detection includes: The 256-dimensional latent state vector output by the cross-attention calculation is passed into the terminal parallel defect classification branch and bounding box regression branch. The defect classification branch is used to calculate the multi-class discrete probability of the current latent vector through a fully connected layer and a Softmax function, and the class corresponding to the highest probability is taken as the defect class of the soluble microneedle; The center coordinate offset of the bounding box is predicted using the bounding box regression branch and the logarithmic scaling rate of change based on the reference width and height. The reference width and height are then decoded and converted into bounding box coordinates in an absolute rectangular coordinate system, and the coordinates of the defect are output.

[0021] This invention uses a region proposal network to generate anchor coordinates for initial defect candidate regions as sparse query locations, reducing model computational redundancy and quickly identifying potential defects. For the local microscopic characteristics of microneedles, local structural tensors and statistical texture features are extracted and fused into feature descriptors, which are then combined with the basic query vector to generate content-enhanced sparse query vectors, improving the model's ability to express subtle defects. In the cross-attention calculation stage, attention scores are modulated using a predicted spatial modulation mask, achieving spatial focusing on key features and filtering out complex background interference. Secondary weighted aggregation of multi-scale feature maps is performed using predicted scale coefficient vectors, achieving deep fusion of multi-scale features and improving the model's detection stability for defects of various sizes. Outputting defect categories and coordinates improves the accuracy and processing efficiency of online detection of soluble microneedles. Attached Figure Description

[0022] Figure 1 This is a flowchart of a machine vision-based online quality detection method for soluble microneedles. Figure 2 This is a schematic diagram of the feature dimension alignment matrix; Figure 3 This is a schematic diagram illustrating the probability screening of candidate regions. Figure 4 This is a diagram illustrating the changes in the aggregation dimensions of descriptors. Detailed Implementation

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0024] It should be understood that the terms “comprising” and “having”, and any variations thereof, in the embodiments of this specification are intended to cover but not exclude inclusion. For example, a product or device that includes a series of components is not necessarily limited to those components that are explicitly listed, but may include other components that are not explicitly listed or that are inherent to such product or device.

[0025] Example 1 A machine vision-based online quality detection method for soluble microneedles is proposed, including the following steps: S1. Obtain a digital image of the soluble microneedle to be detected, and extract a multi-scale pyramid feature map using a backbone network. Analyze the multi-scale pyramid feature map using a region proposal network to generate the anchor point coordinates and reference width and height of the initial defect candidate region, which serve as a guide for sparse query locations.

[0026] High-resolution color digital images of soluble microneedle arrays are acquired in real time. The acquired digital images are converted into tensor format using the imread function in the open-source computer vision library OpenCV and input into the ResNet50 backbone model of the residual network deployed in the deep learning framework PyTorch. Two-dimensional convolution operations and linear rectified activation operations are performed layer by layer to extract four hierarchical feature maps with different downsampling rates. The above four hierarchical feature maps are fed into the Feature Pyramid Network (FPN) for top-down feature fusion and lateral connection operations to generate multi-scale pyramid feature maps with semantic information.

[0027] The multi-scale pyramid feature map is input into the Region Proposal Network (RPN). Multiple anchor boxes with different aspect ratios and areas are pre-defined at the pixel grid positions of each multi-scale pyramid feature map. The AnchorGenerator calculates the coordinates of all grid center points. A 3×3 convolutional layer, combined with two parallel 1×1 convolutional layers, predicts the foreground / background binary classification probability score, center point coordinate offset, and width / height logarithmic offset for each anchor box. The Non-Maximum Suppression (NMS) algorithm filters out redundant anchor boxes with overlap exceeding a set threshold, retaining the top 2,000 anchor boxes by classification score as initial defect candidate regions. The horizontal and vertical coordinates of the center points of these retained anchor boxes are extracted as anchor point coordinates, and the corresponding initial width and height are extracted as baseline width and height. These are combined and concatenated into a two-dimensional vector, which serves as the guide for sparse query positions in subsequent modules.

[0028] In an optional embodiment, acquiring the digital image of the soluble microneedles to be detected and extracting multi-scale pyramid feature maps using a backbone network includes: The digital image is input into a ResNet-50 convolutional neural network containing five concatenated stages; Three corresponding to the original were extracted sequentially through the third, fourth, and fifth stages. Figure 1 Single-layer feature maps at resolution scales of 8, 1 / 16 and 1 / 32; The extracted single-layer feature map is input into a top-down feature pyramid network with lateral additive connections for feature dimension alignment and weighted fusion to generate a multi-scale pyramid feature map with three consecutive scales, each with a uniform number of 256 feature channels.

[0029] In this embodiment, the digital image of the soluble microneedles to be detected is preferably an RGB image with a resolution of 1024×11024 pixels and 3 channels. After preprocessing, the image is input into a pre-trained ResNet50 convolutional neural network. The ResNet50 convolutional neural network consists of five cascaded residual stages. The input is the digital image of the soluble microneedles to be detected, and the output is feature maps at different resolution scales. By sequentially extracting the outputs of the third, fourth, and fifth stages, feature maps with resolutions of the original image are obtained. Figure 1 Feature maps of 1 / 8, 1 / 16, and 1 / 32. Based on an input size of 1024×11024, the spatial dimensions of the three single-layer feature maps are 12×128, 64×64, and 32×132, respectively, corresponding to initial channel numbers of 512, 1024, and 2048 dimensions. To fuse contextual information at different scales and unify the dimensions, the three single-layer feature maps are input into a feature pyramid network. The feature pyramid network has a top-down architecture with lateral additive connections. The input is the three extracted single-layer feature maps, and the output is a multi-scale pyramid feature map with three consecutive scales, each with a unified feature channel number of 256.

[0030] The feature maps are channel-wise reduced using 1×1 convolutional kernels, compressing the number of channels to a uniform 256. A double bilinear interpolation algorithm is then used to upsample the deep, low-resolution feature maps from top to bottom, for example, interpolating and enlarging a 32×32 image to 64×64. This upsampling is then performed element-wise with the laterally reduced, shallow feature maps to achieve weighted feature fusion. Each fused feature map is then passed through a 3×3 convolutional layer with a stride of 1 and padding of 1 to eliminate aliasing from the upsampling, outputting multi-scale pyramid feature maps with sizes of 128×128×256, 64×64×256, and 32×32×256, respectively. Figure 2 As shown, this compares the initial number of channels of the output features in stages 3, 4, and 5 of the ResNet50 backbone network with the compressed number of channels, which is unified to 256 channels after 1×1 convolution dimensionality reduction.

[0031] In an optional embodiment, the step of analyzing the multi-scale pyramid feature map through a region proposal network to generate anchor coordinates and baseline width and height of initial defect candidate regions as guidance for sparse query locations includes: A fully convolutional neural network with three stacked 3x3 convolutional layers was used to construct classification and coordinate regression branches, respectively, and to traverse each spatial location point in the multi-scale pyramid feature map at each scale. The classification branch is used to predict the probability value of the foreground having surface defect features at each pixel using the Sigmoid function; The coordinate regression branch is used to predict the x and y offsets of the reference center anchor point corresponding to each pixel. All pixels are sorted in descending order according to the predicted foreground probability values. The bias values ​​of the top 300 pixels are extracted, and the bias values ​​are multiplied by the downsampling step size of their respective scales and accumulated to their respective reference center coordinates. Combined with the preset width and height dimensions of the corresponding scale, the coordinates of 300 two-dimensional anchor points and the reference width and height of the initial defect candidate region are formed.

[0032] For the generated feature maps at three scales (128×128, 64×64, and 32×32), a fully convolutional neural network (WCNN) is used as the region proposal network. The WCNN structure consists of three stacked cascaded 3×3 convolutional layers, branched into two parallel 1×1 convolutional layers for classification and coordinate regression branches. The input is the spatial location points in the multi-scale pyramid feature map, and the output is the foreground probability value and the horizontal and vertical coordinate biases for each pixel. At each scale, the network independently applies a base network containing three consecutive 3×3 convolutional layers with a ReLU activation function to extract local semantics. The stride of each consecutive 3×3 convolutional layer is 1, the padding is 1, and the output has 256 channels. The output is then branched into two parallel 1×1 convolutional layers for classification and coordinate regression branches. In the classification branch, each spatial location, for example, corresponds to 16384 pixels on a 128×128 scale image, outputting a scalar representing the foreground confidence. This scalar is then constrained to the [0,1] interval by a Sigmoid activation function, representing the probability that the region contains surface defects such as broken needles or bubbles. In the coordinate regression branch, two scalars are output for each spatial location, representing the normalized coordinate biases of the anchor point's horizontal and vertical axes, respectively. To generate initial candidate regions, the probability values ​​corresponding to all pixels across all feature scales are collected, totaling 21504 locations. Non-maximum suppression or global descending sorting is applied, and the top 300 locations with the highest foreground confidence are selected. Taking a selected position at a feature scale with a downsampling step size of 16 as an example, if the grid's horizontal coordinate is 20 and its vertical coordinate is 30, the regressed bias is 0.5 horizontally and -0.2 vertically. Multiplying the bias by the step size of 16 results in a horizontal absolute offset of 8 pixels and a vertical absolute offset of -3.2 pixels. The absolute coordinates mapped from the reference center to the original image are 336 on the horizontal axis and 484.8 on the vertical axis. These absolute coordinates are then combined with preset anchor frame dimensions, such as a reference width w=64 and a height h=64, to generate 300 two-dimensional anchor point absolute coordinates and reference width and height combinations for the initial defect candidate region, such as... Figure 3 As shown, the distribution curves of all pixels are arranged in descending order of foreground probability. The dashed line represents the preset screening threshold, and only the high-probability pixels to the left of the threshold are retained as anchor points for the initial defect candidate region.

[0033] S2, for each anchor point coordinate, extract the structural tensor and statistical texture features of the local neighborhood at the corresponding scale level of the multi-scale pyramid feature map and fuse and encode them into a feature descriptor; fuse the feature descriptor with the learnable basic query embedding vector through Hadamard product and residual connection to generate a content-enhanced sparse query vector; use the content-enhanced sparse query vector as the query and the multi-scale pyramid feature map as the key and value, and perform cross-attention calculation.

[0034] The target scale level is calculated based on the coordinates of each anchor point and the logarithmic values ​​of the reference width and height. The Region of Interest Alignment (RoIAlign) algorithm is used to extract a 3x3 local neighborhood feature slice centered at the anchor point coordinates from the feature map of the determined scale level. The Sobel operator is used to calculate the image gradients of this local neighborhood feature slice in the horizontal and vertical directions. A structure tensor is constructed based on the gradient outer product matrix, and the principal and secondary eigenvalues ​​are solved. The mean, variance, and contrast and energy features based on the gray-level co-occurrence matrix (GLCM) are calculated on the local neighborhood feature slice as statistical texture features. The solved structure tensor eigenvalues ​​and statistical texture features are concatenated by channel and then fed into a multilayer perceptron (MLP) network for dimensionality reduction and feature encoding, outputting a fixed-dimensional feature description. In the network initialization phase, a learnable weight matrix randomly initialized by a normal distribution is constructed as the basic query embedding vector. The feature descriptor and the basic query embedding vector are multiplied element-wise using the torch point mul function in the deep learning framework. The result of the multiplication is added to the original basic query embedding vector to achieve residual connection, thereby generating a content-enhanced sparse query vector containing local texture structure and location prior information. A cross-attention mechanism is constructed, using three different fully connected layers to map the content-enhanced sparse query vector to a query tensor Query. The multi-scale pyramid feature map is flattened in the spatial dimension and mapped to a key tensor Key and a value tensor Value. Multi-head cross-attention calculation is performed through piecewise matrix multiplication.

[0035] In an optional embodiment, the step of extracting the structural tensor and statistical texture features of the local neighborhood at the corresponding scale level of the multi-scale pyramid feature map for each anchor point coordinate and fusing and encoding them into a feature descriptor includes: The center of each extracted two-dimensional anchor point coordinate is obtained by dividing it by the downsampling step size of the scale level and mapping it back to the feature map coordinate system. A fixed 7x7 size spatial neighborhood matrix is ​​then cropped from the multi-scale pyramid feature map of the corresponding scale. Calculate the first-order spatial gradient derivative matrix of the spatial neighborhood matrix along the horizontal axis and vertical axis, calculate the independent square matrix and dot product matrix of the two, and perform spatial summation and aggregation on the three sets of matrices in the spatial neighborhood. Then, concatenate the three sets of feature matrices obtained after aggregation and reshape them into a structure tensor vector representing the local boundary gradient. The average value, statistical variance, and distribution skewness statistical vector of all 256-dimensional feature response pixels within the spatial neighborhood matrix are obtained along the spatial dimension to represent the image coarseness and homogeneity attributes. These are used as statistical texture features and concatenated to the tail of the corresponding structural tensor vector to form a unified feature vector. The unified feature vector is transformed to 256 dimensions using a global linear mapping matrix, and used as a feature descriptor.

[0036] Local feature extraction was performed on 300 sparse anchor points. Taking an anchor point located on a 64×64 feature layer as an example, the absolute anchor point coordinates at the original image scale were divided by the downsampling step size of 16 corresponding to that layer to complete the coordinate inverse transformation. A neighborhood feature tensor with a spatial size of 7×7 and 256 channels was extracted centered on this point. The horizontal and vertical gradient matrices were obtained by differentiating along the spatial dimensions using a first-order finite difference operator or the Sobel operator, resulting in a 7×7×256 horizontal gradient matrix and a vertical gradient matrix. The independent squared matrices of the horizontal and vertical gradients, as well as their element-wise dot product, were calculated. Within a 7×7 spatial window, the above three sets of matrices were summed and aggregated along the spatial dimensions to obtain three one-dimensional feature vectors, each with a length of 256. These three sets of features were concatenated to generate a 768-dimensional structure tensor vector to represent the gradient distribution changes and directional responses at the defect edges.

[0037] Within the same 7×7×256 spatial neighborhood, for a sample set consisting of 49 pixels, the average value, statistical variance, and distribution skewness of the feature response pixels for each channel are calculated along the spatial dimension, yielding three statistical feature vectors of length 256 to represent the roughness and material homogeneity of the soluble microneedles surface. These 768-dimensional statistical texture feature vectors are concatenated to the end of the aforementioned 768-dimensional structural tensor vector, forming a unified joint feature vector with a total dimension of 1536. A fully connected layer is constructed as a global linear mapping network. This global linear mapping network is a single-layer fully connected network, with the input being the 1536-dimensional unified joint feature vector and the output being a 256-dimensional feature descriptor. By setting the weight dimension of this global linear mapping network to 1536×256, the above features are transformed and compressed to 256 dimensions to generate feature descriptors, such as... Figure 4 As shown, this illustrates the dimensionality transformation of the feature descriptor from the structural feature stage, the statistical feature stage, to the feature splicing stage, and then to the mapping output stage, which compresses it into 256 dimensions.

[0038] In an optional embodiment, the step of fusing the feature descriptor with the learnable base query embedding vector through a Hadamard product and residual connection to generate a content-enhanced sparse query vector includes: Extract the initialized 256-dimensional random learnable basic query embedding vector from the model weight matrix; Perform element-wise multiplication between the feature descriptor and the basic query embedding vector to obtain a product multiplication result vector; The product-multiplication result vector is added to the basic query embedding vector to establish a residual channel structure. The summation output is then fed into a layer normalization network for normalization processing to generate a content-enhanced sparse query vector.

[0039] In the initialization phase, a learnable tensor matrix is ​​set, containing 300 independent, randomly initialized basic query embedding vectors of dimension 256, and iteratively optimized during the backpropagation of network training. For the 300 feature descriptors of dimension 256 generated from the current batch of images, each feature descriptor is aligned with its corresponding basic query embedding vector. For each aligned query index, the fused feature descriptor is multiplied element-wise with its corresponding basic query embedding vector using the Hadamard product.

[0040] A residual connection structure is established, and the multiplication result vector is added to the original base query embedding vector. The summed output is then fed into a layer normalization network. This layer normalization network is a single-layer network that performs zero-mean and unit-variance standardization calculations. The input is the summed vector, and the output is the normalized vector. This normalization network performs normalization processing to generate a 256-dimensional content-enhanced sparse query vector.

[0041] In an optional embodiment, the step of using the content-enhanced sparse query vector as the query and the multi-scale pyramid feature map as the key and value, and performing cross-attention calculation, includes: The inner product of the content-enhanced sparse query vector and the multi-scale pyramid feature map of all input scales, flattened and stitched along the spatial dimension as the key in the multi-scale feature sequence, is calculated to obtain the global attention score matrix. The global attention score matrix is ​​scaled by division according to the built-in scale parameter and normalized using the Softmax function to obtain the initial attention probability feature tensor. Match the spatial modulation mask of the corresponding size to the initial attention probability feature tensor, perform element-wise multiplication calculation, realize hard product attenuation modulation, and obtain the attention probability feature tensor with spatial weight redistribution; The attention probability feature tensor with the spatial weight redistribution is multiplied by the multi-scale feature sequence as the value according to scale segmentation to obtain the weighted aggregation result of each independent scale. The weighted aggregation result of the feature maps of different scales is then weighted and summed twice using the predicted scale coefficient vector to achieve scale fusion and obtain the feature vector output by cross-attention calculation.

[0042] The network structure employing the cross-attention mechanism includes a dot product scaling module and a mask weighted fusion module. The input consists of a content-enhanced sparse query vector (as the query) and a multi-scale feature sequence (as the key and value). The output is an optimized feature vector after scale fusion. 300 256-dimensional content-enhanced sparse query vectors are used as query input, and a multi-scale feature sequence of length 21504 is used as key and value input. Batch matrix multiplication is performed on the query matrix and the transpose of the key matrix to obtain a global attention score matrix of size 300×21504. This score matrix is ​​then scaled using division based on the square root of the 256-dimensional channel dimension (i.e., a scaling factor of 1 / 16). The Softmax function is applied along the 21504 sequence dimension to generate an initial attention probability feature tensor of size 300×21504.

[0043] Using a spatial modulation mask of length 21504, element-wise multiplication is performed on the initial attention probability feature tensor to achieve hard product attenuation modulation. For example, the probability of the background region is multiplied by a factor of 0.05 to reduce it to 0.0005, resulting in an attention tensor with spatially redistributed weights. This attention tensor is then multiplied by the multi-scale feature sequence (which serves as the value matrix) into three segments of lengths 16384, 4096, and 1024 at the three scales, respectively, yielding three independent weighted aggregation results of 300 elements in 256 dimensions. A scale coefficient vector of length 3, normalized by Softmax, is mapped in parallel from the query vector. The independent aggregation results from the three different feature layers are multiplied by their corresponding scale coefficient values, weighted twice, and then summed bitwise, outputting a fused optimized feature vector of size 300×256.

[0044] S3, the query is used to generate a spatial modulation mask through a spatial weight generation network, and the attention scores of the query and the key are multiplied element-wise to achieve spatial focusing; the query is also used to predict the scale coefficient vector, and when the values ​​are weighted and aggregated, the aggregation results of feature maps of different scales are weighted twice to achieve scale fusion; the feature vector output by the cross attention calculation is sent to the feedforward network, and the defect category and coordinates are decoded to complete the quality detection.

[0045] The query tensor is input into a spatial weight generation network containing two fully connected layers and a linear rectified activation function. A one-dimensional spatial weight vector with values ​​ranging from 0 to 1 is output through the σ activation function Sigmoid. This spatial weight vector is then reshaped and transformed into a two-dimensional matrix format spatial modulation mask according to the spatial resolution of the multi-scale pyramid feature map. After calculating the inner product of the query tensor and the transpose of the key tensor to obtain the original attention score matrix, an initial attention probability matrix is ​​first generated through the normalized exponential function Softmax. Then, the spatial modulation mask is superimposed on the initial attention probability matrix using element-wise multiplication to perform weight penalty and enhancement, and the modulated attention weight matrix is ​​output to achieve spatial focusing.

[0046] The query tensor is input into another independent fully connected layer network and the softmax function is used to output a one-dimensional scale coefficient vector with a dimension equal to the number of multi-scale pyramid feature layers. The modulation attention weight matrix and the value tensor are multiplied by matrix multiplication according to the corresponding scale to obtain independent preliminary weighted aggregate features under different scale feature maps. Then, the tensor scalar multiplication in the framework is used to multiply the preliminary weighted aggregate features of each independent scale with the elements of the scale coefficient vector of the corresponding scale to achieve secondary weighting. The results of multiplying each scale are summed to achieve scale fusion.

[0047] The cross-attention output tensor, after spatial modulation and scale fusion, is input to a feedforward neural network (FFN) containing layer normalization (LayerNorm), Gaussian error linear unit activation (GELU), and two linear transformation layers to extract deep nonlinear feature vectors. These feature vectors are then input in parallel into a classification detection head and a regression prediction head. The classification detection head uses a linear classifier with cross-entropy loss to calculate and output the highest probability values ​​for categories such as microneedle bubbles, broken needles, bends, or normal to determine the defect category. The regression prediction head uses a multilayer perceptron with smoothed L1 loss to predict the absolute coordinate offset of the candidate region relative to the center point of the anchor point and the scaling ratio of the bounding box. The true pixel coordinates of the defect region in the original digital image are calculated through inverse coordinate transformation, thus completing the quality detection.

[0048] In an optional embodiment, the query generates a network-predicted spatial modulation mask using spatial weights, including: The content-enhanced sparse query vector with a size of 256 dimensions is fed separately into a gating mechanism network consisting of two multilayer perceptrons. The first fully connected linear layer after the input layer reduces the dimensionality of the input layer to a 128-dimensional hidden intermediate state vector and performs activation operations using modified linear units; The second fully connected linear layer restores and expands the number of neurons to 256 dimensions, generates a projection query vector, and performs an inner product calculation on the projection query vector and the feature vector of each spatial position in the multi-scale feature sequence formed by flattening and splicing the multi-scale pyramid feature maps of all scales along the spatial dimension while retaining the feature channel dimension, to generate a one-dimensional spatial weight sequence. Each component of the one-dimensional spatial weight sequence is input into the Sigmoid activation function, and the values ​​are smoothly mapped to the range of 0 to 1 to generate the spatial modulation mask composed of spatial modulation multiplicative factors.

[0049] The aforementioned 300 content-enhanced sparse query vectors, each with a size of 256, are fed one by one into a spatial weight generation network. This spatial weight generation network is a gated network consisting of two multi-layer perceptrons. The input is the 256-dimensional content-enhanced sparse query vector, and the output is the projected query vector. The first fully connected linear layer of the network reduces the dimensionality of the 256-dimensional input to 128 dimensions by half, generating a hidden intermediate state vector, which is then activated by a ReLU-corrected linear unit. The second fully connected linear layer restores and expands the feature space to 256 dimensions, generating the projected query vector. Multi-scale pyramid feature maps with sizes of 128×128, 64×64, and 32×32 are flattened according to their spatial location.

[0050] The feature points are flattened at three scales, yielding 16384, 4096, and 1024 points respectively. These are then concatenated along the positional dimension to form a multi-scale feature sequence with a total length of 21504, where each vector retains 256 channels. The aforementioned 256-dimensional projection query vector and the 21504-dimensional multi-scale feature vectors are then used to perform vector dot product inner product calculations, resulting in a one-dimensional spatial weight sequence of length 21504. The entire sequence is processed by a Sigmoid activation function, mapping the inner product response to a continuous range of [0,1]. For example, the defect center region factor is mapped to 0.98, and the background region to 0.05, forming a spatial modulation mask vector used to control subsequent cross-attention calculations.

[0051] In an optional embodiment, the step of feeding the feature vector output by the cross-attention calculation into the feedforward network, decoding it to obtain the defect category and coordinates, and completing the quality detection includes: The 256-dimensional latent state vector output by the cross-attention calculation is passed into the terminal parallel defect classification branch and bounding box regression branch. The defect classification branch is used to calculate the multi-class discrete probability of the current latent vector through a fully connected layer and a Softmax function, and the class corresponding to the highest probability is taken as the defect class of the soluble microneedle; The center coordinate offset of the bounding box is predicted using the bounding box regression branch and the logarithmic scaling rate of change based on the reference width and height. The reference width and height are then decoded and converted into bounding box coordinates in an absolute rectangular coordinate system, and the coordinates of the defect are output.

[0052] The 300 latent state vectors, each with a dimension of 256, output from the cross-attention module are input into a multilayer feedforward network. The multilayer feedforward network is a feedforward neural network containing a parallel multi-task prediction head, with a defect classification branch containing a single linear layer on one side and a bounding box regression branch with a three-layer multilayer perceptron structure on the other side.

[0053] The input is a 256-dimensional latent state vector, and the output is a normalized classification probability and four regression parameter scalars for the bounding box. In the defect classification branch, assuming the soluble microneedle quality detection task includes four categories: normal, broken needle, air bubble, and bent, a feedforward network with a single linear layer maps the 256-dimensional feature vector into a discrete probability vector with four output nodes. The Softmax function is applied to generate the normalized classification probability, and the defect category of the target is determined according to the maximum probability criterion. In the parallel bounding box regression branch, a feedforward network with a three-layer multilayer perceptron structure maps the 256-dimensional latent vector into a regression output with four scalar parameters. The first two scalars represent the normalized translation offset of the detection box relative to the center coordinates of the reference anchor point, and the latter two scalars represent the scaling factor of the logarithmic scale of the reference width and height of that layer. Taking the decoding process as an example, if the absolute center coordinates of the reference anchor point are 336 horizontally and 484 vertically, and the reference width and height are 64.

[0054] When the predicted center offset output is 0.1 horizontally and -0.2 vertically, and the scaling logarithm coefficient is 0.2 in the width direction and -0.1 in the height direction, the calculated true center x-coordinate is 342.4, the true center y-coordinate is 471.2, and the true predicted width is 64 multiplied by e. 0.2 It is approximately 78.17, and the actual height is 64 times e. -0.1 The value is approximately 57.91. Based on the absolute Cartesian coordinate system parameters obtained after decoding, the coordinates of the defect detection box are output.

[0055] The experimental setup used digital images of soluble microneedle defects with a resolution of 1024×1024 pixels as the dataset, covering categories such as broken needles, bubbles, and bends. The dataset was divided into training, validation, and test sets in an 8:1:1 ratio. A stochastic gradient descent optimizer was used, with an initial learning rate of 0.01, a momentum parameter of 0.9, and a batch size of 8, for a total of 100 training epochs. The benchmark model used a basic residual network 50 combined with a standard top-down feature pyramid and a conventional, unmodulated cross-attention detector. The experimental results are shown in Table 1.

[0056] Table 1

[0057] Local feature descriptors enhance the model's ability to represent the low-level features of microbubbles and weak cracks by aggregating statistical information such as the boundary gradient structure tensor and material homogeneity properties within the neighborhood. A spatial weight generation network, combined with a spatial modulation mask constructed using a cross-attention mechanism, achieves hard product attenuation of the probability of non-defect background regions, reducing interference from complex image noise. Furthermore, the secondary weighting of multi-scale feature sequences by scale coefficient vectors improves the accuracy of localization, regression, and classification of microneedle defect candidate regions at each scale.

[0058] Example 2 A machine vision-based online quality detection system for soluble microneedles is proposed, comprising: The analysis module is used to acquire digital images of the soluble microneedles to be detected, extract multi-scale pyramid feature maps using a backbone network, and analyze the multi-scale pyramid feature maps through a region proposal network to generate anchor point coordinates and reference width and height of the initial defect candidate regions as sparse query location guides. The attention calculation module is used to extract the structural tensor and statistical texture features of the local neighborhood at the corresponding scale level of the multi-scale pyramid feature map for each anchor point coordinate and fuse and encode them into a feature descriptor; the feature descriptor is fused with the learnable basic query embedding vector through Hadamard product and residual connection to generate a content-enhanced sparse query vector; the content-enhanced sparse query vector is used as the query and the multi-scale pyramid feature map is used as the key and value to perform cross-attention calculation; The detection module is used to predict a spatial modulation mask through a spatial weight generation network for the query, and to perform element-wise multiplication modulation on the attention scores of the query and the key to achieve spatial focusing; the query is also used to predict a scale coefficient vector, and when the values ​​are weighted and aggregated, the aggregation results of feature maps at different scales are weighted twice to achieve scale fusion; the feature vector output by the cross-attention calculation is sent to the feedforward network, decoded to obtain the defect category and coordinates, and completes the quality detection.

[0059] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and to aid in understanding a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art, upon reading this specification, may readily identify some of the devices as separate embodiments. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. And the content of each secondary embodiment is valid even if it contains fewer than all the features of a single foregoing disclosed embodiment.

Claims

1. A machine vision-based online method for detecting the quality of soluble microneedles, characterized in that, Including the following steps: A digital image of the soluble microneedles to be detected is acquired, and a multi-scale pyramid feature map is extracted using a backbone network. The multi-scale pyramid feature map is analyzed by a region proposal network to generate the anchor point coordinates and reference width and height of the initial defect candidate region, which serve as a guide for sparse query locations. For each anchor point coordinate, the structural tensor and statistical texture features of the local neighborhood are extracted at the corresponding scale level of the multi-scale pyramid feature map and fused and encoded into a feature descriptor; the feature descriptor is fused with the learnable basic query embedding vector through Hadamard product and residual connection to generate a content-enhanced sparse query vector; the content-enhanced sparse query vector is used as the query and the multi-scale pyramid feature map is used as the key and value to perform cross-attention calculation; The query generates a spatial modulation mask through a spatial weight generation network, and performs element-wise multiplication modulation on the attention scores of the query and the key to achieve spatial focusing. The query is also used to predict the scale coefficient vector. When weighting and aggregating the values, the aggregation results of feature maps at different scales are weighted twice to achieve scale fusion. The feature vectors output by the cross-attention calculation are fed into the feedforward network, decoded to obtain the defect category and coordinates, and complete the quality detection.

2. The method according to claim 1, characterized in that, The process of acquiring a digital image of the soluble microneedles to be detected and extracting a multi-scale pyramid feature map using a backbone network includes: The digital image is input into a ResNet-50 convolutional neural network containing five concatenated stages; Three single-layer feature maps corresponding to the original image resolution scales of 1 / 8, 1 / 16, and 1 / 32 were extracted sequentially through the third, fourth, and fifth stages. The extracted single-layer feature map is input into a top-down feature pyramid network with lateral additive connections for feature dimension alignment and weighted fusion to generate a multi-scale pyramid feature map with three consecutive scales, each with a uniform number of 256 feature channels.

3. The method according to claim 1, characterized in that, The step of analyzing the multi-scale pyramid feature map using a region proposal network to generate anchor point coordinates and baseline width and height of initial defect candidate regions as guidance for sparse query locations includes: A fully convolutional neural network with three stacked 3x3 convolutional layers was used to construct classification and coordinate regression branches, respectively, and to traverse each spatial location point in the multi-scale pyramid feature map at each scale. The classification branch is used to predict the probability value of the foreground having surface defect features at each pixel using the Sigmoid function; The coordinate regression branch is used to predict the x and y offsets of the reference center anchor point corresponding to each pixel. All pixels are sorted in descending order according to the predicted foreground probability values. The bias values ​​of the top 300 pixels are extracted, and the bias values ​​are multiplied by the downsampling step size of their respective scales and accumulated to their respective reference center coordinates. Combined with the preset width and height dimensions of the corresponding scale, the coordinates of 300 two-dimensional anchor points and the reference width and height of the initial defect candidate region are formed.

4. The method according to claim 1, characterized in that, For each anchor point coordinate, the local neighborhood structural tensor and statistical texture features are extracted at the corresponding scale level of the multi-scale pyramid feature map and fused and encoded into a feature descriptor, including: The center of each extracted two-dimensional anchor point coordinate is obtained by dividing it by the downsampling step size of the scale level and mapping it back to the feature map coordinate system. A fixed 7x7 size spatial neighborhood matrix is ​​then cropped from the multi-scale pyramid feature map of the corresponding scale. Calculate the first-order spatial gradient derivative matrix of the spatial neighborhood matrix along the horizontal axis and vertical axis, calculate the independent square matrix and dot product matrix of the two, and perform spatial summation and aggregation on the three sets of matrices in the spatial neighborhood. Then, concatenate the three sets of feature matrices obtained after aggregation and reshape them into a structure tensor vector representing the local boundary gradient. The average value, statistical variance, and distribution skewness statistical vector of all 256-dimensional feature response pixels within the spatial neighborhood matrix are obtained along the spatial dimension to represent the image coarseness and homogeneity attributes. These are used as statistical texture features and concatenated to the tail of the corresponding structural tensor vector to form a unified feature vector. The unified feature vector is transformed to 256 dimensions using a global linear mapping matrix, and used as a feature descriptor.

5. The method according to claim 1, characterized in that, The step of fusing the feature descriptor with the learnable base query embedding vector through a Hadamard product and residual connection to generate a content-enhanced sparse query vector includes: Extract the initialized 256-dimensional random learnable basic query embedding vector from the model weight matrix; Perform element-wise multiplication between the feature descriptor and the basic query embedding vector to obtain a product multiplication result vector; The product-multiplication result vector is added to the basic query embedding vector to establish a residual channel structure. The summation output is then fed into a layer normalization network for normalization processing to generate a content-enhanced sparse query vector.

6. The method according to claim 1, characterized in that, The query generates a network-predicted spatial modulation mask using spatial weights, including: The content-enhanced sparse query vector with a size of 256 dimensions is fed separately into a gating mechanism network consisting of two multilayer perceptrons. The first fully connected linear layer after the input layer reduces the dimensionality of the input layer to a 128-dimensional hidden intermediate state vector and performs activation operations using modified linear units; The second fully connected linear layer restores and expands the number of neurons to 256 dimensions, generates a projection query vector, and performs an inner product calculation on the projection query vector and the feature vector of each spatial position in the multi-scale feature sequence formed by flattening and splicing the multi-scale pyramid feature maps of all scales along the spatial dimension while retaining the feature channel dimension, to generate a one-dimensional spatial weight sequence. Each component of the one-dimensional spatial weight sequence is input into the Sigmoid activation function, and the values ​​are smoothly mapped to the range of 0 to 1 to generate the spatial modulation mask composed of spatial modulation multiplicative factors.

7. The method according to claim 1, characterized in that, The step of using the content-enhanced sparse query vector as the query and the multi-scale pyramid feature map as the key and value, and performing cross-attention calculation, includes: The inner product of the content-enhanced sparse query vector and the multi-scale pyramid feature map of all input scales, flattened and stitched along the spatial dimension as the key in the multi-scale feature sequence, is calculated to obtain the global attention score matrix. The global attention score matrix is ​​scaled by division according to the built-in scale parameter and normalized using the Softmax function to obtain the initial attention probability feature tensor. Match the spatial modulation mask of the corresponding size to the initial attention probability feature tensor, perform element-wise multiplication calculation, realize hard product attenuation modulation, and obtain the attention probability feature tensor with spatial weight redistribution; The attention probability feature tensor with the spatial weight redistribution is multiplied by the multi-scale feature sequence as the value according to scale segmentation to obtain the weighted aggregation result of each independent scale. The weighted aggregation result of the feature maps of different scales is then weighted and summed twice using the predicted scale coefficient vector to achieve scale fusion and obtain the feature vector output by cross-attention calculation.

8. The method according to claim 1, characterized in that, The process of feeding the feature vector output from the cross-attention calculation into the feedforward network, decoding it to obtain the defect category and coordinates, and completing the quality detection includes: The 256-dimensional latent state vector output by the cross-attention calculation is passed into the terminal parallel defect classification branch and bounding box regression branch. The defect classification branch is used to calculate the multi-class discrete probability of the current latent vector through a fully connected layer and a Softmax function, and the class corresponding to the highest probability is taken as the defect class of the soluble microneedle; The center coordinate offset of the bounding box is predicted using the bounding box regression branch and the logarithmic scaling rate of change based on the reference width and height. The reference width and height are then decoded and converted into bounding box coordinates in an absolute rectangular coordinate system, and the coordinates of the defect are output.

9. A machine vision-based online quality detection system for soluble microneedles, characterized in that, include: The analysis module is used to acquire digital images of the soluble microneedles to be detected and to extract multi-scale pyramid feature maps using the backbone network. The multi-scale pyramid feature map is analyzed by a region proposal network to generate the anchor coordinates and baseline width and height of the initial defect candidate region, which serve as a guide for sparse query locations. The attention calculation module is used to extract the structural tensor and statistical texture features of the local neighborhood at the corresponding scale level of the multi-scale pyramid feature map for each anchor point coordinate and fuse and encode them into a feature descriptor; the feature descriptor is fused with the learnable basic query embedding vector through Hadamard product and residual connection to generate a content-enhanced sparse query vector; the content-enhanced sparse query vector is used as the query and the multi-scale pyramid feature map is used as the key and value to perform cross-attention calculation; The detection module is used to generate a spatial modulation mask for the query through a spatial weight generation network, and to perform element-wise multiplicative modulation on the attention scores of the query and the key to achieve spatial focusing; The query is also used to predict the scale coefficient vector. When the values ​​are weighted and aggregated, the aggregation results of feature maps at different scales are weighted twice to achieve scale fusion. The feature vectors output by cross-attention calculation are sent to the feedforward network to decode the defect category and coordinates, thus completing the quality detection.

10. The system according to claim 9, characterized in that, The process of acquiring a digital image of the soluble microneedles to be detected and extracting a multi-scale pyramid feature map using a backbone network includes: The digital image is input into a ResNet-50 convolutional neural network containing five concatenated stages; Three single-layer feature maps corresponding to the original image resolution scales of 1 / 8, 1 / 16, and 1 / 32 were extracted sequentially through the third, fourth, and fifth stages. The extracted single-layer feature map is input into a top-down feature pyramid network with lateral additive connections for feature dimension alignment and weighted fusion to generate a multi-scale pyramid feature map with three consecutive scales, each with a uniform number of 256 feature channels.

Images