Tortoise-shell glue quality detection method and system based on image recognition

Abstract

The invention discloses a tortoise-shell glue quality detection method and system based on image recognition. The method comprises the following steps: firstly, carrying out standardized transmission imaging and preprocessing on a tortoise-shell glue sample to obtain a standardized image; carrying out image segmentation and size standardization processing, and extracting a target area; extracting three depth feature vectors of light transmission uniformity, color purity and internal microscopic flaws by adopting a multi-branch parallel feature sensing network; and finally, through iterative reasoning of the graph neural network and an attention mechanism, self-adaptively fusing a mutual relationship among different features, and generating comprehensive feature representation for quality detection. According to the invention, the automatic, standardized and precise detection of the key quality indexes of the tortoise-shell glue is realized, and an objective and reliable technical means is provided for the quality evaluation of the tortoise-shell glue.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of quality detection of tortoise shell glue, and particularly relates to a quality detection method and system of tortoise shell glue based on image recognition. BACKGROUND

[0002] As a precious traditional Chinese medicinal material, tortoise shell glue has important effects of nourishing yin and suppressing yang, tonifying kidney and strengthening bone, and nourishing blood and stopping bleeding, and plays an important role in clinical application of traditional Chinese medicine. With the rapid development of traditional Chinese medicine industry and the continuous growth of market demand, the quality detection of tortoise shell glue has become a key link to guarantee the quality of medicinal materials and maintain the rights and interests of consumers. The quality of tortoise shell glue directly affects its medicinal value and clinical efficacy, so it is of great significance to establish a scientific, accurate and efficient quality detection method for the healthy development of traditional Chinese medicine industry.

[0003] Traditional quality detection of tortoise shell glue mainly relies on experienced traditional Chinese medicine practitioners to make judgments by visual observation. The detection personnel need to make comprehensive evaluation according to the appearance color, transparency, texture hardness and cross-section characteristics of tortoise shell glue. This traditional detection method has many problems and limitations. First of all, it is highly subjective. Different detection personnel will give different quality evaluation results for the same sample due to differences in experience level, cognition and subjective judgment standard, resulting in a lack of consistency and repeatability of the detection results. Secondly, the detection efficiency is low. Manual detection requires a large amount of time and manpower, which is difficult to meet the rapid detection needs of large quantities of samples, restricting the scale and industrial development of detection work. In recent years, some researchers have tried to use spectral analysis and chemical composition detection methods to evaluate the quality of tortoise shell glue. However, these methods usually require complex sample pretreatment processes, have high detection costs, and mainly focus on chemical composition content, making it difficult to fully reflect the appearance quality characteristics of tortoise shell glue. Although the detection method based on image processing improves the objectivity of detection to some extent, most existing methods use simple image feature extraction techniques such as color histogram and texture features, which have limited expression ability and are difficult to accurately capture the complex quality characteristics of tortoise shell glue.

[0004] The existing image recognition method also has the problem of simple feature fusion in the quality detection of tortoise shell glue. It usually uses simple feature splicing or weighted fusion strategy, which cannot effectively model the complex correlation between different quality characteristics, resulting in low detection accuracy. In addition, the existing method lacks specialized network design for specific quality indicators of tortoise shell glue, and cannot fully utilize the advantages of deep learning technology for accurate quality analysis. Therefore, it is urgent to develop a tortoise shell glue quality detection method based on advanced image recognition technology to solve the shortcomings of traditional detection methods and realize the automatic, standardized and intelligent detection of tortoise shell glue quality. SUMMARY

[0005] Therefore, the application provides a quality detection method for tortoise shell glue based on image recognition, aiming to automatically, standardize and accurately detect key quality indicators of tortoise shell glue, such as light transmission uniformity, color purity and internal microscopic flaws, and provide objective and reliable technical means for quality evaluation of tortoise shell glue.

[0006] To achieve the above-mentioned purpose, the application provides a quality detection method for tortoise shell glue based on image recognition, comprising the following steps: A1: obtaining a tortoise shell glue sample to be detected, placing the tortoise shell glue sample in a standardized image acquisition device for transmission imaging and preprocessing to obtain a standardized tortoise shell glue transmission image; A2: performing image segmentation and size standardization processing on the standardized tortoise shell glue transmission image to obtain a standardized target region image; A3: inputting the standardized target region image into a multi-branch parallel feature perception network for feature extraction to obtain three depth feature vectors of light transmission uniformity, color purity and internal microscopic flaws; A4: constructing the three depth feature vectors of light transmission uniformity, color purity and internal microscopic flaws into a feature map, adaptively fusing the mutual relationship between different features through iterative reasoning and attention mechanism of a graph neural network, and performing end-to-end training optimization to generate a comprehensive feature representation for quality detection of tortoise shell glue.

[0007] As a further improved method of the application: Optionally, in the A1 step of obtaining a tortoise shell glue sample to be detected, placing the tortoise shell glue sample in a standardized image acquisition device for transmission imaging and preprocessing to obtain a standardized tortoise shell glue transmission image, comprising: A11: preparing a standardized image acquisition device; the standardized image acquisition device comprises a backlight light source, a sample stage and a high-resolution digital camera; A12: controlling the high-resolution digital camera to perform transmission imaging on the tortoise shell glue sample under constant illumination conditions and fixed shooting parameters to obtain an original transmission image ; A13: preprocessing the original transmission image , including denoising processing, brightness standardization processing and color correction processing, to obtain a standardized tortoise shell glue transmission image ; Wherein, the denoising processing is performed by using a Gaussian filter; when the tortoise shell glue sample has different thickness distribution, the denoising processing uses a thickness-aware adaptive filter, and the filtering parameter is dynamically adjusted according to the local transmission intensity gradient, the filtering parameter is the standard deviation of the Gaussian filter, specifically: ; wherein, is a standard deviation of the thickness-aware adaptive filter at the pixel position ; is a base filter parameter, is a thickness sensitivity coefficient, is a normalized shellac transmission image at the pixel position ; denotes a 2-norm; The brightness normalization process adjusts the average brightness value of the image to a preset target brightness value; the color correction process performs white balance correction based on a standard color reference board.

[0008] Optionally, the A2 step of performing image segmentation and size normalization processing on the normalized shellac transmission image to obtain a normalized target region image, comprising: performing a grayscale conversion process on the normalized shellac transmission image to obtain a grayscale image, the grayscale conversion using a weighted average method; performing an adaptive threshold segmentation process on the grayscale image, calculating an adaptive threshold value based on the local statistical properties of the image, the adaptive threshold value being determined according to the average grayscale value and the grayscale standard deviation of the image; performing a binaryzation process on the grayscale image based on the adaptive threshold value to obtain a binary image, setting pixel points with a grayscale value greater than or equal to the adaptive threshold value to white, and setting pixel points with a grayscale value less than the adaptive threshold value to black; performing a morphological process on the binary image, including an opening operation and a closing operation, to eliminate noise points and fill in holes; the opening operation uses a circular structural element to remove small noise connected regions; the closing operation uses a circular structural element to fill in small holes inside the shellac body; performing a connected domain analysis on the binary image after morphological processing to identify all connected regions and calculate the area of each connected region; selecting the largest connected region as the shellac body region and determining the bounding rectangle boundary box of the region; performing a cropping process on the normalized shellac transmission image based on the bounding rectangle boundary box to obtain a target region image ; performing a size normalization process on the target region image to scale it to a preset standard size to obtain a normalized target region image .

[0009] Optionally, the standardized target region is input into a multi-branch parallel feature perception network for feature extraction in the A3 step, including: The standardized target region image is input into a multi-branch parallel feature perception network, which includes three independent branch structures: a light transmission uniformity feature extraction branch, a color purity feature extraction branch, and an internal microscopic flaw feature extraction branch. The calculation process of the feature map of the first layer of the light transmission uniformity feature extraction branch is as follows: ; wherein, represents the feature map of the first layer of the light transmission uniformity feature extraction branch, , represents the convolution kernel weight matrix of the first layer of the light transmission uniformity feature extraction branch, represents a convolution operation, represents the bias vector of the first layer of the light transmission uniformity feature extraction branch, represents an S-shaped activation function. The calculation process of the feature map of the first layer of the color purity feature extraction branch is as follows: ; wherein, represents the feature map of the first layer of the color purity feature extraction branch, , represents the convolution kernel weight matrix of the first layer of the color purity feature extraction branch, represents the bias vector of the first layer of the color purity feature extraction branch. The calculation process of the feature map of the first layer of the internal microscopic flaw feature extraction branch is as follows: ; wherein, represents the feature map of the first layer of the internal microscopic flaw feature extraction branch, , represents the convolution kernel weight matrix of the first layer of the internal microscopic flaw feature extraction branch, represents the bias vector of the first layer of the internal microscopic flaw feature extraction branch. The feature maps of the last layer of the three branches are respectively reduced in dimension by a global average pooling operation to obtain corresponding deep feature vectors 、 and ; wherein, represents a light transmission uniformity deep feature vector, represents a color purity deep feature vector, represents an internal microscopic flaw deep feature vector.

[0010] The A3 step adopts three independent branch structures to extract features for light transmission uniformity, color purity, and internal microscopic flaws respectively, avoiding the problem of mutual interference of different features in a traditional single network structure, and ensuring that each branch can focus on feature learning of a specific quality dimension.

[0011] The parallel processing architecture of the A3 step can simultaneously extract feature information of multiple dimensions, greatly improving the computing efficiency and shortening the detection time compared to a serial processing mode. Each branch adopts a deep convolutional neural network structure, and through layer-by-layer feature abstraction, more rich and abstract high-level feature representations can be extracted from the original image. These deep features have stronger expression ability and generalization performance than traditional hand-designed features.

[0012] The branches in the A3 step adopt the same activation function and network depth design, ensuring that different feature vectors have the same dimension and numerical range, providing a good foundation for subsequent feature fusion. The final feature map is reduced in dimension by a global average pooling operation, which not only greatly reduces the number of parameters, but also effectively suppresses the overfitting phenomenon and improves the generalization ability of the model.

[0013] Optionally, in the A4 step, the light transmission uniformity, color purity, and internal microscopic flaw deep feature vectors are constructed into a feature graph, the mutual relationship between different features is adaptively fused through iterative reasoning and attention mechanism of the graph neural network, and end-to-end training and optimization are performed to generate a comprehensive feature representation for tortoise shell glue quality detection, including: A41: constructing the light transmission uniformity deep feature vector , the color purity deep feature vector , and the internal microscopic flaw deep feature vector into a feature graph , the feature graph is composed of a node set and an edge set , wherein each deep feature vector is a node in the graph; constructing an adjacency matrix of the feature graph, Adjacency matrix is ​​used to represent the association strength between different feature nodes. The initialization formula is: ; in, Represents a node and nodes Edge weights between them , and Representing nodes respectively and nodes The corresponding feature vector, Describing the L2 norm, Hyperparameters for controlling the distribution of adjacency weights; When the condition number of the adjacency matrix is ​​too large, a regularized stable algorithm is used to iteratively update the adjacency matrix: ; in, For global regularization parameters, It is the identity matrix. Here are the diagonal regularization parameters. Indicates extraction of adjacency matrix The regularization strength of the diagonal matrix, consisting of the main diagonal elements, is adaptively determined by the condition number: ; in, This is a maximum value function used to select the maximum value from the input values; for the adjacency matrix... Perform symmetric normalization to obtain the normalized adjacency matrix. ; A42: Iterative inference and fusion of feature maps using graph neural networks: A421: The first... of graph neural networks Layer unaggregated node feature matrix The calculation method is as follows: ; in, Indicates the first The node feature matrix of the layer Represents the initial node feature matrix. Indicates the first The learnable weight matrix of the layer, Indicates the first Layer bias vector; A422: Introduces a feature relationship attention mechanism to calculate the dynamic relationship weights between different feature nodes. : ; wherein, denotes the layer, node in the layer, is a learnable attention parameter vector, denotes the matrix transpose, denotes the linear transformation matrix of the layer, denotes the feature vector of the node and the node in the un-aggregated node feature matrix of the layer, is a normalization exponential function, is a leaky linear rectified activation function; A423: adaptive aggregation based on dynamic relationship weights to obtain the feature vector of each node in the node feature matrix of the layer, and the adaptive aggregation is specifically: ; wherein, denotes the feature vector of the node in the node feature matrix of the layer, denotes the neighbor node set of the node, has the same value range as but , denotes the self-connection weight matrix of the layer; denotes the attention weight of the node to the node, denotes the feature vector of the node in the un-aggregated node feature matrix of the layer; repeating the processes of A421 to A423 times to obtain the feature evolution sequence , after the iterative inference of the layer graph neural network, performing a global pooling operation on the node feature matrix of the last layer to obtain the comprehensive feature representation : ; wherein, denotes the global pooling operation, respectively represent the mean-pooling, max-pooling and standard deviation-pooling operations, represents the node feature matrix of the last layer, represents the feature concatenation operation; A43: using the tortoise shell sample with quality label to perform end-to-end training on the graph neural network, defining the training loss function as cross-entropy loss, updating the network parameters using the Adam optimizer, obtaining the trained graph neural network, inputting the normalized target region image of the to-be-detected tortoise shell sample into the multi-branch parallel feature perception network and the trained graph neural network, obtaining the comprehensive feature representation, and performing quality classification prediction based on the comprehensive feature representation, and outputting the quality grade and confidence score of the tortoise shell sample.

[0014] The A4 step constructs three deep feature vectors of light transmission uniformity, color purity and internal microscopic flaw into a graph structure, can fully excavate and utilize the internal correlation between different quality features, breaks through the limitation of simply splicing features in traditional methods, and realizes more intelligent feature fusion.

[0015] The A4 step adopts the message passing mechanism of the graph neural network, enables each feature node to perceive the information of other nodes through a multi-layer iterative reasoning process, realizes mutual enhancement and complementation between features, and makes the final comprehensive feature representation better reflect the overall quality condition of the tortoise shell.

[0016] The attention mechanism introduced in the A4 step can dynamically calculate the relationship weight between different feature nodes, so that the network can automatically adjust the feature fusion strategy according to the characteristics of the specific sample, improve the flexibility and generalization ability of the model.

[0017] The application further discloses a tortoise shell quality detection system based on image recognition, comprising: An imaging module: acquires a to-be-detected tortoise shell sample, performs transmission imaging on the tortoise shell sample in a standardized image acquisition device, and performs preprocessing; A segmentation and size standardization module: performs image segmentation and size standardization processing on the standardized tortoise shell transmission image, and obtains a standardized target region image; A deep feature extraction module: inputs the standardized target region into a multi-branch parallel feature perception network for feature extraction; ​The graph neural network module: the three deep feature vectors of light transmission uniformity, color purity and internal microscopic flaw are constructed into feature maps, the mutual relationship between different features is adaptively fused through iterative reasoning and attention mechanism of the graph neural network, and end-to-end training optimization is carried out, so as to generate comprehensive feature representation for tortoise shell glue quality detection.

[0018] Compared with the prior art, the present application has at least the following beneficial effects: The present application realizes the specialized feature extraction of different quality dimensions of tortoise shell glue through the multi-branch parallel feature perception network, significantly improves the accuracy and richness of the feature representation. The traditional method often adopts a single feature extraction strategy, which is difficult to fully capture the complex quality features of tortoise shell glue, while the three independent branches designed in the present application respectively learn the depth of light transmission uniformity, color purity and internal microscopic flaw, avoiding the mutual interference between different features, and ensuring that each branch can focus on the feature learning of a specific quality dimension. The parallel processing architecture not only improves the computing efficiency, but also extracts more abstract and discriminative high-level features through the deep convolutional neural network, laying a solid foundation for subsequent feature fusion and quality judgment.

[0019] The present application adopts the graph neural network to model the complex relationship between different features, realizes intelligent feature relationship reasoning and adaptive fusion. By constructing the three deep feature vectors into a graph structure, the message passing mechanism of the graph neural network is used for multi-layer iterative reasoning, so that each feature node can perceive and utilize the information of other nodes, realizing the mutual enhancement and complementation between features. The introduced attention mechanism can dynamically calculate the relationship weight between different feature nodes, automatically adjust the feature fusion strategy according to the characteristics of specific samples, avoiding the subjectivity of manually designed weight coefficients in traditional methods. This feature fusion method based on graph structure can better mine and utilize the internal correlation between different quality features, and generate more accurate and complete comprehensive feature representation. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A flowchart of a tortoise shell glue quality detection method based on image recognition provided by the present application is shown in the figure; Figure 2 A multi-branch parallel feature perception network structure diagram provided by the present application is shown in the figure; Figure 3 A t-SNE dimensionality reduction visualization diagram of an embodiment of the present application is shown in the figure, (a) is the t-SNE dimensionality reduction visualization result of the features extracted by the light transmission uniformity branch in the multi-branch parallel feature perception network, and (b) is the t-SNE dimensionality reduction visualization result of the features extracted by the ResNet-50 network. DETAILED DESCRIPTION

[0021] The present invention will be further described below with reference to the accompanying drawings, but this is not intended to limit the present invention in any way. Any modifications or substitutions made based on the teachings of the present invention shall fall within the protection scope of the present invention.

[0022] Example 1: A method for quality inspection of tortoise shell glue based on image recognition, such as... Figure 1 As shown, it includes the following steps: A1: Obtain the tortoise shell glue sample to be tested, place the tortoise shell glue sample in a standardized image acquisition device for transmission imaging and preprocessing, and obtain a standardized tortoise shell glue transmission image: A11: Prepare a standardized image acquisition device; the standardized image acquisition device includes a backlight source, a sample stage, and a high-resolution digital camera; A12: Under constant illumination conditions and fixed shooting parameters, control the high-resolution digital camera to perform transmission imaging on the tortoise shell glue sample to obtain the original transmission image. ; A13: Regarding the original transmission image Preprocessing was performed, including noise reduction, brightness normalization, and color correction, to obtain a standardized tortoise shell resin transmission image. ; The denoising process employs a Gaussian filter. In this embodiment, the filter kernel size is 5×5, and the standard deviation is 1.2. When the tortoise shell glue sample has different thickness distributions, a thickness-aware adaptive filter is used for denoising. Its filtering parameters are dynamically adjusted based on the local transmission intensity gradient. These filtering parameters are the standard deviation of the Gaussian filter, specifically: ; in, For thickness-aware adaptive filters at pixel positions The standard deviation at that point; Based on the basic filter parameters, For thickness sensitivity coefficient, Standardized transmission images of tortoise shell glue At pixel position The light intensity gradient vector at that location, Represents the 2-norm; The brightness normalization process adjusts the average brightness value of the image to a preset target brightness value, which is 140 in this embodiment; the color correction process performs white balance correction based on a standard color reference chart.

[0023] A2: Perform image segmentation and size normalization on the standardized tortoise shell glue transmission image to obtain a standardized target region image: The standardized transmission image of the tortoise shell glue A grayscale image is obtained by performing grayscale conversion. The grayscale conversion adopts a weighted average method. In this embodiment, the weights of the red channel, green channel and blue channel are 0.299, 0.587 and 0.114, respectively. The grayscale image is subjected to adaptive threshold segmentation. In this embodiment, the Otsu algorithm is used to calculate the adaptive threshold based on the local statistical characteristics of the image. The adaptive threshold is determined according to the average grayscale value and the grayscale standard deviation of the image. The grayscale image is binarized based on the adaptive threshold to obtain a binary image. Pixels with grayscale values ​​greater than or equal to the adaptive threshold are set to white, and pixels with grayscale values ​​less than the adaptive threshold are set to black. The binary image is subjected to morphological processing, including opening and closing operations, to eliminate noise points and fill holes. The opening operation uses a circular structuring element, and in this embodiment, the radius of the circular structuring element for the opening operation is 3 pixels, which is used to remove small noisy connected regions. The closing operation uses a circular structuring element, and in this embodiment, the radius of the circular structuring element for the closing operation is 5 pixels, which is used to fill small holes inside the tortoise shell glue body. Connectivity analysis is performed on the morphologically processed binary image to identify all connected regions and calculate the area of ​​each connected region. Select the largest connected region as the main region of the tortoise shell glue, and determine the bounding rectangle of this region. Based on the circumscribed rectangular bounding box, the standardized tortoise shell glue transmission image The target region image is obtained by cropping. ; For the target region image The image is then scaled to a preset standard size to obtain a standardized target region image. In this embodiment, the standardized size is 224×224.

[0024] A3: Input the standardized target region image into a multi-branch parallel feature perception network for feature extraction, such as... Figure 2 As shown, it includes: Standardize the target region image The input is fed into a multi-branch parallel feature perception network, which includes three independent branch structures: a light transmission uniformity feature extraction branch, a color purity feature extraction branch, and an internal microscopic defect feature extraction branch. Transmittance uniformity feature extraction branch The calculation process for the feature map of a layer is as follows: ; in, The first branch representing the feature extraction of light transmission uniformity a feature map of the layer, , denotes a convolution operation, a kernel weight matrix of the layer, denotes a convolution operation, denotes a bias vector of the layer, denotes a sigmoid activation function; a feature map of the layer, a feature map of the layer, the calculation process of the feature map of the layer is as follows: ; wherein, a feature map of the layer, a feature map of the layer, , a kernel weight matrix of the layer, a bias vector of the layer, a feature map of the layer, a feature map of the layer, the calculation process of the feature map of the layer is as follows: ; wherein, a feature map of the layer, a feature map of the layer, , a kernel weight matrix of the layer, a bias vector of the layer, a feature map of the layer, a feature map of the layer, the dimension of the feature map of the last layer of each branch is reduced by a global average pooling operation to obtain a corresponding deep feature vector: ; wherein, denotes a light transmission uniformity deep feature vector, denotes a color purity deep feature vector, denotes an internal microscopic flaw deep feature vector, denote global average pooling operations of the corresponding branches respectively, denote feature maps of the last layer of the three branches respectively, and in the embodiment .

[0025] ​A4: the three depth feature vectors of light transmission uniformity, color purity and internal microscopic flaw are constructed into a feature graph, the mutual relationship between different features is adaptively fused through iterative inference and attention mechanism of the graph neural network, and end-to-end training and optimization are performed to generate a comprehensive feature representation for tortoise shell glue quality detection: A41: the depth feature vector of light transmission uniformity , the depth feature vector of color purity , and the depth feature vector of internal microscopic flaw are constructed into a feature graph , the feature graph is composed of a node set and an edge set , wherein each depth feature vector is a node in the graph; An adjacency matrix of the feature graph is constructed , the adjacency matrix is used to represent the association strength between different feature nodes, and the initialization formula of the adjacency matrix is: ; Wherein, represents the edge weight between node and node , , and represent the feature vectors corresponding to node and node , represents the L2 norm, is a hyperparameter for controlling the distribution of adjacency weights, which is 0.9 in this embodiment; When the condition number of the adjacency matrix is too large, a regularization stabilization algorithm is used to iteratively update the adjacency matrix: ; Wherein, is a global regularization parameter, is an identity matrix, is a diagonal regularization parameter, which is 0.2 in this embodiment, represents a diagonal matrix composed of the main diagonal elements of the adjacency matrix , and the regularization strength is adaptively determined by the condition number: ; Wherein, is a maximum function for selecting the maximum value from the input numerical value; The adjacency matrix is symmetrically normalized to obtain a normalized adjacency matrix : ; in, For degree matrix, Represents a node The degree, Degree matrix The negative second power; A42: Iterative inference and fusion of feature maps using graph neural networks: A421: The first... of graph neural networks Layer unaggregated node feature matrix The calculation method is as follows: ; in, Indicates the first The node feature matrix of the layer Represents the initial node feature matrix. Indicates the first The learnable weight matrix of the layer, Indicates the first Layer bias vector; A422: Introduces a feature relationship attention mechanism to calculate the dynamic relationship weights between different feature nodes. : ; in, Indicates the first Layer nodes For nodes Attention weights This is a learnable attention parameter vector. For matrix transpose, Indicates the first The linear transformation matrix of the layer, and They represent the first Nodes in the feature matrix of unaggregated nodes in the layer and nodes eigenvectors, This indicates a feature concatenation operation. For normalized exponential functions, It is a linear rectified activation function with leakage; A423: Adaptive aggregation based on dynamic relation weights yields the... In the layer node feature matrix The feature vector of each node, the adaptive aggregation specifically is as follows: ; in, Represents a node In the Layer node feature matrix The feature vector in Represents a node The set of neighboring nodes, The range of values ​​and Same but , Indicates the first The self-connection weight matrix of the layer; Indicates the first Layer nodes For nodes Attention weights Indicates the first Nodes in the feature matrix of unaggregated nodes in the layer eigenvectors; right Repeat steps A421 to A423 Next, the feature evolution sequence is obtained. ,go through After iterative inference in the layered graph neural network, a global pooling operation is performed on the feature matrix of the last layer's nodes to obtain a comprehensive feature representation. : ; in, This indicates a global pooling operation. These represent mean pooling, max pooling, and standard deviation pooling operations, respectively. This represents the final node feature matrix of the last layer, in this embodiment. , Indicates feature concatenation operation; A43: The graph neural network is trained end-to-end using tortoise shell glue samples with quality labels. The training loss function is defined as cross-entropy loss. The Adam optimizer is used to update the network parameters, with a learning rate of 0.001, a training epoch count of 200, and a batch size of 32. The standardized target region image of the tortoise shell glue sample to be detected is then input into the trained multi-branch parallel feature perception network and graph neural network to obtain a comprehensive feature representation. Based on this comprehensive feature representation, quality classification prediction is performed, and the quality grade and confidence score of the tortoise shell glue sample are output. During the training phase, the graph neural network is trained end-to-end using tortoise shell glue samples with quality labels, and the training loss function is defined as cross-entropy loss. During the inference phase, the standardized target region image of the tortoise shell glue sample to be detected is input into the trained multi-branch parallel feature perception network and graph neural network to obtain a comprehensive feature representation. Based on this feature representation, quality classification prediction is performed, and the quality grade and confidence score of the tortoise shell glue sample are output.

[0026] The present application uses samples from different batches of tortoise shell glue to establish a data set containing 1200 samples, of which 360 are superior, 420 are good, 300 are medium, and 120 are inferior. All samples are manually labeled by traditional Chinese medicine experts and used as true labels for experimental verification.

[0027] The standardized image acquisition device specifically includes an LED backlight light source (color temperature 6500K, brightness uniformity >95%), a high-resolution digital camera (2048x1536 pixels), and a precision sample stage. The deep learning network uses the PyTorch 1.9.0 framework.

[0028] The feature extraction capability of the multi-branch parallel feature perception network is verified by t-SNE dimension reduction visualization, which is more obvious than Figure 3 the features extracted by the ResNet-50 network shown in (b), Figure 3 The features extracted by the light transmission uniformity branch in the multi-branch parallel feature perception network shown in (a) show more obvious clustering distribution between different quality grades.

[0029] The comparison experiment results of the existing method are shown in Table 1:

[0030] The method of the present application is significantly better than the comparison method in all evaluation indicators, and the overall accuracy reaches 93.6%, which is 5.7 percentage points higher than the optimal comparison method.

[0031] Example 2: The present application also discloses an image recognition-based tortoise shell glue quality detection system, which includes the following four modules: Imaging module: obtain the tortoise shell glue sample to be detected, place the tortoise shell glue sample in the standardized image acquisition device for transmission imaging and pretreatment; Segmentation and size standardization module: image segmentation and size standardization processing are performed on the standardized tortoise shell transmission image to obtain a standardized target region image; Deep feature extraction module: input the standardized target region into the multi-branch parallel feature perception network for feature extraction; Quality detection module: the light transmission uniformity, color and luster purity, and internal microscopic flaw three deep feature vectors are constructed into a feature map, the mutual relationship between different features is adaptively fused through the iterative reasoning of the graph neural network and the attention mechanism, and end-to-end training optimization is performed to generate a comprehensive feature representation for tortoise shell glue quality detection.

[0032] It should be noted that the above-mentioned embodiment serial numbers of the present application are only for description, not representing the advantages and disadvantages of the embodiments. Also, the terms "comprising", "containing" or any other variants thereof in this document are intended to cover the non-exclusive inclusion, so that the process, device, article or method including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, device, article or method. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0033] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the methods described in various embodiments of the present application.

[0034] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent flow transformation made by using the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for quality inspection of tortoise shell glue based on image recognition, characterized in that, Includes the following steps: A1: Obtain the tortoise shell glue sample to be tested, place the tortoise shell glue sample in a standardized image acquisition device for transmission imaging and preprocessing, and obtain a standardized tortoise shell glue transmission image. A2: Perform image segmentation and size normalization on the standardized tortoise shell glue transmission image to obtain a standardized target region image; A3: Input the standardized target area image into a multi-branch parallel feature perception network for feature extraction to obtain three depth feature vectors: light transmission uniformity, color purity, and internal micro-defects; A4: The three deep feature vectors of light transmission uniformity, color purity, and internal micro-defects are constructed into a feature map. The relationships between different features are adaptively fused through iterative reasoning and attention mechanism of graph neural network, and end-to-end training and optimization are performed to generate a comprehensive feature representation for the quality detection of tortoise shell glue.

2. The method for quality detection of tortoise shell glue based on image recognition according to claim 1, characterized in that, Step A1 includes: A11: Prepare a standardized image acquisition device; the standardized image acquisition device includes a backlight source, a sample stage, and a high-resolution digital camera; A12: Under constant illumination conditions and fixed shooting parameters, control the high-resolution digital camera to perform transmission imaging on the tortoise shell glue sample to obtain the original transmission image. ; A13: Regarding the original transmission image Preprocessing was performed, including noise reduction, brightness normalization, and color correction, to obtain a standardized tortoise shell resin transmission image. ; The denoising process uses a Gaussian filter; the brightness normalization process adjusts the average brightness value of the image to a preset target brightness value; and the color correction process performs white balance correction based on a standard color reference chart.

3. The image recognition-based method for detecting the quality of tortoise shell glue according to claim 2, characterized in that, Step A2 includes: The standardized tortoise shell glue transmission image is converted to grayscale to obtain a grayscale image; the grayscale image is then subjected to adaptive threshold segmentation, with an adaptive threshold calculated based on the local statistical characteristics of the grayscale image; binarization is performed based on the adaptive threshold to obtain a binary image; morphological processing, including opening and closing operations, is performed on the binary image to obtain a morphologically processed image; connected component analysis is performed on the morphologically processed image, and the connected region with the largest area is selected as the main region of the tortoise shell glue; the standardized tortoise shell glue transmission image is then segmented based on the bounding rectangle of the main region of the tortoise shell glue. The target region image is obtained by cropping. and the target area image Scale to a preset standard size to obtain a standardized target area image. .

4. The image recognition-based method for detecting the quality of tortoise shell glue according to claim 3, characterized in that, Step A3 includes: Standardize the target region image The input is fed into a multi-branch parallel feature perception network, which includes three independent branch structures: a light transmission uniformity feature extraction branch, a color purity feature extraction branch, and an internal microscopic defect feature extraction branch. Transmittance uniformity feature extraction branch The calculation process for the feature map of a layer is as follows: ； in, The branch representing the feature extraction of light transmission uniformity Feature map of the layer , The branch representing the feature extraction of light transmission uniformity The convolution kernel weight matrix of the layer, This represents the convolution operation. The branch representing the feature extraction of light transmission uniformity The layer's bias vector, Represents the sigmoid activation function; Color purity feature extraction branch The calculation process for the feature map of a layer is as follows: ； in, The first branch representing the feature extraction of color purity Feature map of the layer , The first branch representing the feature extraction of color purity The convolution kernel weight matrix of the layer, The first branch representing the feature extraction of color purity Layer bias vector; Internal micro-defect feature extraction branch The calculation process for the feature map of a layer is as follows: ； in, The branch representing the extraction of internal micro-defect features Feature map of the layer , The branch representing the extraction of internal micro-defect features The convolution kernel weight matrix of the layer, The branch representing the extraction of internal micro-defect features Layer bias vector; Global average pooling is used to reduce the dimensionality of the feature maps in the last layer of each of the three branches, resulting in the corresponding deep feature vectors. , and ;in, This represents the depth feature vector representing the uniformity of light transmission. This represents the depth feature vector representing color purity. This represents the depth feature vector of internal micro-defects.

5. The image recognition-based method for detecting the quality of tortoise shell glue according to claim 4, characterized in that, Step A4 includes: A41: Construct a feature map from three deep feature vectors. Construct an adjacency matrix Adjacency matrix represents the association strength between different feature nodes. The initialization formula is: ； in, Represents a node and nodes Edge weights between them , and Representing nodes respectively and nodes The corresponding feature vector, Represents the L2 norm. Hyperparameters for controlling the distribution of adjacency weights; For adjacency matrix Perform symmetric normalization to obtain the normalized adjacency matrix. ; A42: Utilizing graph neural networks for iterative reasoning and fusion of feature maps; introducing a feature relation attention mechanism to calculate dynamic relation weights; and performing [further steps] on the graph neural network. After layer-by-layer iterative reasoning, a global pooling operation is performed to obtain a comprehensive feature representation; A43: The graph neural network is trained end-to-end using samples of tortoise shell glue with quality labels. The training loss function is defined as cross-entropy loss. The network parameters are updated using the Adam optimizer to obtain the trained graph neural network. The standardized target region image of the tortoise shell glue sample to be detected is input into the multi-branch parallel feature perception network and the trained graph neural network to obtain a comprehensive feature representation. Based on the comprehensive feature representation, quality classification prediction is performed, and the quality grade and confidence score of the tortoise shell glue sample are output.

6. The image recognition-based method for detecting the quality of tortoise shell glue according to claim 5, characterized in that: A421: The first... of graph neural networks Layer unaggregated node feature matrix The calculation method is as follows: ； in, Indicates the first The node feature matrix of the layer, Represents the initial node feature matrix. Indicates the first The learnable weight matrix of the layer, Indicates the first Layer bias vector; A422: Introduces a feature relationship attention mechanism to calculate the dynamic relationship weights between different feature nodes. : ； in, Indicates the first Layer nodes For nodes Attention weights This is a learnable attention parameter vector. For matrix transpose, Indicates the first The linear transformation matrix of the layer, and They represent the first Nodes in the feature matrix of unaggregated nodes in the layer and nodes eigenvectors, This indicates a feature concatenation operation. For normalized exponential functions, It is a linear rectified activation function with leakage; A423: Adaptive aggregation based on dynamic relation weights yields the... In the layer node feature matrix The feature vector of each node, the adaptive aggregation specifically is as follows: ； in, Represents a node In the Layer node feature matrix The feature vector in Represents a node The set of neighboring nodes, The range of values ​​and Same but , Indicates the first The self-connection weight matrix of the layer; Indicates the first Layer nodes For nodes Attention weights Indicates the first Nodes in the feature matrix of unaggregated nodes in the layer eigenvectors; right Repeat steps A421 to A423 Next, the feature evolution sequence is obtained. ,go through After iterative inference in the layered graph neural network, a global pooling operation is performed on the feature matrix of the last layer's nodes to obtain a comprehensive feature representation. : ； in, This indicates a global pooling operation. These represent mean pooling, max pooling, and standard deviation pooling operations, respectively. This represents the feature matrix of the nodes in the last layer. This indicates a feature splicing operation.

7. A quality inspection system for tortoise shell glue based on image recognition, characterized in that, include: Imaging module: Acquires the tortoise shell glue sample to be tested, places the tortoise shell glue sample in a standardized image acquisition device for transmission imaging and preprocessing; Segmentation and Size Standardization Module: Performs image segmentation and size standardization on the standardized tortoise shell glue transmission image to obtain a standardized target region image; Deep feature extraction module: Inputs the standardized target region into a multi-branch parallel feature perception network for feature extraction; Quality Inspection Module: The module constructs a feature map by using three deep feature vectors: light transmission uniformity, color purity, and internal micro-defects. It adaptively fuses the relationships between different features through iterative inference and attention mechanism of graph neural network, and performs end-to-end training and optimization to generate a comprehensive feature representation for the quality inspection of tortoise shell glue. To achieve the image recognition-based method for detecting the quality of tortoise shell glue as described in any one of claims 1-6.

Images