Method, system, device and medium for ginseng slice grading based on visual features
By combining frequency domain hybrid enhancement and asymmetric visual coding and decoding networks with a self-supervised pre-training of a perceptual prior loss function, the problems of human subjectivity and high cost in the grading of Panax notoginseng slices are solved, and efficient and accurate slice grading is achieved.
Patent Information
- Application Number
- CN202611579783.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-13
- Publication Date
- 2026-11-10
AI Technical Summary
Existing methods for grading Panax notoginseng slices suffer from several problems, including strong subjectivity in manual evaluation, low detection efficiency, high cost of chemical detection, reliance on a large number of labeled samples for supervised learning models, and difficulty in effectively learning the overall radial structural features of "radial texture" using traditional self-supervised image reconstruction methods.
A visual feature-based three-seven slice grading method is adopted, which combines frequency domain hybrid enhancement processing, asymmetric visual coding and decoding network and perceptual prior loss function with few-shot prototype learning to achieve self-supervised pre-training and grade determination.
The model's robustness and accuracy to Panax notoginseng slices were improved, while computational load and GPU memory usage were reduced. Overfitting was avoided, high-precision grade determination was achieved, and annotation costs were lowered.
Smart Images

Figure CN123071986A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and quality evaluation of Chinese medicinal materials, and in particular to a method, system, device and medium for grading Panax notoginseng slices based on visual features. Background Technology
[0002] Panax notoginseng is a commonly used and valuable traditional Chinese medicine, and its slices are one of the important forms of Panax notoginseng in market circulation, processing, and application. With the development of primary processing and fresh-cut processing models at the place of origin, the main root of Panax notoginseng is washed, sliced, and dried to form Panax notoginseng slices. The overall structure of the main root is destroyed, and the traditional method of judging the grade based on the number of roots per unit weight is not directly applicable to the slice morphology. Therefore, establishing a rapid, low-cost, non-destructive, and large-scale quality grading method suitable for Panax notoginseng slices is of great significance for the production, processing, circulation, and quality control of Panax notoginseng slices.
[0003] In traditional Panax notoginseng quality evaluation, judgments are often made based on the color depth, texture distribution, and overall appearance characteristics of the radial striations on the cut surface. Generally, samples with darker radial striations and more prominent radial striations are considered to be of higher quality, while samples with lighter colors and weaker striations are considered to be of relatively lower quality. This type of quality evaluation method based on appearance can provide a quick and intuitive basis for judgment in market transactions, but it is highly dependent on human experience and suffers from problems such as strong subjectivity, poor evaluation consistency, low efficiency, and difficulty in large-scale application.
[0004] Existing methods for quality inspection of Panax notoginseng slices include chemical composition detection, spectroscopic detection, electronic nose, electronic tongue, and near-infrared analysis. While chemical detection methods can achieve quantitative analysis of specific components, they typically suffer from problems such as sample destruction, long detection cycles, high equipment costs, and difficulty in performing online detection of each slice. Although near-infrared, electronic nose, and electronic tongue methods have certain rapid detection capabilities, they are mostly applicable to powder samples or specific detection scenarios, and they do not fully utilize the appearance and structural information of Panax notoginseng slices. Furthermore, their detection accuracy and industrial applicability remain limited.
[0005] In recent years, deep learning and computer vision technologies have provided new technical pathways for the appearance recognition and intelligent grading of Chinese medicinal materials. Deep learning models such as convolutional neural networks and Transformer models can automatically extract color, texture, shape, and spatial structure features from images and use them for quality classification of Chinese medicinal materials. However, most existing image grading models rely on supervised learning paradigms, requiring a large amount of high-quality labeled data for training. For Panax notoginseng slices, reliable grade labels usually require expert experience or experimental testing support, resulting in high labeling costs, limited sample size, and difficulty in constructing large-scale labeled datasets. Directly training supervised models using only a small number of labeled images can easily lead to overfitting, resulting in weak model generalization ability.
[0006] Furthermore, the "radial texture" of Panax notoginseng slices is not simply local color points or isolated textures, but a macroscopic radial topological structure and gradient texture extending outward from the center of the slice. Traditional image self-supervised methods based on pixel reconstruction errors usually focus more on local pixel differences, making it difficult to fully constrain the overall structural consistency of the image and easily ignoring multi-scale radial texture and global visual perception features. Therefore, how to transform the traditional empirical judgment of the visual features of "radial texture" into a computable deep learning constraint, and achieve accurate classification of Panax notoginseng slices with a small number of labeled samples, is an urgent problem to be solved in the current intelligent quality evaluation of Panax notoginseng slices. Summary of the Invention
[0007] This invention aims to address the problems in existing Panax notoginseng slice grading methods, such as strong subjectivity in manual evaluation, low detection efficiency, high cost of chemical detection, reliance on a large number of labeled samples for supervised learning models, and the difficulty of traditional self-supervised image reconstruction methods in effectively learning the overall radial structural features of "radial texture". The invention provides a Panax notoginseng slice grading method, system, device and medium based on visual features.
[0008] In a first aspect, the present invention discloses a method for grading Panax notoginseng slices based on visual features, comprising the following steps: Obtain a set of Panax notoginseng slice cross-sectional images to be processed, and divide the set of Panax notoginseng slice cross-sectional images into an unlabeled image set and a labeled image set; Frequency domain blending enhancement processing is performed on the cross-sectional images of Panax notoginseng slices in the unlabeled image set to obtain enhanced images; The enhanced image is processed by image slicing and random masking to obtain visible image blocks and masked image blocks; The visible image patch is input into the visual encoder in the asymmetric visual encoder-decoder network for feature extraction to obtain a semantic feature matrix; The semantic feature matrix is input into the lightweight decoder in the asymmetric visual encoder-decoder network to reconstruct the image, thereby obtaining the reconstructed image; A perceptual prior loss function is constructed based on the reconstructed image and the enhanced image, and the asymmetric visual coding and decoding network is pre-trained in a self-supervised manner based on the perceptual prior loss function. After the self-supervised pre-training is completed, the network parameters of the visual encoder are frozen, and prototype vectors of different levels of categories are calculated using the labeled samples in the labeled image set. The visual encoder, after freezing network parameters, inputs the cross-sectional image of the Panax notoginseng slice to be tested into the visual encoder to extract the feature vector of the sample to be tested, and determines the target level of the cross-sectional image of the Panax notoginseng slice to be tested based on the distance between the feature vector of the sample to be tested and the prototype vector of each level category.
[0009] As an optional implementation of the first aspect of this application, frequency domain hybrid enhancement processing is performed on the cross-sectional images of Panax notoginseng slices in the unlabeled image set to obtain an enhanced image, including: selecting a first Panax notoginseng slice image and a second Panax notoginseng slice image from the unlabeled image set; performing two-dimensional fast Fourier transform on the first Panax notoginseng slice image and the second Panax notoginseng slice image respectively to obtain a first spectrum and a second spectrum; using a binary low-frequency mask centered on the DC component of the spectrum to perform spectrum fusion on the first spectrum and the second spectrum to obtain a hybrid spectrum, wherein the binary low-frequency mask has a value of 1 within the center radius and a value of 0 at other positions; and obtaining the enhanced image by performing an inverse Fourier transform on the hybrid spectrum.
[0010] As an optional implementation of the first aspect of this application, the enhanced image is subjected to image slicing and random masking to obtain visible image blocks and masked image blocks, including: normalizing the enhanced image to map the pixel values of the enhanced image to the range of [-1, 1]; dividing the normalized enhanced image into multiple non-overlapping image blocks; randomly selecting a portion of the non-overlapping image blocks as the masked image blocks according to a preset masking ratio, and using the unmasked non-overlapping image blocks as the visible image blocks.
[0011] As an optional implementation of the first aspect of this application, the visible image patch is input to a visual encoder in an asymmetric visual coding and decoding network for feature extraction to obtain a semantic feature matrix, including: obtaining the spatial location code corresponding to the visible image patch; mapping the visible image patch into a feature vector and superimposing it with the spatial location code to obtain an encoder input sequence; inputting the encoder input sequence to the visual encoder, extracting global visual semantics through the visual encoder, and outputting the semantic feature matrix.
[0012] As an optional implementation of the first aspect of this application, the semantic feature matrix is input into a lightweight decoder in the asymmetric visual coding and decoding network for image reconstruction to obtain a reconstructed image, comprising: mapping the semantic feature matrix to the decoder feature dimension through a linear projection matrix to obtain mapped features; concatenating the mapped features with learnable mask markers according to the original image patch space index, and superimposing decoder position encoding to obtain a decoder input sequence; inputting the decoder input sequence into the lightweight decoder, updating the features layer by layer through a multi-head self-attention module and a feedforward neural network, and obtaining the reconstructed image through a pixel prediction head.
[0013] As an optional implementation of the first aspect of this application, constructing a perceptual prior loss function based on the reconstructed image and the enhanced image includes: calculating the mean square error loss of the mask region between the predicted image patch corresponding to the mask image patch in the reconstructed image and the mask image patch; calculating the full image multi-scale structural similarity loss between the enhanced image and the reconstructed image; and using the product of the mask region mean square error loss and a first weight coefficient, plus the product of the full image multi-scale structural similarity loss and a second weight coefficient, as the perceptual prior loss function.
[0014] As an optional implementation of the first aspect of this application, the visual encoder after freezing network parameters inputs the cross-sectional image of the Panax notoginseng slice to be tested into the visual encoder to extract the feature vector of the sample to be tested, and determines the target level of the cross-sectional image of the Panax notoginseng slice to be tested based on the distance between the feature vector of the sample to be tested and the prototype vector of each level category, including: inputting the cross-sectional image of the Panax notoginseng slice to be tested into the visual encoder after freezing network parameters, obtaining the feature vector of the sample to be tested after forward propagation and global average pooling; calculating the mean of the feature vector of the sample to be tested to obtain the prototype vector of the corresponding level category; calculating the Euclidean distance between the feature vector of the sample to be tested and the prototype vector of each level category respectively; and taking the level category corresponding to the smallest Euclidean distance as the target level of the cross-sectional image of the Panax notoginseng slice to be tested.
[0015] Secondly, this application discloses a visual feature-based grading system for Panax notoginseng slices, comprising: The image acquisition and segmentation module is used to acquire a set of Panax notoginseng slice cross-sectional images to be processed, and to divide the set of Panax notoginseng slice cross-sectional images into an unlabeled image set and a labeled image set. The frequency domain enhancement module is used to perform frequency domain hybrid enhancement processing on the cross-sectional images of Panax notoginseng slices in the unlabeled image set to obtain an enhanced image; The masking module is used to perform image slicing and random masking on the enhanced image to obtain visible image blocks and masked image blocks. The feature extraction module is used to input the visible image patch into the visual encoder in the asymmetric visual encoder-decoder network for feature extraction to obtain a semantic feature matrix; The image reconstruction module is used to input the semantic feature matrix into the lightweight decoder in the asymmetric visual coding and decoding network to reconstruct the image and obtain the reconstructed image. The pre-training module is used to construct a perceptual prior loss function based on the reconstructed image and the enhanced image, and to perform self-supervised pre-training on the asymmetric visual coding and decoding network based on the perceptual prior loss function. The prototype calculation module is used to freeze the network parameters of the visual encoder after the self-supervised pre-training is completed, and to calculate prototype vectors of different levels of categories using the labeled samples in the labeled image set. The grade prediction module is used to input the cross-sectional image of the Panax notoginseng slice to be tested into the visual encoder after freezing the network parameters to extract the feature vector of the sample to be tested, and determine the target grade of the cross-sectional image of the Panax notoginseng slice to be tested based on the distance between the feature vector of the sample to be tested and the prototype vector of each grade category.
[0016] Thirdly, this application discloses an electronic device, comprising: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the method as described in the first aspect.
[0017] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the method described in the first aspect.
[0018] Compared with the prior art, the present invention has the following beneficial effects: (1) By introducing frequency domain hybridization enhancement processing, this invention effectively improves the diversity of image textures and enhances the robustness of the model to different shooting samples and appearance differences while keeping the macroscopic spatial structure of radial texture of Panax notoginseng slices intact.
[0019] (2) The present invention uses an asymmetric visual encoder-decoder network, which enables the visual encoder to process only the visible image blocks that are not masked, greatly reducing the amount of computation and memory usage, and uses a lightweight decoder to complete image reconstruction, which significantly improves the efficiency of self-supervised pre-training.
[0020] (3) The present invention constructs a perceptual prior loss function that integrates the mean square error of the mask region and the multi-scale structural similarity of the complete image. It transforms the empirical judgment of radial texture color depth and structure in the traditional evaluation of the quality of Sanqi slices into an optimizable visual perceptual constraint, so that while focusing on local pixel reconstruction, the model can deeply learn the overall radial topology and gradient texture features in the cross section of the slice.
[0021] (4) The present invention adopts a few-sample prototype learning method, which can complete the high-precision level determination with only a small number of labeled samples at each level. This effectively avoids the problem of overfitting of traditional supervised learning models under small sample conditions, and greatly improves the practicality and industrial promotion value of the model in low labeling cost scenarios. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0023] Figure 1 A schematic diagram of the overall process of a visual feature-based method for grading Panax notoginseng slices, provided in an embodiment of the present invention; Figure 2 A schematic diagram of the task-independent perception prior guidance pre-training stage provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the small-sample prototype evaluation stage provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a visual feature-based grading system for Panax notoginseng slices according to an embodiment of the present invention. Detailed Implementation
[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0025] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.
[0026] Example 1 This invention provides a method for grading Panax notoginseng slices based on visual features (e.g., Figure 1 As shown). This method mainly consists of two core stages: a task-independent perception prior-guided pre-training stage (e.g. Figure 2 As shown), the small sample prototype evaluation stage (such as...) Figure 3(As shown). Among them, the task-agnostic prior-guided pre-training stage aims to use a large number of unlabeled Panax notoginseng slice cross-sectional images to enable the deep neural network to automatically learn the color depth, radial texture distribution, spatial topological relationship and overall macroscopic structural features of the "radial texture" in the Panax notoginseng slice cross-section through a carefully designed proxy task (image reconstruction). The few-shot prototype evaluation stage, after the pre-trained model has a strong feature representation capability, uses only a very small number of expert-labeled samples to complete the final Panax notoginseng slice grade determination through metric learning.
[0027] Specifically, the method provided in this embodiment of the invention is performed according to the following steps: S11: Obtain the set of Panax notoginseng slice cross-sectional images to be processed, and divide the set of Panax notoginseng slice cross-sectional images into an unlabeled image set and a labeled image set.
[0028] In actual industrial production or quality inspection scenarios, the first step is to acquire cross-sectional images of Panax notoginseng slices using industrial cameras, high-definition cameras, or scanning equipment. Panax notoginseng slices are the product of processing the main root of Panax notoginseng through washing, slicing, and drying. Because slicing disrupts the overall appearance of the main root, the traditional grading standard based on the "head count" (i.e., the number of heads per unit weight) is no longer applicable. In the sliced form, the core basis for quality evaluation shifts to the visual characteristics of the cut surface, especially the "radial texture" that extends radially from the center to the edge.
[0029] In this embodiment, the acquired raw images may have issues such as inconsistent size and uneven illumination. Therefore, after acquiring the image set, a unified preprocessing operation is required for the raw images. Preferably, the size of all input images is uniformly adjusted to 224×224 pixels to adapt to the input requirements of the subsequent deep learning model. At the same time, the pixel values of the images are normalized, for example, the original pixel values in the [0, 255] interval are linearly mapped to the [-1, 1] interval. This helps to accelerate the convergence speed of the neural network and improve the stability of numerical calculation.
[0030] Based on the color depth and texture characteristics of the radial texture of Panax notoginseng slices, this embodiment classifies the Panax notoginseng slices into three categories: Grade 1 (denoted as F), Grade 2 (denoted as S), and Grade 3 (denoted as T). The specific classification criteria are as follows: Grade F slices are usually darker in color, with extremely obvious and continuous radial texture, representing the best quality; Grade S slices have moderate color and texture, belonging to medium quality; Grade T slices have lighter in color, with relatively weak or even blurry radial texture, representing relatively low quality.
[0031] Regarding dataset partitioning, to fully leverage the advantages of self-supervised learning, the vast majority of collected images were assigned to the "unlabeled image set." These images do not require manual F, S, and T level labeling, significantly reducing labor costs. Only a very small number of images were retained and precisely labeled by experts in Chinese medicinal herb identification, and assigned to the "labeled image set" for subsequent few-sample classification.
[0032] S12: Perform frequency domain mixing enhancement processing on the cross-sectional images of Panax notoginseng slices in the unlabeled image set to obtain enhanced images.
[0033] In deep learning, data augmentation is a crucial method for improving a model's generalization ability. However, traditional data augmentation methods (such as random cropping, large-angle rotation, and mesh occlusion) often operate directly in the spatial domain, which can easily disrupt the continuous spatial topology of the vital "radial texture" in the cross-section of a scallion pancake. Once this macroscopic radial structure is destroyed, the model will struggle to learn the correct quality assessment features.
[0034] To address this issue, this invention innovatively introduces a "Frequency Domain Mixup" strategy. The core idea of this strategy is to fuse images in the frequency domain, thereby greatly enriching the texture details and appearance diversity of the image without destroying the macroscopic spatial structure.
[0035] The specific implementation process is as follows: First, during the training process, a preset trigger probability is used. (Preferably, set to 0.5) Two different cross-sectional images of Panax notoginseng slices are randomly selected from the unlabeled image set and denoted as the first Panax notoginseng slice image. Image of slice 237 .
[0036] Next, a two-dimensional Fast Fourier Transform (2D-FFT) is performed on each of the two images. The Fourier Transform converts the image from the spatial domain to the frequency domain, yielding the corresponding first spectrum. Second spectrum In the spectrum, low-frequency components typically represent areas of gradual change in an image (such as overall contours, macroscopic structure, and basic colors), while high-frequency components represent areas of dramatic change in an image (such as edges, detailed textures, and noise).
[0037] Subsequently, a binary low-frequency mask centered on the DC component of the spectrum was employed. For the first spectrum Second spectrum Spectrum fusion is performed, where M takes a value of 1 within the central radius r and a value of 0 at other locations. This binary low-frequency mask... Its function is to distinguish between low-frequency and high-frequency regions in the spectrum. The formula for fusion is: in Indicates a mixed spectrum. This represents element-wise multiplication; In this way, the mixed spectrum In fact, it retains the low-frequency macroscopic structural information of the first 37 slice image, while injecting the high-frequency texture detail information of the second 37 slice image.
[0038] Finally, by analyzing the mixed spectrum Performing an inverse fast fourier transform (IFFT) converts the signal back from the frequency domain to the spatial domain, thus obtaining the final enhanced image. .
[0039] The enhanced image generated through the above frequency domain blending enhancement process Visually, the model retains the overall radial outline of the Panax notoginseng slices in the first slice image, but incorporates features from the second slice image in its local texture details and subtle color gradients. This allows the model to perceive a vast array of texture combinations within a limited data scale, significantly improving its robustness to samples taken from different batches under different lighting conditions.
[0040] S13: Perform image slicing and random masking on the enhanced image to obtain visible image blocks and masked image blocks.
[0041] In obtaining enhanced images Subsequently, this embodiment employs a masked autoencoder (MAE) paradigm for data processing. First, the enhanced image is normalized to ensure that pixel values are within the range of [-1, 1].
[0042] Subsequently, the enhanced image with a size of 224×224 is divided into multiple image patches of fixed size that do not overlap. Preferably, the size of the image patch is set to... Pixel, in this embodiment =16. Therefore, a complete image will be divided into (224 / 16)×(224 / 16) = 14×14 = 196 non-overlapping image patches.
[0043] Next, these image blocks are randomly masked according to a preset masking ratio γ. In this invention, the preset masking ratio is preferably 90%. This means that out of 196 image blocks, approximately 176 image blocks will be randomly selected and replaced with learnable mask markers, becoming "masked image blocks". Only about 20 image patches remained unchanged, becoming "visible image patches"; ".
[0044] A key technical feature of this invention is the use of a mask ratio as high as 90%. In traditional natural image processing, the mask ratio is typically set to 75%. However, images of Panax notoginseng slices exhibit high structural redundancy (i.e., adjacent regions have very similar colors and textures). If the mask ratio is too low, the model can complete the reconstruction task simply through local pixel interpolation or neighboring block copying, resulting in the model being unable to learn the global structure of radial texture. By setting an extremely high mask ratio of 90%, the model faces an extremely stringent reconstruction challenge: it must rely solely on the scattered 10% visible area to infer the macroscopic contour of the entire Panax notoginseng slice cross-section, the direction of radial texture extension, and the color gradient trend. This highly challenging surrogate task forces the model to abandon local pixel matching and instead delve into understanding the global visual semantics and topological structure of the Panax notoginseng slice.
[0045] S14: Input the visible image block into the visual encoder in the asymmetric visual coding and decoding network for feature extraction to obtain the semantic feature matrix.
[0046] This invention constructs an asymmetric vision encoder-decoder network. The term "asymmetric" refers to the significant differences between the encoder and decoder in terms of the number of network layers, feature dimensions, and the amount of data processed.
[0047] During the encoding phase, the visual encoder Only receive the "visible image patch" that is not masked. "As input. Specifically, the spatial location codes of these visible image patches in the original image are first obtained." Positional encoding is crucial for the Transformer architecture because it provides the model with information about the spatial arrangement of image patches. After mapping the pixel features of visible image patches to feature vectors via linear projection, these vectors are element-wise superimposed with the corresponding spatial positional encodings to obtain the encoder input sequence.
[0048] Subsequently, the encoder input sequence is input to the visual encoder. Preferably, the visual encoder employs a standard Vision Transformer (ViT) structure, such as the ViT-Base network. This network contains 12 Transformer encoding modules with a feature embedding dimension of 768. In each layer, a multi-head self-attention mechanism is used to capture long-distance dependencies between different visible image patches, extracting high-level global visual semantic features, and finally outputting a semantic feature matrix.
[0049] in This represents the semantic feature matrix output by the visual encoder; Because the visual encoder processes only 10% of the visible image patch instead of all 196 tokens, the computational complexity of the self-attention mechanism (which is proportional to the square of the number of tokens) is significantly reduced. This design not only greatly reduces computation and memory usage, improving pre-training speed, but also preserves key visual semantic information related to the "radial texture" of the 3 / 7 slice.
[0050] S15: Input the semantic feature matrix into the lightweight decoder in the asymmetric visual coding and decoding network to reconstruct the image and obtain the reconstructed image.
[0051] In the decoding stage, the goal is to recover the complete original image based on the semantic features extracted by the encoder. To achieve this goal, this invention designs a lightweight decoder. .
[0052] The specific steps are as follows: First, since the feature dimension of the decoder is usually smaller than that of the encoder (to reduce computation), a linear projection matrix is required. The semantic feature matrix output by the encoder Mapping to the decoder feature dimension yields the mapped features.
[0053] Next, learnable masking tags are introduced. These mask markers This represents the image patches that were occluded in step S13. The mapping features (representing visible areas) and mask markers (representing occluded areas) are strictly concatenated sequentially according to their true spatial indices in the original image. After concatenation, to allow the decoder to know the specific physical location of each token, decoder-specific location encoding is superimposed again. Thus, the complete decoder input sequence is obtained. .
[0054] in Indicates splicing.
[0055] Then, the decoder input sequence Input to lightweight decoder Preferably, the lightweight decoder contains 8 Transformer decoding modules (fewer than the encoder's 12 layers, reflecting asymmetry). Inside the decoder, sequence features are updated layer by layer using a multi-head self-attention module (MSA), layer normalization (LN), and a feedforward neural network (FFN). The decoder infers pixel information of occluded areas step by step by calculating the attention weights between visible features and mask markers.
[0056] Finally, after all the decoding layers, the sequence features are fed into a pixel prediction head (usually a simple linear layer). The pixel prediction head maps the feature vector of each token back to the pixel space of the original image patch (i.e., 16×16×3 = 768 dimensions), and then performs a rearrangement operation ( Finally, the images are stitched together to generate a complete reconstructed image. .
[0057] in, Indicates the first in the decoder The output of the layer, Indicates the first in the decoder The output of the layer, Indicates the current floor number. This represents the features after processing by the multi-head self-attention module (MSA). Indicates the number of decoder layers. Indicates the first in the decoder The output of the layer, This represents the pixel prediction head weight matrix.
[0058] S16: Construct a perceptual prior loss function based on the reconstructed image and the enhanced image, and perform self-supervised pre-training on the asymmetric visual coding and decoding network based on the perceptual prior loss function.
[0059] The design of the loss function is crucial in guiding the model's learning direction. Traditional MAE models typically use only the mean squared error (MSE) as the loss function, calculating the pixel-level difference between the reconstructed and original images. However, for 37-slice images, the key to quality evaluation lies in the overall structure and color gradation of the "radial texture," not just the accuracy of isolated pixels. If only MSE is used, the model is prone to getting bogged down in fitting local pixels, neglecting the macroscopic radial topological structure.
[0060] To this end, this invention innovatively proposes a "perceptual prior loss function", which consists of two parts: the mean square error loss of the mask region and the multi-scale structural similarity loss of the complete image.
[0061] Part 1: Mean Square Error Loss in Mask Region This loss is calculated only for the region corresponding to the masked image patch. Specifically, it calculates the mean squared error (MSE) between the predicted image patch at the location corresponding to the masked image patch in the reconstructed image and the true masked image patch. Calculating the MSE only in the masked region forces the model to make true inference predictions rather than simply copying pixels from the visible region, thus avoiding shortcut learning. This part of the loss ensures that the model possesses basic local pixel-level reconstruction capabilities.
[0062] The calculation method is as follows: in, This represents the i-th mask image block. This represents the predicted image patch in the reconstructed image Y that corresponds to the i-th mask image patch. This indicates the number of patches that are masked in a single sample (i.e., a masked image). This represents the squared Frobenius norm, which is the sum of the squares of the differences between all elements.
[0063] Part Two: Multi-scale Structural Similarity Loss for Complete Images This loss directly affects the relationship between the complete enhanced image (i.e., the network's target output) and the complete reconstructed image. Structural similarity (SSIM) is a metric that measures the similarity between two images, comprehensively evaluating them from three dimensions: brightness, contrast, and structure. Multi-scale structural similarity further calculates SSIM at different receptive field scales. By introducing this loss, the model is forced to maintain consistency between the reconstructed image and the original image at a macroscopic level. For a 3 / 7 slice, this means the model must accurately reproduce the radial texture structure extending from the center to the edges, the overall contour shape, and the macroscopic gradient trend of the slice color.
[0064] The calculation method is as follows: in, Indicates enhanced image With reconstructed images Multiscale structural similarity.
[0065] Ultimately, perceiving prior loss The calculation formula is: in, and These represent the weighting coefficients for the mean squared error loss of the masked region and the multi-scale structural similarity loss, respectively. Preferably, =0.6, =0.4.
[0066] Through this joint optimization strategy, the model is constrained by both local pixel details and global radial texture structure during the self-supervised pre-training process. This transforms the experienced judgments of senior experts on the "radial texture" of Panax notoginseng slices into a computationally calculable and optimizable visual perception prior for the deep learning model.
[0067] Based on this perceptual prior loss function, an optimizer (such as the AdamW optimizer, with a base learning rate of 3×10) is used. -4 The asymmetric visual encoder-decoder network is backpropagated and its parameters are updated using a cosine annealing learning rate decay strategy until pre-training is complete.
[0068] S17: After the self-supervised pre-training is completed, freeze the network parameters of the visual encoder and use the labeled samples in the labeled image set to calculate the prototype vectors of different level categories.
[0069] After self-supervised pre-training on the massive amount of unlabeled data, the visual encoder has acquired extremely powerful feature extraction capabilities, enabling it to deeply understand the radial texture and structural features of Panax notoginseng slices. At this point, it enters the few-shot prototype evaluation stage.
[0070] First, discard the lightweight decoder, keep only the visual encoder, and "freeze" the visual encoder. Freezing all network parameters means that the encoder weights will not be changed in subsequent classification stages. This operation eliminates the risk of severe overfitting caused by fine-tuning a large model with extremely small samples.
[0071] Next, a classification benchmark is constructed using the very limited number of expert-annotated samples (i.e., the annotated image set) reserved in step S11. This invention employs the prototype network concept from few-shot learning. For any level category of Panax notoginseng slices... Select k labeled samples from this set to form the "support set" for that category. ".
[0072] Support set Each labeled sample in the image is input into the visual encoder after the freeze parameters are applied. In the image, after the encoder outputs the image patch feature sequence, it compresses the sequence features into a fixed-length global feature vector through a global average pooling (GAP) operation. This feature vector The quality characterization of this sample is highly condensed.
[0073] Subsequently, for the same support set The arithmetic mean of the feature vectors of all labeled samples within the same grade category is calculated. This mean vector is called the "prototype vector" of that grade category. "Prototype vector" This represents the central position or most typical feature of the category in the feature space. The F-level prototype vector (P) can be obtained through separate calculations. F S-level prototype vector (P) S ) and T-level prototype vector (P T ).
[0074] The calculation method is as follows: Where k represents the number of labeled samples for each category, preferably k = 5; Indicate category Support set; This represents the image feature vector obtained after freezing the visual encoder and global average pooling; Indicates support set The category label corresponding to the i-th sample in the dataset. Indicates support set The i-th labeled image in the image.
[0075] S18: Input the cross-sectional image of the Panax notoginseng slice to be tested into the visual encoder after freezing the network parameters to extract the feature vector of the sample to be tested, and determine the target level of the cross-sectional image of the Panax notoginseng slice to be tested based on the distance between the feature vector of the sample to be tested and the prototype vector of each level category.
[0076] In practical online detection or quality grading applications, when a new, unknown cross-sectional image of a Panax notoginseng slice is input into the system, the following operations are performed: First, the cross-sectional image of the Panax notoginseng slice to be tested is divided into image blocks in the same way as in step S11 (without masking) and then input into the encoder.
[0077] That is, the pre-processed cross-sectional image of the Panax notoginseng slice to be tested (As a query sample) Visual encoders with the same freeze parameters In the process of forward propagation and global average pooling operations by the encoder, the "feature vector of the sample to be tested" is extracted from the cross-sectional image of the Panax notoginseng slice. ".
[0078] Next, in the feature space, the feature vector of the sample to be tested is calculated respectively. With pre-calculated prototype vectors of each level category The distance between them is measured. In this embodiment, Euclidean distance is preferably used as the metric. The smaller the Euclidean distance, the closer the sample to the center of the category in the feature space, and the more similar its visual features are to the category.
[0079] The calculation method is as follows: in, This indicates the prediction level of the cross-sectional image of the Panax notoginseng slice to be tested.
[0080] Finally, the magnitudes of these three Euclidean distances are compared, and the grade category corresponding to the smallest Euclidean distance is directly used as the final prediction result (i.e., the target grade) of the cross-sectional image of the Panax notoginseng slice to be tested.
[0081] Through the above steps, this invention achieves high-precision, automated, and intelligent grading of Panax notoginseng slices based on radial texture visual features, without the need to construct a large labeled dataset, perform complex chemical tests, or damage the physical morphology of the slices, and with only a very small number of expert-labeled samples.
[0082] Example 2 like Figure 4 As shown, the present invention also discloses a visual feature-based grading system for Panax notoginseng slices, which includes the following key modules: The image acquisition and segmentation module 11 is used to acquire a set of Panax notoginseng slice cross-sectional images to be processed, and to divide the set of Panax notoginseng slice cross-sectional images into an unlabeled image set and a labeled image set. Frequency domain enhancement module 12 is used to perform frequency domain hybrid enhancement processing on the cross-sectional images of Panax notoginseng slices in the unlabeled image set to obtain an enhanced image; The masking module 13 is used to perform image slicing and random masking on the enhanced image to obtain visible image blocks and masked image blocks; Feature extraction module 14 is used to input the visible image patch into the visual encoder in the asymmetric visual encoder-decoder network for feature extraction to obtain a semantic feature matrix; Image reconstruction module 15 is used to input the semantic feature matrix into the lightweight decoder in the asymmetric visual coding and decoding network to reconstruct the image and obtain the reconstructed image. The pre-training module 16 is used to construct a perceptual prior loss function based on the reconstructed image and the enhanced image, and to perform self-supervised pre-training on the asymmetric visual coding and decoding network based on the perceptual prior loss function. The prototype calculation module 17 is used to freeze the network parameters of the visual encoder after the self-supervised pre-training is completed, and to calculate prototype vectors of different levels of categories using the labeled samples in the labeled image set. The grade prediction module 18 is used to input the cross-sectional image of the Panax notoginseng slice to be tested into the visual encoder after freezing the network parameters to extract the feature vector of the sample to be tested, and determine the target grade of the cross-sectional image of the Panax notoginseng slice to be tested based on the distance between the feature vector of the sample to be tested and the prototype vector of each grade category.
[0083] The visual feature-based 3 / 7 slicing grading system in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), etc. This application embodiment does not impose specific limitations.
[0084] The visual feature-based grading system for Panax notoginseng slices in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.
[0085] The present application provides a visual feature-based grading system for Panax notoginseng slices that can achieve... Figure 1 To avoid repetition, the various processes of the method embodiments shown will not be described again here.
[0086] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described embodiment of the method for grading slices of Panax notoginseng based on visual features, and can achieve the same technical effect. To avoid repetition, they will not be described again here.
[0087] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of the visual feature-based slicing and grading method for slicing and can achieve the same technical effect. To avoid repetition, they will not be described again here.
[0088] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.
[0089] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0090] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0091] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.
Claims
1. A method for grading Panax notoginseng slices based on visual features, characterized in that, Includes the following steps: Obtain a set of Panax notoginseng slice cross-sectional images to be processed, and divide the set of Panax notoginseng slice cross-sectional images into an unlabeled image set and a labeled image set; Frequency domain blending enhancement processing is performed on the cross-sectional images of Panax notoginseng slices in the unlabeled image set to obtain enhanced images; The enhanced image is processed by image slicing and random masking to obtain visible image blocks and masked image blocks; The visible image patch is input into the visual encoder in the asymmetric visual encoder-decoder network for feature extraction to obtain a semantic feature matrix; The semantic feature matrix is input into the lightweight decoder in the asymmetric visual encoder-decoder network to reconstruct the image, thereby obtaining the reconstructed image; A perceptual prior loss function is constructed based on the reconstructed image and the enhanced image, and the asymmetric visual coding and decoding network is pre-trained in a self-supervised manner based on the perceptual prior loss function. After the self-supervised pre-training is completed, the network parameters of the visual encoder are frozen, and prototype vectors of different levels of categories are calculated using the labeled samples in the labeled image set. The visual encoder, after freezing network parameters, inputs the cross-sectional image of the Panax notoginseng slice to be tested into the visual encoder to extract the feature vector of the sample to be tested, and determines the target level of the cross-sectional image of the Panax notoginseng slice to be tested based on the distance between the feature vector of the sample to be tested and the prototype vector of each level category.
2. The method according to claim 1, characterized in that, Frequency domain blending enhancement processing is performed on the cross-sectional images of Panax notoginseng slices in the unlabeled image set to obtain enhanced images, including: Select the first and second slice images of Panax notoginseng from the unlabeled image set; Two-dimensional fast Fourier transforms are performed on the first and second Panax notoginseng slice images to obtain the first and second spectra, respectively. A binary low-frequency mask centered on the DC component of the spectrum is used to perform spectrum fusion on the first spectrum and the second spectrum to obtain a mixed spectrum, wherein the binary low-frequency mask takes a value of 1 within the center radius and a value of 0 at other positions; The enhanced image is obtained by performing an inverse Fourier transform on the mixed spectrum.
3. The method according to claim 1, characterized in that, The enhanced image is subjected to image slicing and random masking to obtain visible image blocks and masked image blocks, including: The enhanced image is normalized to map the pixel values of the enhanced image to the range of [-1, 1]. The enhanced image after normalization is divided into multiple non-overlapping image blocks; According to a preset mask ratio, a portion of the non-overlapping image blocks are randomly selected as the mask image blocks, and the unmasked non-overlapping image blocks are used as the visible image blocks.
4. The method according to claim 1, characterized in that, The visible image patch is input into the visual encoder in an asymmetric visual encoder-decoder network for feature extraction, resulting in a semantic feature matrix, including: Obtain the spatial location code corresponding to the visible image block; After mapping the visible image patch into a feature vector, it is superimposed with the spatial location code to obtain the encoder input sequence; The encoder input sequence is input into the visual encoder, and the global visual semantics are extracted by the visual encoder to output the semantic feature matrix.
5. The method according to claim 4, characterized in that, The semantic feature matrix is input into the lightweight decoder in the asymmetric visual encoder-decoder network for image reconstruction, resulting in a reconstructed image, including: The semantic feature matrix is mapped to the decoder feature dimension using a linear projection matrix to obtain the mapped features; The mapping features and learnable mask labels are concatenated according to the original image patch spatial index, and the decoder position encoding is superimposed to obtain the decoder input sequence; The decoder input sequence is input into the lightweight decoder, and the features are updated layer by layer through a multi-head self-attention module and a feedforward neural network, and the reconstructed image is obtained through a pixel prediction head.
6. The method according to claim 1, characterized in that, Constructing a perceptual prior loss function based on the reconstructed image and the enhanced image includes: Calculate the mean square error loss of the mask region between the predicted image block at the position corresponding to the mask image block in the reconstructed image and the mask image block; Calculate the full image multi-scale structural similarity loss between the enhanced image and the reconstructed image; The product of the mean squared error loss of the masked region and the first weight coefficient, plus the product of the multi-scale structural similarity loss of the complete image and the second weight coefficient, is used as the perceptual prior loss function.
7. The method according to claim 1, characterized in that, The visual encoder, after freezing network parameters, inputs the cross-sectional image of the Panax notoginseng slice to be tested into the visual encoder to extract the feature vector of the sample to be tested. The target level of the cross-sectional image of the Panax notoginseng slice to be tested is determined based on the distance between the feature vector of the sample to be tested and the prototype vector of each level category, including: The cross-sectional image of the Panax notoginseng slice to be tested is input into the visual encoder after freezing the network parameters. After forward propagation and global average pooling, the feature vector of the sample to be tested is obtained. Calculate the mean of the feature vectors of the sample to be tested to obtain the prototype vector of the corresponding level category; Calculate the Euclidean distance between the feature vector of the sample to be tested and the prototype vector of each of the grade categories; The grade category corresponding to the smallest Euclidean distance is taken as the target grade of the cross-sectional image of the Panax notoginseng slice to be tested.
8. A visual feature-based grading system for Panax notoginseng slices, characterized in that, include: The image acquisition and segmentation module is used to acquire a set of Panax notoginseng slice cross-sectional images to be processed, and to divide the set of Panax notoginseng slice cross-sectional images into an unlabeled image set and a labeled image set. The frequency domain enhancement module is used to perform frequency domain hybrid enhancement processing on the cross-sectional images of Panax notoginseng slices in the unlabeled image set to obtain an enhanced image; The masking module is used to perform image slicing and random masking on the enhanced image to obtain visible image blocks and masked image blocks. The feature extraction module is used to input the visible image patch into the visual encoder in the asymmetric visual encoder-decoder network for feature extraction to obtain a semantic feature matrix; The image reconstruction module is used to input the semantic feature matrix into the lightweight decoder in the asymmetric visual coding and decoding network to reconstruct the image and obtain the reconstructed image. The pre-training module is used to construct a perceptual prior loss function based on the reconstructed image and the enhanced image, and to perform self-supervised pre-training on the asymmetric visual coding and decoding network based on the perceptual prior loss function. The prototype calculation module is used to freeze the network parameters of the visual encoder after the self-supervised pre-training is completed, and to calculate prototype vectors of different levels of categories using the labeled samples in the labeled image set. The grade prediction module is used to input the cross-sectional image of the Panax notoginseng slice to be tested into the visual encoder after freezing the network parameters to extract the feature vector of the sample to be tested, and determine the target grade of the cross-sectional image of the Panax notoginseng slice to be tested based on the distance between the feature vector of the sample to be tested and the prototype vector of each grade category.
9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of a visual feature-based method for grading Panax notoginseng slices as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the steps of a visual feature-based method for grading Panax notoginseng slices as described in any one of claims 1 to 7.
