Chinese medicinal material identification and classification method based on computer vision

Abstract

The invention discloses a traditional Chinese medicine identification and classification method based on computer vision, identification is carried out by a traditional Chinese medicine identification model obtained by training, and training samples of the traditional Chinese medicine identification model comprise original samples and expanded samples generated by adopting a generative adversarial network algorithm based on feature coupling; wherein the activation function of the generative adversarial network is subjected to feature coupling in a weighted sum and sine transformation mode, and the loss function of a discriminator in the generative adversarial network comprises a sample distance loss function which comprises an Euclidean distance between data points and a topological structure relation of the data points in a high-dimensional feature space. The method can capture the interaction and periodic relationship between complex features, and can improve the quality and diversity of the generated image.

Description

Technical Field

[0001] The present invention relates to the technical fields of machine learning, Chinese medicinal material identification and the like, and in particular to a Chinese medicinal material identification and classification method based on computer vision. Background Art

[0002] With the continuous development of the traditional Chinese medicine industry, especially in the field of circulation and quality control of traditional Chinese medicine, it is particularly important to accurately and quickly identify and classify traditional Chinese medicines. Traditional identification of traditional Chinese medicines relies on experienced experts to judge the type of medicine by observing its appearance, color, morphology and other characteristics with the naked eye. However, this method not only relies on human subjective judgment, but also the identification process is time-consuming and laborious, and is prone to misjudgment and omission, especially when there are many types of medicines and they are highly similar.

[0003] With the development of artificial intelligence, machine learning has been widely used in the field of Chinese medicine identification. For example, the invention patent with publication number CN118968508A proposes a method for identifying Chinese medicine powder based on image features, the invention patent with publication number CN111914902B proposes a method for identifying Chinese medicine and detecting surface defects based on deep neural networks, and the invention patent with publication number CN112101300A proposes a method, device and electronic device for identifying medicinal materials. However, in the existing Chinese medicinal material identification and classification tasks, many medicinal material identification systems have poor generalization ability in practical applications due to incomplete data collection or insufficient samples. Summary of the invention

[0004] One of the purposes of the present invention is to provide a method for identifying and classifying Chinese medicinal materials based on computer vision, so as to solve the problem that the existing Chinese medicinal materials data collection is incomplete or the samples are insufficient, resulting in poor generalization ability of the recognition model in practical applications.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A method for identifying and classifying Chinese medicinal materials based on computer vision, wherein the Chinese medicinal materials are identified by a trained Chinese medicinal materials identification model, wherein the training samples of the Chinese medicinal materials identification model include original samples and expanded samples generated by a generative adversarial network algorithm based on feature coupling;

[0007] The sample expansion steps include:

[0008] Initialize the generator and discriminator of the generative adversarial network, and set the activation function based on quantum state entanglement for the generator, expressed as:

[0009]

[0010] In the formula, represents the hidden state of the generator layer l, n c is the dimension of the Chinese medicinal materials image data, is the weight influencing factor of the i-th neuron in the l-th layer, and They represent the weights of the i-th neuron and the j-th neuron in the l-th layer of the generator, respectively. They represent the i-th input feature value and the j-th input feature value of the generator respectively. are the bias influencing factors of the i-th neuron and the j-th neuron in the l-th layer, respectively. is the periodic influence factor of the jth neuron in the lth layer, ⊙ represents the Hadamard product;

[0011] The generator and discriminator are trained by alternating optimization, and the trained generative adversarial network is used to generate virtual Chinese medicinal materials image data samples;

[0012] Among them, the objective function used in the discriminator training is expressed as:

[0013]

[0014] In the formula, is the loss function of the discriminator, D c () is the discriminator function, x c is the input data of the discriminator, G c (z c ) represents the medicinal material image data sample generated by the generator according to the input noise, L M is the sample distance loss function;

[0015] The sample distance loss function includes the Euclidean distance between data points and the topological structure relationship of data points in the high-dimensional feature space.

[0016] Preferably, the Chinese medicinal material identification model includes a cascaded feature extraction model, a feature dimension reduction model, and a classifier model.

[0017] Preferably, the feature extraction model adopts a neural network algorithm based on ecological optimization, and its training process includes:

[0018] Generate an initial set of neural network parameters. Each parameter set is regarded as a species in the ecosystem, and the initial ecological niche is randomly assigned, corresponding to the parameter initialization of the neural network.

[0019] In each iterative training, the loss value of each species under the current fully connected neural network is calculated as its fitness score, and the species is simulated to migrate and mutate parameters according to its fitness score until the preset stop iteration condition is met.

[0020] Preferably, the simulated species performs niche migration and variation of parameters according to their adaptability scores, including:

[0021] The adaptive score is used to migrate the ecological niche of the parameters, which is expressed as:

[0022]

[0023]

[0024] In the formula, and are the updated weights and biases of the neural network, γ p is the learning rate of the neural network; is the fitness score S of the εth species p，ε Gradients with respect to weight parameters; The gradient of the fitness score of the εth species with respect to the bias parameter;

[0025] The adaptability score is used to perform niche variation on parameters under the migrated niche, expressed as:

[0026]

[0027]

[0028] In the formula, Represents the weight of the neural network after migration and mutation; η p is the dynamic mutation rate based on adaptability; σ pse represents migration and mutation influencing factors; α p is the migration and variation adjustment coefficient; ΔE p，ε For variation energy.

[0029] Preferably, in the variation of the ecological niche, the adaptive energy regulation mechanism is used to dynamically adjust the variation energy of each species according to its historical performance, which is expressed as:

[0030]

[0031] In the formula, Ω p,ε is the energy regulation factor of the εth species; E p，max is the maximum energy input; is the parameter orthogonal term;

[0032] The calculation method of the energy adjustment factor is expressed as:

[0033]

[0034] In the formula, Ω p，ε is the energy regulation factor of the εth species; Hp，ε represents the historical fitness of the εth species; ζ p is the sensitivity parameter of energy regulation; is the average of the accumulated fitness scores before this iteration.

[0035] Preferably, the feature dimensionality reduction model adopts an autoencoder algorithm based on reconstruction loss, which includes an encoder and a decoder. The encoder is used to compress the input data features into a low-dimensional representation, and the decoder is used to reconstruct high-dimensional data features from the low-dimensional representation using an inverse reconstruction strategy.

[0036] Preferably, during the training process of the autoencoder algorithm based on reconstruction loss, the inverse reconstruction loss of the autoencoder is set to:

[0037]

[0038] Where, L r is the inverse reconstruction loss function; Ntr is the number of samples input in the current batch; a is the sample index of the current batch input; || || represents the L2 norm; λ r is the regularization parameter; D KL (P r ||Q r ) is the KL divergence.

[0039] Preferably, the classifier model adopts a high-order neural network model, and the model is trained by constructing a loss function with multiple constraints.

[0040] Preferably, the loss function of the multiple constraints is expressed as:

[0041] L u =α uas L cls (Y u , Y true )+α u L cons (X u )+β u L balance (Y u )+γ u L smooth (W u )

[0042] Where, L cls () is the classification loss function; Y true is the true label of the sample input to the high-order neural network; Y u is the sample category output by the high-order neural network; L cons () is the feature consistency loss; x uThe feature vector of the image data of traditional Chinese medicine after the feature dimension reduction is input into the high-order neural network; L balance () is the category balance loss; L smooth () is the network smoothness loss.

[0043] Preferably, in each round of training, the high-order neural network model uses the gradient information of the first-order derivative to perform gradient descent optimization, uses the second-order Taylor expansion to correct the gradient direction, and combines the dynamic correction mechanism of the local gradient to correct the gradient of each layer of the high-order neural network to update the weights of the high-order neural network.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1) In the task of identifying and classifying traditional Chinese medicines, a generative adversarial network is used to generate virtual image data, which enhances the diversity of the data set and avoids the problem of model overfitting caused by insufficient training data. The generative adversarial network uses a feature coupling mechanism to capture the interaction and periodicity between complex features, which can improve the quality and diversity of generated images.

[0046] 2) In the image feature extraction stage, a neural network algorithm based on ecological optimization is used to simulate the process of biological adaptation in nature to optimize the weights and biases of the neural network, avoiding the gradient vanishing, explosion and local optimal problems that may exist in traditional neural networks. Through adaptive optimization, the neural network can better adapt to the complex image features of traditional Chinese medicines, thereby improving the accuracy and stability of feature extraction;

[0047] 3) Before using the extracted high-dimensional image features for classification and recognition, the dimension of the data is reduced through the dimensionality reduction model, which reduces redundant information and improves computational efficiency. This is also conducive to data visualization and understanding. On this basis, the inverse reconstruction strategy is used to limit the dimensionality reduction loss during the feature dimensionality reduction process, so as to minimize the dependency between different features without losing important feature information, thereby improving the quality of feature representation after dimensionality reduction.

[0048] 4) A high-order neural network based on multiple constraints is used to classify medicinal materials images. During the training process, multiple constraints are combined for collaborative optimization to improve the performance of the model in multiple dimensions, avoiding the problems of overfitting and uneven performance in traditional methods. At the same time, the second-order Taylor expansion is used to correct the gradient direction, making the network update more accurate, which helps to improve training efficiency and classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flowchart of a method for identifying and classifying Chinese medicinal materials based on computer vision is provided in an embodiment.

[0050] Figure 2This is the training flow chart of the feature dimensionality reduction model. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0052] Reference Figure 1 As shown, this embodiment provides a method for identifying and classifying traditional Chinese medicinal materials based on computer vision, which specifically includes 6 steps.

[0053] S1, data collection and annotation.

[0054] In this embodiment, professionally certified medicinal material collectors collect images of various types of Chinese medicinal materials in different geographical regions, climatic conditions and planting environments. The collection of Chinese medicinal material image data covers different varieties, maturity, cutting methods, pests and diseases, etc., to ensure the comprehensiveness and representativeness of the data; the image data is mainly taken by a high-resolution digital camera. When collecting the image data of each medicinal material, the lighting conditions, background interference-free, and shooting angles are standardized to minimize the impact of external factors on the data quality. At the same time, the data collectors take multi-angle and multi-scale photos according to the different characteristics of the medicinal materials (such as color, morphology, texture, etc.), to ensure that the images of each medicinal material variety have sufficient diversity, and each image has standard image label information, which is convenient for subsequent annotation and processing. The collected data are all two-dimensional image data, and the storage format is an image file format (such as JPEG, PNG or TIFF). The resolution of each image is set to 2048×2048 pixels to ensure that the details in the image are fully presented; when collecting data, ensure that the quality factors such as color, lighting, and clarity of each image remain consistent between different medicinal materials for subsequent model training.

[0055] S2, Chinese medicinal materials image data expansion.

[0056] In the task of classifying images of traditional Chinese medicines of the present invention, the acquisition, labeling and preprocessing of training data are time-consuming and labor-intensive, and insufficient training samples easily lead to poor generalization ability of the model, while affecting the accuracy of the model. This embodiment uses a generative adversarial network algorithm based on feature coupling to generate samples, thereby achieving data expansion.

[0057] The generative adversarial network consists of two parts: a generator and a discriminator; the generator is responsible for generating virtual data similar to real Chinese medicinal materials image data, and the discriminator is responsible for distinguishing the generated Chinese medicinal materials image data from the real Chinese medicinal materials image data.

[0058] This embodiment uses the real collected Chinese medicinal material images as original samples, trains the generative adversarial network to obtain an image expansion model to expand each category of Chinese medicinal material images, and the specific process is as follows:

[0059] S201, initialize the generator and discriminator of the generative adversarial network.

[0060] In S201, the generator accepts a random noise z c As input, this latent vector is represented as:

[0061] z c ~U(-1,1)

[0062] In the formula, z c Represents the input of the generator, a random vector that obeys a uniform distribution; U(-1, 1) represents uniform sampling in the interval [-1, 1].

[0063] S202, simulating the idea of ​​quantum state entanglement, setting the activation function of the generative adversarial network to couple features through weighted sum and sine transform, expressed as:

[0064]

[0065]

[0066]

[0067] In the formula, represents the hidden state of the generator layer l, represents the weight of the i-th neuron in the l-th layer of the generator, represents the i-th input feature value of the generator, is the bias of the generator layer l, Q() is the activation function of quantum state entanglement; n c is the dimension of the Chinese medicinal material image data; Q1 is the first quantum state entanglement calculation item; Q2 is the second quantum state entanglement calculation item; is the weight influencing factor of the i-th neuron in the l-th layer in the first quantum state entanglement calculation term, and is a training parameter that controls the weight of the influence of the quantum state on the feature; is the bias influence factor of the i-th neuron in the l-th layer under the first quantum state entanglement calculation, which is a training parameter; is the periodic influence factor of the jth neuron in the lth layer in the second quantum state entanglement calculation term, and is a training parameter that controls the periodic adjustment of the feature; is the bias influencing factor of the jth neuron in the lth layer in the second quantum state entanglement calculation term, and is a training parameter; represents the j-th input feature value of the generator; represents the weight of the j-th neuron in the l-th layer of the generator; ⊙ represents the Hadamard product, that is, element-by-element multiplication.

[0068] In S202, feature coupling can capture the interactions and periodic relationships between complex features through the combination of weighted sum and sine transform, thereby enhancing the ability of the generator in generating Chinese medicinal materials image data and reducing the training difficulty caused by the inconsistent distribution of Chinese medicinal materials image data.

[0069] S203, in each training iteration, the generated random noise is input into the generator, and a virtual Chinese medicinal material image data sample is generated through multi-layer transformation. The goal of these virtual samples is to be as close as possible to the statistical characteristics of the real Chinese medicinal material image data set, which is expressed as:

[0070]

[0071] In the formula, F G () is the output function of the generator; is the lth component of the latent vector; is the weight matrix of the generator layer l, is the bias of the generator layer l, G c () is the generator function; n cec is the number of layers of the generator.

[0072] This process gradually transforms the input random noise z through the network layers of the generator c , so that the generated Chinese medicinal materials image data sample G c (z c ) can be as close as possible to the characteristics of real Chinese medicinal materials image data samples.

[0073] S204, optimizing the discriminator using the loss function of the discriminator.

[0074] In the present invention, the discriminator receives two types of input Chinese medicinal material image data in each iteration, one type comes from a real Chinese medicinal material image data set, and the other type comes from virtual Chinese medicinal material image data generated by a generator. The discriminator classifies each input Chinese medicinal material image data through a neural network and outputs a probability value, indicating the probability that the input Chinese medicinal material image data comes from the real Chinese medicinal material image data set. The goal of the discriminator is to distinguish the difference between the two types of Chinese medicinal material image data as accurately as possible, thereby guiding the generator to learn higher quality virtual Chinese medicinal material image data.

[0075] The output of the discriminator is expressed as:

[0076]

[0077] Where D c () is the discriminator function; x c is the input data of the discriminator; Sig() is the Sigmoid activation function; m cec is the number of layers of the discriminator; is the weight matrix of the discriminator layer l; is the input feature vector of the discriminator layer l; is the bias of the discriminator layer l.

[0078] For generated samples, the output of the discriminator should be close to 0, while for real samples, it should be close to 1. Therefore, the loss function of the discriminator uses the cross entropy loss function to measure the classification effect of the discriminator. The calculation method is expressed as:

[0079]

[0080]

[0081] In the formula, is the loss function of the discriminator; L M is the sample distance loss function, || || M is the distance calculation norm based on the symmetric positive definite matrix; Nor() is the normalization function; G c (z c ) represents the TCM image data sample generated by the generator according to the input noise; x real These are real Chinese medicinal material image data samples.

[0082] The calculation method of the distance calculation norm based on the symmetric positive definite matrix is ​​expressed as:

[0083]

[0084] Where M is a symmetric positive definite matrix whose elements are obtained through learning and represent the importance and relevance of different dimensions in the feature space; T is the transpose.

[0085] In the loss function of this discriminator, the sample distance loss function not only considers the Euclidean distance between data points, but also considers the topological structure relationship of data points in the high-dimensional feature space, so as to better capture and maintain the intrinsic structural characteristics of the data, and can effectively enhance the quality of generated data in the data expansion process.

[0086] S205, optimizing the generator using the generator's loss function.

[0087] In the present invention, the generator optimizes the quality of the generated Chinese medicinal material image data by minimizing the misclassification probability of the discriminator. The loss function of the generator is fed back through the output of the discriminator, and the goal is to make the discriminator unable to distinguish between the real Chinese medicinal material image data and the generated Chinese medicinal material image data. In this embodiment, the loss function of the generator is expressed as:

[0088]

[0089] In the formula, is the loss function of the generator, and its goal is to minimize the probability that the generated Chinese medicinal material image data is judged as real Chinese medicinal material image data.

[0090] S206, training the generative adversarial network by alternately optimizing the generator and the discriminator.

[0091] In each training of S206, the parameters of the discriminator are first updated to optimize its discrimination ability, and then the parameters of the generator are updated to optimize the quality of the virtual Chinese medicinal material image data generated by it. Through this alternating optimization, the generator gradually learns the characteristic distribution of real Chinese medicinal material image data, and the discriminator continuously improves its discrimination ability for virtual Chinese medicinal material image data.

[0092] In the above adversarial training process, the weight update method of the discriminator is expressed as:

[0093]

[0094] And, the weight update method of the generator is expressed as:

[0095]

[0096] Where ← is the parameter update operation; η c is the learning rate of the generative adversarial network; is the gradient of the discriminator's loss function with respect to the weight; represents the weight of the discriminator; represents the weight of the generator; is the gradient of the generator's loss function with respect to the weights.

[0097] S207, repeating the above steps until a preset stop iteration condition is met, which means that the training of the generative adversarial network model is completed, and the model after the training is completed is the image expansion model.

[0098] In one embodiment, the preset condition for stopping iteration is reaching a preset maximum number of iterations. Preferably, the preset maximum number of iterations is set to 1000 times.

[0099] In this embodiment, the image expansion model is used to increase the number of Chinese medicinal materials image data samples. Assuming that the originally collected Chinese medicinal materials image data samples are 800, and the Chinese medicinal materials image data samples expanded and generated by the image expansion model are 200, the expanded Chinese medicinal materials image data set contains 1000 samples.

[0100] In the present invention, the Chinese medicinal material recognition model includes a feature extraction model, a feature dimension reduction model, and a classifier model that are connected in sequence. During the training process of the Chinese medicinal material recognition model, the above three models are trained one by one to achieve the training task of the Chinese medicinal material recognition model.

[0101] S3, image feature extraction model training.

[0102] The present invention adopts a 5-layer fully connected neural network as the network architecture of the feature extraction model, inputs the expanded Chinese medicinal material image data into the feature extraction model, and trains the Chinese medicinal material image feature extraction model. In the prior art, some schemes use neural networks to extract features of Chinese medicinal material images. In some neural network structures, problems such as gradient vanishing, gradient explosion, or falling into a local optimal solution may be encountered, affecting the stability of training and the performance of the model.

[0103] The present invention adopts a neural network algorithm based on ecological optimization as a model for extracting features from images of Chinese medicinal materials. Inspired by the concept of ecological niche of ecosystems in nature, each species has its own specific ecological niche, which specifically refers to the position a species occupies in an ecosystem, the functions it performs, and the interaction with the environment; the survival and prosperity of each species depends on whether it can effectively adapt to changes in its ecological niche. This principle is applied to the optimization of weights and biases of neural networks, simulating the process of organisms constantly adapting to environmental changes in their ecological niches to improve their survival rates. During the training process of the neural network, each set of parameters (weights and biases) is regarded as a species in a high-dimensional parameter space, and each species attempts to optimize its ecological niche in the problem space.

[0104] Specifically, the training process of the fully connected neural network algorithm based on ecological optimization is as follows:

[0105] S301, generating an initial set of network parameters, each parameter set is regarded as a species in the ecosystem, and an initial ecological niche is randomly assigned, corresponding to the initialization of the network parameters.

[0106] In S301, the parameter set includes all parameters of the fully connected neural network. For example, if it is a 5-layer fully connected neural network with 100 neurons in each layer, then the parameter set is a 3*100 matrix, which corresponds to one species. During ecological optimization and update, multiple species will be randomly generated, corresponding to multiple parameter sets of fully connected neural networks.

[0107] In one embodiment, the initialization method is expressed as:

[0108]

[0109]

[0110] In the formula, and They represent the initial weight and initial bias of the εth species, that is, the initial values ​​of the weight and bias of the corresponding neural network; W p，ε and b p，ε Respectively represent the weight and bias of the εth species; σ p is the initialization standard deviation of the neural network, which is used to control the initial distribution range of the parameters; is a normal distribution with a mean of 0 and a variance of 1; represents normal distribution; ~ represents obedience to a specific distribution.

[0111] Each set of parameters is randomly assigned to its ecological niche in the ecosystem, expressed as:

[0112] L p，ε ~rand(0,1)

[0113] L p，ε represents the ecological niche of the εth species, that is, the ecological niche corresponding to the neural network parameters; rand(0, 1) represents a random value between 0 and 1.

[0114] S302, evaluate each species and calculate its loss function value under the current neural network architecture as its adaptability score. The adaptability evaluation method is expressed as:

[0115] S p，ε =exp(-λ p ·Loss(W p，ε , b p，ε ))

[0116] In the formula, S p，ε is the fitness score of the εth species; λ p is the fitness score hyperparameter, which is used to adjust the loss function to the fitness score S p，ε The influence of Loss(W p，ε , bp，ε ) represents the usage weight W p，ε and bias b p，ε The calculated network loss, Loss() represents the loss function of the neural network. Preferably, λ p Set to 5.

[0117] In one embodiment, considering the model complexity and prediction error, the calculation method of the loss function of the neural network is expressed as:

[0118] Loss(W p，ε , b p，ε )=ρ p MSE+(1-ρ p )·Com(W p，ε , L p，ε )

[0119] In the formula, ρ p is the weight coefficient for adjusting the prediction error and model complexity; MSE is the mean square error of the neural network; Com() is the model complexity function. Preferably, ρ p Set to 0.5.

[0120] The loss function Loss(W p，ε , b p，ε ), the calculation methods of the mean square error and model complexity function of the neural network are expressed as:

[0121]

[0122]

[0123] In the formula, is the model prediction value obtained by the preset Softmax function of the fully connected neural network for the feature vector of the Chinese medicinal material image output by the jsth sample; y js is the true label of the jsth sample; n MSE is the number of samples input to the fully connected neural network in the current batch; W p,εk and represents the kth parameter in the weight parameter combination of the εth species, L p,ε represents the ecological niche of the εth species.

[0124] S303, simulate the migration of species parameters according to their adaptability. Specifically, simulate the niche competition strategy, that is, the species compete with each other. Species with low loss function (high adaptability) will occupy a better niche, while species with low adaptability may be forced to migrate to a poorer niche or be eliminated, which is expressed as:

[0125]

[0126]

[0127] In the formula, and are the updated weights and biases of the fully connected neural network, ΔW p，ε and Δb p，ε Update increments for neural network weights and biases.

[0128] In one embodiment, the update increment calculation method of the weights and biases of the fully connected neural network is expressed as:

[0129]

[0130]

[0131] In the formula, γ p is the learning rate of the neural network; is the gradient of the fitness score of the εth species with respect to the weight parameter; The gradient of the fitness score of the εth species with respect to the bias parameter. Preferably, γ p Set to 0.01.

[0132] S304, simulates the variation of species parameters according to their adaptability, and uses the mutated and migrated parameters to perform ecological optimization of the fully connected neural network. The calculation method is expressed as:

[0133]

[0134] In the formula, Represents the weight of the fully connected neural network after migration and mutation, which is used as the weight of the neural network for the next iteration; is the weight after migration; η p is a dynamic mutation rate based on adaptability, which reflects the inverse of species adaptability. The lower the adaptability, the higher the mutation rate; σ pse Represents the migration and mutation impact factor, which is set to 0.1 here.

[0135] In S304, the parameters of species with high adaptability have a small mutation rate and remain stable; species with low adaptability try to make a large mutation to seek improvement of ecological niche. In one embodiment, the mutation rate calculation formula is:

[0136]

[0137] In the formula, α p is the migration and variation adjustment coefficient, which is set to 0.3 here; ΔE p,ε For variation energy.

[0138] In one embodiment, an adaptive energy adjustment mechanism is used to dynamically adjust the variation energy of each species (parameter set) according to its historical performance. By adjusting the energy input at each variation, species with lower adaptability are given greater adjustment opportunities, while species with higher adaptability remain in a relatively stable state, thereby increasing the algorithm's global search capability and avoiding local optimality. On this basis, an environment-aware parameter orthogonalization adjustment mechanism is used to orthogonalize parameters to ensure that appropriate diversity is maintained between different parameter sets (species), thereby avoiding premature convergence to local optimality and enhancing global search capabilities. Specifically, the calculation of variation energy using the adaptive energy adjustment mechanism is expressed as:

[0139]

[0140] In the formula, Ω p，ε is the energy regulation factor of the εth species; E p，max is the maximum energy input, which is the preset value; is the parameter orthogonal term;

[0141] In the calculation formula of the variation energy, the calculation method of the energy adjustment factor is expressed as:

[0142]

[0143] In the formula, Ω p，ε is the energy regulation factor of the εth species; H p，ε represents the historical adaptability of the εth species, that is, the accumulated adaptability score value before this iteration; ζ p is the sensitivity parameter of energy regulation, which is the preset value; is the average of the accumulated fitness scores before this iteration.

[0144] In the calculation formula of the variation energy, the calculation method of parameter orthogonalization using the environment-aware parameter orthogonalization adjustment mechanism is expressed as:

[0145]

[0146]

[0147] In the formula, ∈ p is the orthogonally adjusted learning rate; Ortho p，ε represents the orthogonalized energy of the εth species; <, > represents the vector dot product; || || represents the L2 norm; W p，ω represents the weight and bias of the ω-th species; here, E p,max Set to 0.4, ζ p Set to 0.1, ∈ p Set to 0.01.

[0148] In the present invention, by utilizing orthogonal adjustment of parameters in the process of adjusting the ecological niche, it is ensured that the parameter updating process not only responds to the current adaptability but also maintains the independence between parameters, thereby promoting the algorithm to explore unknown and potentially better solution space.

[0149] S304, repeat the above steps until the preset stop iteration condition is met, which means that the model training is completed.

[0150] In one embodiment, the preset condition for stopping iteration is reaching a preset maximum number of iterations. Preferably, the preset maximum number of iterations is set to 1000 times.

[0151] S4, train the feature dimensionality reduction model.

[0152] In this embodiment, an autoencoder algorithm based on reconstruction loss is used as a feature dimensionality reduction model. The autoencoder model includes an encoder and a decoder. The encoder is responsible for compressing the features of the Chinese medicinal materials image after feature extraction into a low-dimensional representation, and the decoder reconstructs the data features from this low-dimensional representation. In order to enhance the adaptability of the model's feature dimensionality reduction, an inverse reconstruction strategy is used to process the input features, so that the dependency between different features is minimized, thereby improving the quality of encoding and the accuracy of decoding.

[0153] Reference Figure 2 As shown, specifically, the training process of the autoencoder algorithm based on reconstruction loss is as follows:

[0154] S401, initializing the parameters of the autoencoder algorithm, including the weight and bias parameters of the autoencoder.

[0155] In one embodiment, the initialization method is expressed as:

[0156]

[0157] b r，init =0

[0158] Where W r，init is the initial weight matrix of the autoencoder; b r，init is the initial bias of the autoencoder; n r,in and n r，out Represent the number of neurons in the input and output layers respectively; U(-1, 1) represents a random number uniformly distributed in the interval [-1, 1].

[0159] S402, the input feature of the Chinese medicinal material image after feature extraction is converted into a low-dimensional space representation through the encoder, and the low-dimensional representation is reconstructed into an output with the same dimension as the original data through the decoder. In the forward propagation process, the output calculation method of the encoder is expressed as:

[0160] z r =Sig(W r,enc ·x r +b r.enc )

[0161] And, the reconstructed output of the decoder is:

[0162]

[0163] In the formula, z r is the low-dimensional feature representation of the code; Sig() is the Sigmoid activation function; W r，enc and W r，dec are the weights of the encoder and decoder respectively; b r,enc and b r,dec are the biases of the encoder and decoder respectively; x r It is the image features of Chinese medicinal materials extracted by the feature extraction model; is the output of the reconstruction.

[0164] S403, calculate the inverse reconstruction loss of the autoencoder, and define the calculation method of the inverse reconstruction loss function as:

[0165]

[0166] Where, L r is the inverse reconstruction loss function; Ntr is the number of samples input in the current batch; a is the sample index of the current batch input; || || represents the L2 norm; λ r is the regularization parameter; D KL (P r ||Q r ) is the KL divergence.

[0167] The KL divergence is used to measure the original data distribution P r and reconstruct the data distribution Q r The difference between them is used to ensure that the key features of the data are not lost during the dimensionality reduction process. The calculation method is expressed as:

[0168]

[0169] In the formula, p r,sl and q r，sl are the probability distributions of the original data and the reconstructed data in the slth dimension respectively, and Mtr is the feature dimension before feature dimensionality reduction.

[0170] S404, based on the gradient calculated by the loss function, the weights and biases of the encoder and decoder are updated using the gradient descent algorithm. The updating method is expressed as:

[0171]

[0172]

[0173]

[0174]

[0175] Where η c is the learning rate; is the updated encoder weight; is the updated encoder bias; is the updated decoder weight; is the updated decoder bias.

[0176] With gradient For example, the calculation method is expressed as:

[0177]

[0178] In the formula, represents the partial derivative of the loss function with respect to the reconstructed output, is the partial derivative during decoding, is the partial derivative during encoding.

[0179] S405, repeat the above steps until the preset stop iteration condition is met, which means that the feature dimensionality reduction model training is completed.

[0180] In one embodiment, the preset stop iteration condition is reaching the preset maximum number of iterations, at which time multiple optimized parameter sets are obtained, and the parameter set with the smallest model loss value is selected as the parameter set after the fully connected neural network training is completed. Preferably, the preset maximum number of iterations is set to 1200 times.

[0181] S5, classifier model training.

[0182] In the present invention, a high-order neural network model is used as the network architecture of the classification model, and the image data of Chinese medicinal materials after feature dimensionality reduction is input into the classifier model to train the classifier model. Since traditional high-order neural network training usually only focuses on the optimization of a single goal and ignores the synergy between different features, the present invention adopts a multi-constraint collaborative optimization mechanism during the training process, taking into account the interaction between multiple constraints to ensure good performance of the model in multiple dimensions.

[0183] Specifically, the training process of the high-order neural network based on multiple constraints is as follows:

[0184] S501, initialize the parameters of the high-order neural network, and set the weight of the high-order neural network to W u, the bias term of the high-order neural network is b u .

[0185] In one embodiment, the initialization method is random initialization, and the initial parameters of the high-order neural network obey a normal distribution with a mean of 0 and a variance of a unit matrix.

[0186] S502, inputting the image data of Chinese medicinal materials after feature dimension reduction into a high-order neural network for forward propagation, and the forward propagation method is expressed as:

[0187]

[0188]

[0189] In the formula, is the linear transformation result of the f1 layer of the high-order neural network, is the activation output of the f1th layer of the high-order neural network; and are the weight matrix and bias term of the flth layer of the high-order neural network respectively; is the output of the f1-1 layer of the high-order neural network; Re() is the ReLU activation function.

[0190] S503, in each round of training, high-order features are modeled through the Taylor expansion strategy. First, basic gradient descent optimization is performed through the gradient information of the first-order derivative, and then the second-order Taylor expansion is used to correct the gradient direction to more accurately adjust the network weights and improve the learning efficiency and accuracy of high-order neural networks. On this basis, the dynamic correction mechanism of local gradients is used to adaptively adjust the gradient to adapt to dynamic changes during training.

[0191] In S503, specifically, the gradient correction formula of the high-order neural network is expressed as:

[0192]

[0193]

[0194]

[0195] In the formula, ΔW n is the gradient corrected by the second-order Taylor expansion; is the second-order gradient of the high-order neural network; u is the regularization coefficient; I is the unit matrix; is the first-order gradient of a high-order neural network; is the adaptive local gradient correction term; L u is the loss function of the high-order neural network; is the symbol of partial derivative.

[0196] In this embodiment, a dynamic correction mechanism of local gradient is adopted to calculate the corrected gradient of each layer. The corrected gradient depends on the local changes of the weight matrix and activation value of each layer. The problem of gradient disappearance or explosion is solved by adaptively adjusting the gradient direction.

[0197] In one embodiment, for the kth layer of a high-order neural network, its gradient is the partial derivative of the loss function with respect to the weight matrix of this layer, expressed as:

[0198]

[0199] In the formula, is the activation value of the skth layer of the high-order neural network, is the gradient of the loss function with respect to the activation value, is the gradient of the activation value with respect to the weight; is the weight matrix of the kth layer of the high-order neural network; nus is the number of layers of the high-order neural network.

[0200] In the dynamic adjustment mechanism of the local gradient, the gradient of the current layer is adjusted according to the gradient information of the previous layer, so as to correct the error in the gradient propagation process. The calculation method is expressed as:

[0201]

[0202] In the formula, λ ucc is the local gradient correction coefficient, It is the second-order derivative of the loss function of the high-order neural network with respect to the activation values ​​of the skth layer and the sk-1th layer, representing the local changes in the gradient propagation process. is the activation value of the sk-1th layer of the high-order neural network. Here, λ ucc Set to 0.2.

[0203] S504, setting the calculation method of the loss function of the high-order neural network to be expressed as:

[0204] L u =α uas L cls (Y u , Y true )+α u L cons (X u )+β u L balance (Y u )+γ u L smooth (W u )

[0205] Where, Lcls () is the classification loss function, which measures the difference between the classifier output and the true label; Y true is the true label of the sample input to the high-order neural network; Y u is the sample category output by the high-order neural network; L cons () is the feature consistency loss, which ensures the consistency of input features; X u The feature vector of the image data of traditional Chinese medicine after the feature dimension reduction is input into the high-order neural network; L balance () is the category balance loss to ensure that the categories are evenly distributed; L smooth () is the network smoothness loss to prevent overfitting; α u is the first weight factor of the high-order neural network; β u is the second weight factor of the high-order neural network; γ u is the third weight factor of the high-order neural network; α uas is the excitation parameter.

[0206] In the present invention, the feature consistency loss ensures that the data distribution in the feature space remains consistent between different training rounds. Specifically, the feature consistency loss can be expressed as the difference between the feature distributions of two different training rounds, and the calculation method is expressed as:

[0207]

[0208] In the formula, and are the feature matrices for the t-th and t-1-th training respectively; || || represents the L2 norm, which is used to measure the difference between the two feature matrices.

[0209] In the present invention, by setting the category balance loss, it is possible to prevent the situation in which the model tends to predict categories with higher frequencies due to the low frequency of some categories in multi-category classification tasks. The category balance loss used in the present invention aims to balance the prediction probabilities of various categories, and the calculation method is expressed as:

[0210]

[0211] In the formula, p v is the true probability distribution of the vth class, is the probability of the high-order neural network predicting the vth category, where V is the number of categories.

[0212] In the present invention, in order to prevent the model from overfitting, a smoothness constraint is applied to the weight of the network to ensure its stability during the entire training process. The calculation method is expressed as:

[0213]

[0214] In the formula, and are the weight matrices for the t-th and t-1-th training respectively.

[0215] During the training process of high-order neural networks, the present invention not only considers the error minimization goal, but also considers the constraints on feature distribution, category balance and network stability. Through the coordinated optimization of these multi-dimensional constraints, overfitting can be avoided during the training process while ensuring the balanced performance of the model in various dimensions.

[0216] S505, in each training iteration of the network, the excitation function parameters of each layer are dynamically adjusted through the adaptive excitation mechanism, so that the network can adaptively adjust the learning strategy at different training stages. The excitation function parameters will be automatically updated according to the performance feedback of the current model to ensure the efficiency of the training process. The calculation method is expressed as:

[0217]

[0218] In the formula, is the excitation parameter at the tth iteration; is the excitation parameter at the t-1th iteration; is the learning rate adjustment factor at the tth iteration; It is the gradient of the loss function of the high-order neural network with respect to the excitation parameter, which represents the improvement of model performance by optimizing the excitation coefficient.

[0219] S506, repeat the above steps until the preset stop iteration condition is met, which means that the model training is completed.

[0220] In one embodiment, the preset condition for stopping iteration is reaching a preset maximum number of iterations. Preferably, the preset maximum number of iterations is set to 1000 times.

[0221] S6, Identification and classification of Chinese medicinal materials.

[0222] The trained Chinese medicinal material recognition model is used to process new Chinese medicinal material image samples, and finally the identification and classification tasks of medicinal materials are completed. In one embodiment, the collected original Chinese medicinal material image data is input into the trained feature extraction and feature dimension reduction model for feature processing, and then the processed features are input into the classifier model for classification, thereby obtaining the classification results. In this embodiment, the classification categories are different types of Chinese medicinal materials.

[0223] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying and classifying Chinese medicinal materials based on computer vision, characterized in that: Identify using a trained Chinese herbal medicine recognition model, wherein the training samples of the Chinese herbal medicine recognition model include original samples and expanded samples generated using a generative adversarial network algorithm based on feature coupling; The sample expansion steps include: Initialize the generator and discriminator of the generative adversarial network, and set the activation function based on quantum state entanglement for the generator, expressed as: In the formula represents the hidden state of the first layer of the generator, n c is the dimension of the Chinese medicinal material image data is the weight influencing factor of the i-th neuron in the l-th layer, and They represent the weights of the i-th neuron and the j-th neuron in the l-th layer of the generator, respectively. They represent the i-th input feature value and the j-th input feature value of the generator respectively. are the bias influencing factors of the i-th neuron and the j-th neuron in the l-th layer, respectively. is the periodic influence factor of the jth neuron in the lth layer, ⊙ represents the Hadamard product; The generator and discriminator are trained by alternating optimization, and the trained generative adversarial network is used to generate virtual Chinese medicinal materials image data samples; Among them, the objective function used in the discriminator training is expressed as: In the formula, is the loss function of the discriminator, D c () is the discriminator function, x c is the input data of the discriminator, G c (z c ) represents the medicinal material image data sample generated by the generator according to the input noise, L M is the sample distance loss function; The sample distance loss function includes the Euclidean distance between data points and the topological structure relationship of data points in the high-dimensional feature space.

2. A method for identifying and classifying Chinese medicinal materials based on computer vision as claimed in claim 1, characterized in that: The traditional Chinese medicine identification model includes a cascade feature extraction model, a feature dimension reduction model, and a classifier model.

3. A method for identifying and classifying Chinese medicinal materials based on computer vision as claimed in claim 2, characterized in that: The feature extraction model adopts a neural network algorithm based on ecological optimization, and its training process includes: Generate an initial set of neural network parameters. Each parameter set is regarded as a species in the ecosystem, and the initial ecological niche is randomly assigned, corresponding to the parameter initialization of the neural network. In each iterative training, the loss value of each species under the current fully connected neural network is calculated as its fitness score, and the species is simulated to migrate and mutate parameters according to its fitness score until the preset stop iteration condition is met.

4. A method for identifying and classifying Chinese medicinal materials based on computer vision as claimed in claim 3, characterized in that: The simulated species undergo niche migration and variation of parameters according to their fitness scores, including: The adaptive score is used to migrate the ecological niche of the parameters, which is expressed as: In the formula, and are the updated weights and biases of the neural network, γ p is the learning rate of the neural network; is the fitness score S of the εth species p，ε Gradients with respect to weight parameters; The gradient of the fitness score of the εth species with respect to the bias parameter; The adaptability score is used to perform niche variation on parameters under the migrated niche, expressed as: In the formula, Represents the weight of the neural network after migration and mutation; η p is the dynamic mutation rate based on adaptability; σ pse represents migration and mutation influencing factors; α p is the migration and variation adjustment coefficient; ΔE p，ε For variation energy.

5. A method for identifying and classifying Chinese medicinal materials based on computer vision as claimed in claim 4, characterized in that: In the variation of the ecological niche, the adaptive energy regulation mechanism is used to dynamically adjust the variation energy of each species according to its historical performance, which is expressed as: In the formula, Ω p，ε is the energy regulation factor of the εth species; E p，max is the maximum energy input; is the parameter orthogonal term; The calculation method of the energy adjustment factor is expressed as: In the formula, Ω p，ε is the energy regulation factor of the εth species; H p，ε represents the historical fitness of the εth species; ζ p is the sensitivity parameter of energy regulation; is the average of the accumulated fitness scores before this iteration.

6. A method for identifying and classifying Chinese medicinal materials based on computer vision as claimed in claim 2, characterized in that: The feature dimensionality reduction model adopts an autoencoder algorithm based on reconstruction loss, which includes an encoder and a decoder. The encoder is used to compress input data features into a low-dimensional representation, and the decoder is used to reconstruct high-dimensional data features from the low-dimensional representation using an inverse reconstruction strategy.

7. A method for identifying and classifying Chinese medicinal materials based on computer vision as claimed in claim 6, characterized in that: During the training process of the autoencoder algorithm based on reconstruction loss, the inverse reconstruction loss of the autoencoder is set as: Where, L r is the inverse reconstruction loss function; Ntr is the number of samples input in the current batch; a is the sample index of the current batch input; || || represents the L2 norm; λ r is the regularization parameter; D KL (P r ||Q r ) is the KL divergence.

8. A method for identifying and classifying Chinese medicinal materials based on computer vision as claimed in claim 2, characterized in that: The classifier model adopts a high-order neural network model, and trains the model by constructing a loss function with multiple constraints.

9. The important medicinal material identification method based on computer vision as claimed in claim 8 is characterized in that: The loss function of the multi-constraint conditions is expressed as: L u =a uas L cls (Y u ,Y true )+a u L cons (X u )+b u L balance (Y u )+c u L smooth (W u ) Where, L cls () is the classification loss function; Y true is the true label of the sample input to the high-order neural network; Y u is the sample category output by the high-order neural network; L cons () is the feature consistency loss; X u The feature vector of the image data of traditional Chinese medicine after the feature dimension reduction is input into the high-order neural network; L balance () is the category balance loss; L smooth () is the network smoothness loss.

10. The method for identifying important medicinal materials based on computer vision as claimed in claim 8, characterized in that: In each round of training, the high-order neural network model uses the gradient information of the first-order derivative to perform gradient descent optimization, uses the second-order Taylor expansion to correct the gradient direction, and combines the dynamic correction mechanism of the local gradient to correct the gradient of each layer of the high-order neural network to update the weight of the high-order neural network.

Images