Chinese medicinal material positive sample-based Chinese medicinal material pest and disease identification method, device and equipment

Abstract

The invention relates to a Chinese herbal medicine positive sample-based Chinese herbal medicine pest and disease identification method, device and equipment, and the method comprises the steps: obtaining a normalized feature vector through balance augmentation and feature extraction based on a positive sample Chinese herbal medicine plant image, generating a feature vector database, and identifying the Chinese herbal medicine plant image based on a multi-modal large model. The method comprises the following steps: obtaining a purified foreground region through foreground extraction, carrying out feature extraction and normalization, generating a traditional Chinese medicinal material plant feature vector, judging the traditional Chinese medicinal material plant feature vector through a similarity threshold and a highest similarity category occurrence frequency threshold according to a feature vector database, obtaining and verifying a negative sample traditional Chinese medicinal material plant, and realizing closed-loop iteration. According to the method, the positive sample feature vector database is constructed, and the multi-modal large model and the instance segmentation model work cooperatively, so that accurate identification of abnormal plants is realized on the premise that negative sample training data is not needed, misjudged samples are corrected and augmented to be put in storage, and iterative updating of the system is realized.

Description

Technical Field

[0001] This application relates to the field of pest and disease identification technology for Chinese medicinal materials, and in particular to a method, apparatus and equipment for identifying pests and diseases of Chinese medicinal materials based on positive samples of Chinese medicinal materials. Background Technology

[0002] Traditional identification of pests and diseases in Chinese medicinal herbs relies on human experience. Due to varying levels of experience among farmers, misjudgments are common, leading to incorrect pesticide application and irreversible losses. Using deep learning-based artificial intelligence methods, pest and disease identification in Chinese medicinal herbs can be achieved within seconds, significantly improving the identification rate. Furthermore, it allows for accurate identification at the early stages of pest and disease outbreaks, helping farmers intervene promptly to prevent widespread disease spread. Targeted pesticide application reduces the amount of chemical agents used, ensuring both the quality and yield of the medicinal herbs.

[0003] However, traditional deep learning recognition algorithms face several major challenges in identifying pests and diseases in Chinese medicinal herbs. First, data scarcity is a significant issue; pest and disease samples are difficult to collect and insufficient in quantity, easily leading to overfitting and poor generalization during model training. Second, the collected samples cannot cover all uploaded category data from real-world scenarios (it cannot exhaustively represent all negative classes of non-medicinal herbs). Uploading category data not in the training set (such as data on grasslands or roses) will cause the recognition model to classify it as belonging to a class in the training dataset based on the highest predicted probability, resulting in identification errors. This reduces farmers' trust in the system and decreases its usability. Furthermore, when in-class and out-of-class data are added, the model must be retrained and redeployed to improve accuracy on this additional data, which is time-consuming and costly.

[0004] Based on the above analysis, this embodiment proposes an algorithm for identifying diseases and pests in Chinese medicinal materials, centered on positive samples of the materials to be identified and constructed using cosine similarity retrieval and matching based on multimodal features of images. This algorithm significantly reduces the data requirements during algorithm construction and the cost of subsequent algorithm updates, while also ensuring the accuracy of the system in identifying diseases and pests. Summary of the Invention

[0005] This application provides a method for identifying diseases and pests in Chinese medicinal materials based on positive samples, characterized by comprising: Based on positive sample images of Chinese medicinal plants, normalized feature vectors are obtained through balance augmentation and feature extraction to generate a feature vector database. Based on the multimodal large model for recognizing Chinese medicinal plant images, the foreground region is obtained by foreground extraction, and features are extracted and normalized to generate feature vectors of Chinese medicinal plants. Based on the feature vector database, the feature vectors of Chinese medicinal plants are judged by similarity threshold and the frequency threshold of the highest similarity category, and negative sample Chinese medicinal plants are obtained and verified to achieve closed-loop iteration.

[0006] Optionally, the step of obtaining normalized feature vectors from positive sample medicinal plant images through balance augmentation and feature extraction to generate a feature vector database includes: Based on the health and pest / disease categories of Chinese medicinal plants, the images of the remaining categories are randomly sampled using the minimum number of samples for each category to generate original positive sample images for each category. Based on the positive sample images of each category, image augmentation is performed to obtain augmented positive sample images of each category; Based on the augmented positive sample images of each category and the original positive sample images, vector features are extracted through a multimodal model and normalized data is generated to form a feature vector database.

[0007] Optionally, the step of recognizing medicinal plant images based on a multimodal large model involves obtaining a purified foreground region through foreground extraction, performing feature extraction and normalization, and generating a feature vector for the medicinal plant, including: Uploaded images are analyzed using a multimodal large model to identify medicinal plant plants and obtain images that are identified as plants. A foreground extraction model is used to segment the foreground of images identified as plants and remove non-plant content in the foreground to obtain a purified foreground region. Based on the purification prospect area, vector features are extracted through a multimodal model and normalized data is generated to produce the feature vector of Chinese medicinal plants.

[0008] Optionally, the step of judging the feature vectors of medicinal plants based on the feature vector database using a similarity threshold and a frequency threshold for the highest similarity category, obtaining and verifying negative sample medicinal plants, and achieving closed-loop iteration includes: Based on a multimodal model, cosine similarity is generated by comparing the feature vectors of Chinese medicinal plants with the feature vectors of positive samples in the feature vector database. Based on the cosine similarity, the plant is compared with a preset similarity threshold and a threshold for the frequency of occurrence of the highest similarity category to identify negative samples. The negative sample plants were verified through user feedback and human expert review. Images with incorrect identification were obtained, and the images were augmented and normalized features were extracted. The corrected labels and augmented features were then stored in the feature vector database.

[0009] Optionally, the step of performing image augmentation based on positive sample images of each category to obtain augmented positive sample images of each category includes: For positive sample images of various categories, through geometric transformations such as rotation, flipping, and central cropping, the diversity simulation of the perspective and composition of plant images is realized; For positive sample images of various categories, through HSV channel enhancement and gamma transformation, the disturbance of the saturation, brightness, and color distribution of plant images is realized, and different lighting and environmental conditions are simulated; For positive sample images of various categories, through contrast / brightness adjustment and random expansion, the adjustment of the overall visual effect and background range of the images is realized.

[0010] Optionally, the foreground extraction model is used to perform foreground segmentation on the images identified as plants, and the non-plant content in the foreground is removed to obtain a purified foreground area, including: Through the foreground extraction model, masks and bounding boxes of all foreground objects in the image are generated, and foreground instances are obtained through segmentation; Semantic understanding and recognition are performed on each foreground instance, and non-target plant instance boxes in the foreground instances are screened and removed to obtain instance boxes determined to be plants; By obtaining the union of the contents of all instance boxes determined to be plants, a purified foreground area is generated.

[0011] Optionally, the negative sample judgment of plants is realized by comparing the cosine similarity with a preset similarity threshold and the occurrence frequency threshold of the highest similarity category, including: The similarity threshold is set as N, the occurrence frequency threshold of the highest similarity category is set as M, the cosine similarity is set as n, and the occurrence frequency of the highest similarity category of the cosine similarity is set as m; When n≥N and m≥M, the plant characteristics are similar to multiple similar samples in the database, and the feature space distribution is concentrated, belonging to normal samples, and a positive sample label is added to the plant; When n<N and m≥M, the plant characteristics are not similar to multiple similar samples in the database, and the feature space distribution is concentrated, belonging to unseen abnormal samples, and a negative sample label is added to the plant; When n≥N and m<M, the plant characteristics are similar to multiple similar samples in the database, and the feature space distribution is dispersed, belonging to unseen abnormal samples, and a negative sample label is added to the plant; When n<N and m<M, the plant characteristics are not similar to multiple similar samples in the database, and the feature space distribution is dispersed, belonging to unseen abnormal samples, and a negative sample label is added to the plant.

[0012] This application also provides a traditional Chinese medicine pest and disease identification device based on positive samples of traditional Chinese medicine, characterized in that the device includes: The positive sample feature library construction module is used to obtain normalized feature vectors based on positive sample Chinese medicinal plant images through balance augmentation and feature extraction, and generate a feature vector database. The image purification and feature extraction module is used to identify Chinese medicinal plant images based on a multimodal large model. It obtains the purified foreground region through foreground extraction, performs feature extraction and normalization, and generates the feature vector of Chinese medicinal plant. The feature matching and negative sample discrimination module is used to judge the feature vectors of Chinese medicinal plants based on the feature vector database, through similarity threshold and the frequency threshold of the highest similarity category, to obtain and verify negative sample Chinese medicinal plants and realize closed-loop iteration.

[0013] Optionally, the medicinal herb pest and disease identification device based on positive samples of medicinal herbs is characterized in that the feature matching and negative sample discrimination module further includes: The feature similarity calculation module is used to receive the feature vector to be identified and perform batch comparison calculations with the feature vector database. The dual threshold negative discrimination module is used to identify negative samples based on the preset dual threshold rules in the similarity calculation results; The manual verification and iteration module is used to handle misjudgment cases, receive feedback from users and experts, and realize a closed-loop process of system self-optimization.

[0014] This application also provides an electronic device, characterized in that it is used to implement the method for identifying diseases and pests of Chinese medicinal materials based on positive samples of Chinese medicinal materials as described in any one of claims 1 to 7, comprising: The processor is used to perform all computational tasks and implement a method for identifying diseases and pests in Chinese medicinal materials based on positive samples of Chinese medicinal materials. Memory is used to store processor-executable instructions and statically stored data.

[0015] The beneficial effects of this application are as follows: by constructing a highly generalizable positive sample feature vector database, the fundamental problem of obtaining negative samples of pests and diseases in agricultural scenarios is solved. Through the collaborative work of a multimodal large model and an instance segmentation model, plant identification and non-plant foreground removal are achieved, a purified foreground region is obtained, the interference of complex field background on feature extraction is reduced, and the accuracy of main feature representation is improved. Furthermore, by integrating the absolute value of cosine similarity and the statistical distribution of results into a dual threshold judgment mechanism, the accurate identification of abnormal plants is achieved without the need for negative sample training data. Finally, through the interactive closed loop of user feedback and expert review, misjudged samples are corrected and expanded into the database, realizing the continuous self-evolution of the system's recognition ability and the enhancement of its generalization. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings required in the description of the embodiments or the prior art are briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a specific embodiment of the method for identifying diseases and pests in Chinese medicinal materials based on positive samples of Chinese medicinal materials according to this application is shown. Figure 2 This invention illustrates a structural diagram of Panax notoginseng identification based on a specific embodiment of the method for identifying diseases and pests in Chinese medicinal materials using positive samples. Figure 3 A flowchart illustrating the identification process of Panax notoginseng based on a specific embodiment of the method for identifying diseases and pests in Chinese medicinal materials using positive samples of Chinese medicinal materials, according to a specific embodiment of this application; Figure 4 This is a block diagram of an apparatus for identifying diseases and pests in Chinese medicinal materials based on positive samples of Chinese medicinal materials, according to a specific embodiment of this application. Detailed Implementation

[0018] Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0019] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0020] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0021] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented without certain specific details. In some instances, methods, means, components, and circuits well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0022] This application proposes a method for identifying diseases and pests in Chinese medicinal herbs based on positive samples, addressing the problems of scarce negative samples, complex background interference, and weak model generalization ability in the field of agricultural artificial intelligence. Firstly, to address the sample imbalance problem, a targeted equalization augmentation strategy is designed. Using the least number of sample categories as a benchmark, high-intensity, multimodal image augmentation is performed on positive samples of each category. Normalized features are extracted using a pre-trained multimodal model to construct a highly generalizable positive sample feature database. In the identification stage, a hierarchical processing flow is adopted. First, a large multimodal model is used for coarse-grained plant discrimination to quickly filter out non-plant images. Then, an instance segmentation model is used to extract the foreground from images identified as plants, and semantic understanding is introduced to remove bounding boxes of non-target plants. Finally, a highly purified foreground region is generated, eliminating noise interference from complex backgrounds. Secondly, a multimodal model is used to extract feature vectors of the foreground to be identified, and retrieval is performed based on the cosine similarity with the positive sample database. A dual negative sample judgment mechanism is proposed, which not only relies on the absolute value of the highest similarity, but also analyzes the frequency of occurrence of the highest similarity category in the retrieval results to comprehensively judge the distribution dispersion of the feature space, thereby achieving high-precision interception of negative samples under zero-sample conditions. Finally, by introducing a human collaborative closed loop of user feedback and expert review, misjudged samples of the system are corrected and the feature database is iteratively updated, realizing the system's self-optimization.

[0023] Example 1 like Figure 1 The diagram shown is a flowchart of a method for identifying diseases and pests in Chinese medicinal materials based on positive samples, according to an embodiment of this application. The method specifically includes the following: S100, based on positive sample images of medicinal plants, obtains normalized feature vectors through balance augmentation and feature extraction, and generates a feature vector database.

[0024] Specifically, given the reality of imbalanced sample categories for medicinal herb diseases and pests, a balanced sampling and augmentation process based on the minimum sample category is established. First, the sampling strategy requires random number extraction from images of categories with an excess of data to ensure a strictly consistent number of original positive samples for each category. Second, the augmentation strategy introduces various geometric and optical transformations to simulate the morphological and appearance variations that plants may exhibit in real-world imaging environments. Geometric transformations include rotation and mirroring based on the image center, as well as cropping the central image region at a fixed ratio to enhance the model's robustness to changes in viewpoint and composition. Optical transformations involve randomizing the saturation and brightness channels in the color space and adjusting the image's contrast and brightness to simulate color differences caused by different lighting and acquisition devices. Finally, deep feature extraction is performed on all original and augmented images using a pre-trained multimodal large-scale model, and the resulting feature vectors are normalized according to the L2 norm, thereby constructing a positive sample feature vector database.

[0025] S200 is based on a multimodal large model to identify images of Chinese medicinal plants. It obtains a purified foreground region through foreground extraction, performs feature extraction and normalization, and generates a feature vector of Chinese medicinal plants.

[0026] Specifically, based on a foreground extraction model, refined feature extraction is achieved from user-uploaded images. First, a large-scale visual and language model is used to perform deep semantic analysis on the input image. Natural language prompts guide the model to classify whether the image content contains plants, filtering out irrelevant scene images. Second, for images determined to contain plants, foreground purification is performed. All foreground objects in the image are segmented and bounding boxes are located. Then, based on semantic understanding, all segmented instances are filtered, actively removing interfering instance boxes identified as not being the main plant subject. Finally, the geometric union of all remaining plant instance boxes is taken, and a clean plant subject region is cropped from the original image. This purified image region is used as input for feature extraction, mapping it into a normalized feature vector.

[0027] S300, based on the feature vector database, judges the feature vector of Chinese medicinal plants by using similarity threshold and the frequency threshold of the highest similarity category, obtains and verifies negative sample Chinese medicinal plants, and realizes closed-loop iteration.

[0028] Specifically, a dual negative sample determination mechanism is constructed based on a positive sample feature vector database to identify negative samples of unknown categories under an open set setting. First, the cosine similarity of the feature vector to be identified with all positive sample features in the database is compared to retrieve the K results with the highest similarity and their category labels. K is an integer from 3 to 10, which can be set according to actual needs. For example, in a specific scheme, K can be set to 5, retrieving the 5 results with the highest similarity. These results can be sorted in descending order of similarity. Second, the dual negative sample determination mechanism makes a judgment. The first criterion is a similarity threshold. If the highest similarity value is lower than a preset lower limit, the input sample is directly determined to be significantly different from the known positive sample database and classified as a negative sample. The second criterion is a distribution statistics threshold. For samples with the highest similarity value higher than the absolute threshold but still questionable, the frequency of the highest similarity category label in the top K results is counted. If this frequency is too low, it indicates that the features of the input sample are discretely distributed in the positive sample feature space, and it is also determined to be a negative sample. Finally, for all negative samples and positive samples misjudged by the system, a human expert review process is introduced for final verification. Samples confirmed to be erroneous are augmented and their features extracted, and then incorporated into the feature database along with correction labels, thereby enhancing the system's self-iteration and generalization capabilities during continuous use.

[0029] In summary, this solution addresses the core challenges of identifying diseases and pests in Chinese medicinal herbs, such as the scarcity of negative samples, complex background interference, and weak model generalization ability. It proposes a recognition method based on positive sample learning and multimodal large-scale model collaboration. First, by constructing an equalization augmentation and feature extraction process, the imbalance of sample categories is effectively solved. This method uses the smallest sample category as a benchmark for random sampling to ensure the balance of data volume across categories. It also comprehensively utilizes geometric and optical transformations to simulate the morphological and phenotypic variations of plants in real-world scenarios, significantly enhancing the diversity and representativeness of the data. Finally, features are extracted and normalized through a pre-trained model to construct a high-quality positive sample feature database, laying a reliable data foundation for subsequent recognition tasks. Second, a hierarchical image processing mechanism is employed to achieve accurate feature extraction of the plant subject against complex backgrounds. Utilizing the semantic understanding capabilities of the multimodal large-scale model, non-plant images are quickly filtered out. Combined with instance segmentation and semantic filtering techniques, non-plant interference areas such as soil and debris are actively identified and removed, extracting a clean plant foreground. This greatly reduces the interference of background noise on feature representation and improves the accuracy and robustness of subsequent feature matching. Finally, by employing a dual-threshold negative sample judgment mechanism and a manual collaborative iterative process, the system achieved continuous self-optimization. By integrating absolute similarity comparison and feature space distribution statistics, it effectively identified negative samples of unknown categories, overcoming the limitations of traditional methods that rely on negative sample training data. Furthermore, by introducing user feedback and expert review processes, misjudged samples were corrected and the feature library was iteratively updated, enabling the system to continuously expand its recognition boundaries and generalization performance. This provides efficient and reliable technical support for the intelligent identification of Chinese medicinal plants and the monitoring of pests and diseases.

[0030] As an optional implementation of this application, optionally, in step S100, based on the medicinal plant images in the positive samples, normalized feature vectors are obtained through balance augmentation and feature extraction to generate a feature vector database, including: S101. Based on the health and pest / disease categories of medicinal plant plants, select the category with the smallest number of samples to randomly sample the images of the remaining categories, generating original positive sample images for each category.

[0031] Specifically, the first step is to statistically group all labeled images of medicinal plants according to their health status or specific pest / disease category, forming multiple independent category sets. These categories include viral diseases, gray mold, powdery mildew, black spot, slime mold, nutrient imbalance, and healthy. Then, the algorithm iterates through all categories, identifying the category with the fewest samples, and defines the total number of samples in that category as the baseline. This baseline becomes the target value for subsequent random sampling. For all other categories with more samples than this baseline, the system initiates a sampling process based on a pseudo-random number generator. This process assigns a unique random number to each image in these excess categories and performs random sampling without replacement based on this number until the number of extracted images reaches the aforementioned baseline. Through this mechanism, the image sets of all categories are forced to align to the same level in terms of quantity, thereby eliminating potential model training bias caused by an excessive number of samples in certain categories at the data level. This lays a solid foundation for constructing an unbiased feature representation space.

[0032] S102, perform image augmentation based on the positive sample images of each category to obtain augmented positive sample images of each category.

[0033] Specifically, the equalized original positive sample images obtained in step S101 are augmented with data to simulate the visual diversity that Chinese medicinal plants may exhibit in real complex environments, thereby substantially improving the generalization ability and robustness of the feature model constructed subsequently.

[0034] The augmentation strategy is systematically divided into two parts: geometric transformation and optical transformation. Geometric transformation mainly operates on the spatial structural properties of the image, including arbitrary angle rotation transformation with the image center point as the origin, horizontal mirror flip transformation along the vertical central axis, and deterministic cropping transformation of the image center region with a fixed scaling ratio, enabling the model to adapt to different plant orientations, viewing angles, and composition ratios during shooting. Optical transformation targets the pixel intensity and color attributes of the image. Its operation is performed in a specific color space, including applying linear perturbations with random coefficients to the saturation and brightness channels to simulate the effects of different lighting conditions and device color reproduction differences. At the same time, it performs global contrast enhancement and brightness adjustment based on random parameters to cover various ambient light intensities from dark to bright. Each original image undergoes this combination of transformations to generate a series of visually diverse but semantically unlabeled augmented images, significantly expanding the effective training dataset in both quantity and quality.

[0035] S103. Based on the augmented positive sample images of each category and the original positive sample images, vector features are extracted through a multimodal model and normalized data is generated to form a feature vector database.

[0036] Specifically, all the original images and their corresponding augmented images generated in steps S101 and S102 are fed into a large-scale visual model pre-trained on massive multimodal data for forward propagation. This model uses a deep convolutional neural network as a feature extractor, outputting a floating-point vector, which is considered a high-level semantic representation of the input image. Then, L2 norm normalization is performed on each extracted original feature vector, dividing each element value by the L2 norm of the vector, converting any length of original feature vector into a unit vector of length 1. Finally, all normalized feature vectors and their corresponding image category labels are structured and stored together to construct a positive sample feature vector database.

[0037] As an optional implementation of this application, optionally, in step S200, the image of the Chinese medicinal plant is identified based on a multimodal large model, a clean foreground region is obtained through foreground extraction, and feature extraction and normalization are performed to generate a feature vector of the Chinese medicinal plant, including: S201 uses multimodal large model analysis to upload images, performs plant identification of Chinese medicinal materials, and obtains images that are identified as plants.

[0038] Specifically, a multimodal large model integrating visual and language understanding capabilities is used to construct specific natural language prompts, which are input into the model along with the user-uploaded image. These prompts guide the model to perform semantic-based deep analysis and binary classification of the image content. The core question is "Does the image contain a certain plant?" The model performs scene understanding and object recognition on the input image and outputs a confidence score based on its internal reasoning. A preset judgment threshold is set in the system. If the confidence score output by the model is higher than this threshold, the image is determined to be a valid image containing the target plant and is passed to subsequent processes. If it is lower than the threshold, it is determined to be an irrelevant image, the process terminates, and the corresponding prompt is returned to the user.

[0039] S202, using a foreground extraction model, performs foreground segmentation on images identified as plants and removes non-plant content in the foreground to obtain a purified foreground region.

[0040] Specifically, firstly, a general instance segmentation model is used to process the image acquired in step S201. This model performs pixel-level segmentation on each potential foreground object in the image and generates a precise binary mask and its bounding rectangle for each identified instance. Then, the image region within each instance bounding box is fed back into a multimodal large model for fine-grained object recognition. The system pre-defines a list of non-plant interference objects (such as soil, stones, plastic, other weeds, etc.), and any instance boxes identified as belonging to this category are automatically marked and removed. Finally, all instance boxes determined to be plants or parts thereof are retained. The system calculates the geometric union of these retained boxes and crops the final purified foreground region from the original image based on this union region.

[0041] S203, based on the purification foreground area, extract vector features through a multimodal model and generate normalized data to generate the feature vector of Chinese medicinal plant.

[0042] Specifically, first, the purified foreground region image output in step S202 is input. This image is adjusted to the input size required by the multimodal large model and fed into the model's deep neural network for forward propagation. The output of the network's final hidden layer is extracted to generate a floating-point vector. Then, by dividing the value of each dimension of the original feature vector by the L2 norm of the entire vector, the feature vector is transformed into a unit vector with a magnitude of 1, achieving normalization. Finally, this normalized vector is output as the feature vector of the medicinal plant.

[0043] As an optional implementation of this application, optionally, in step S300, based on the feature vector database, the feature vectors of medicinal plants are judged by a similarity threshold and a frequency threshold of the highest similarity category, negative sample medicinal plants are obtained and verified, and closed-loop iteration is achieved, including: S301, based on a multimodal model, calculates cosine similarity by comparing the feature vectors of Chinese medicinal plants with the feature vectors of positive samples in the feature vector database.

[0044] Specifically, first, the input consists of the normalized feature vector of the medicinal plant obtained in step S203, and the normalized positive sample feature vectors from the feature vector database generated in step S103. Since all vectors have been normalized, the calculation of their cosine similarity can be efficiently simplified to a dot product operation between vectors.

[0045] Among them, the system performs a dot product operation between the vector to be queried and each vector in the database. The result of the dot product is a scalar, and its numerical range is between [-1, 1]. However, in this scenario, since the feature vectors are located in the positive quadrant of the high-dimensional space, their values actually distribute in the interval [0, 1]. This value is directly equal to the cosine value of the angle between the two vectors. The closer the value is to 1, the more consistent the directions of the two vectors are, and the higher the similarity. The closer the value is to 0, the less similar they are. The system will calculate the cosine similarity score corresponding to the vector to be queried and each vector in the database, and usually retains the top K results with the highest similarity and their corresponding sample labels to form a list.

[0046] S302. According to the cosine similarity, compare it with the preset similarity threshold and the threshold of the frequency of the highest similarity category to achieve the judgment of the negative sample of the plant.

[0047] Specifically, the determination is based on the sequence of cosine similarity results calculated in step S301. The first criterion for determination is the similarity threshold. The system presets a lower similarity threshold, which defines the absolute standard of sufficient similarity. First, check the value of the highest cosine similarity. If this maximum value is lower than this preset threshold, it is directly determined that the current input sample is not similar enough to any sample in the known positive sample database, so it is classified as a negative sample. If the highest similarity is higher than this absolute value threshold, the second criterion for determination is started, that is, the distribution statistics determination. The system analyzes the distribution of the class labels corresponding to the top K highest similarity results. The system presets a lower frequency threshold. If this frequency is lower than this threshold, it indicates that the features of the sample to be identified are far from the clustering center of its nearest class in the feature space, or in other words, its feature representation appears isolated or scattered in this class. This discreteness in distribution proves that it is an abnormal sample, so it is determined as a negative sample. Only when the sample passes both of these two determinations will it be finally recognized as a positive sample.

[0048] Among them, the similarity threshold is set to N, the threshold of the frequency of the highest similarity category is set to M, the cosine similarity is set to n, and the frequency of the highest similarity category of the cosine similarity is set to m; When n≥N and m≥M, the plant features are similar to multiple similar samples in the database, and the feature space distribution is concentrated, belonging to normal samples, and add a positive sample label to the plant; When n<N and m≥M, the plant features are not similar to multiple similar samples in the database, and the feature space distribution is concentrated, belonging to unseen abnormal samples, and add a negative sample label to the plant; When n≥N and m<M, the plant features are similar to multiple similar samples in the database, and the feature space distribution is scattered, belonging to unseen abnormal samples, and add a negative sample label to the plant; When n < N and m < M, the plant characteristics are not similar to multiple similar samples in the database, and the feature space distribution is scattered, belonging to unseen abnormal samples. Add a negative sample label to this plant.

[0049] S303. Verify the negative sample plants through user feedback and manual expert review, obtain the misrecognized images, perform image augmentation and extract normalized features, and store the corrected labels and augmented features in the feature vector database together.

[0050] Specifically, when the user has doubts about the negative sample results automatically determined by the system or discovers obvious misjudgments, the user can submit correction feedback through the interactive interface provided by the system and mark the image as pending review. For the images misjudged by the system confirmed by experts, the system will subject the original misjudged images to exactly the same image augmentation process as in step S102, accept various geometric and optical transformations, generate a series of new augmented images that can cover potential scene variations, and then these augmented images together with the original misjudged images are batch-fed into the pre-trained multi-modal feature extraction model for forward propagation to extract their high-dimensional deep feature vectors, and strictly perform L2 norm normalization processing on each of the extracted original feature vectors to convert them into unit vectors with unified modulus lengths, ensuring their consistency with the existing data in the feature vector database in the metric space. Finally, these new feature vectors generated by the misjudged images and their augmented versions and strictly normalized, together with their corresponding corrected class labels, are formed into new entries, and the system incrementally incorporates these new entries into the existing positive sample feature vector database to achieve iterative optimization.

[0051] Embodiment 2 As an application case of this method, the specific implementation process is as follows: As Figure 2 shown, in this embodiment, a multi-modal feature embedding vector database for Panax notoginseng pests and diseases images is first constructed, and then for the images uploaded by the user, feature extraction and similarity retrieval are performed on the vector database, and based on the retrieval results, the identification of the plant category and the pests and diseases suffered is further achieved.

[0052] 1. Construction of the vector database The construction process of the Panax notoginseng pests and diseases database is as follows (as Figure 3 shown): Image sampling: 1-1 First, for the categories of Panax notoginseng pests and diseases: viral disease, gray mold, powdery mildew, black spot, slime mold, nutritional imbalance, and healthy, 10 pictures are selected respectively.

[0053] Since the number of diseased plants varies greatly, in order to ensure the balance of various samples in feature extraction and storage in the feature vector database, the images of other categories are randomly sampled using the sample of the category with the fewest number of samples (10 images in this example). The sampling strategy is as follows: the images are labeled with numbers from 0 to N, and 10 sequences are extracted from them using the Python random sampling function numpy.random.choice to realize the sampling of the images with the corresponding sequence numbers.

[0054] Image augmentation: 1-2 respectively performed image augmentation (rotation, flipping, center-based cropping, HSV channel enhancement, gamma transformation, contrast / brightness adjustment, random padding) on ​​10 images for each disease category. To enhance the generalization of extracted image features, the 10 sampled images of various plants were rotated, flipped, cropped centered on the image center, HSV channel enhanced, gamma transformed, contrast / brightness adjusted, and a random padding image augmentation strategy was applied. Details are as follows: Rotation: Using OpenCV's getRotationMatrix2D function, an image rotation matrix is ​​constructed with the center of the image as the coordinate center and rotation angles of 45, 90, 135, 180, 225, and 270 degrees respectively, to perform rotation transformation on the input image.

[0055] Flip: Using OpenCV's flip function, the image is flipped horizontally and vertically with the center point of the image as the coordinate center, the horizontal axis as the X-axis, and the vertical axis as the Y-axis.

[0056] Center cropping: Using the center of the input image as the center, obtain the width and height of the cropped image at 80% of the input image's width and height. Based on the width and height of the cropped image, calculate the coordinates of the left, right, top, and bottom boundaries of the cropped image, and then crop the image.

[0057] HSV Channel Enhancement: The input image is converted from BGR space to HSV space. For the S (saturation) channel, OpenCV's add function is used to select a value from the range (-30, 30) as the enhancement parameter to enhance saturation. For the luminance channel, a value from the range (-30, 30) is selected as the enhancement parameter to enhance luminance. The transformed image is then converted from HSV space back to BGR space.

[0058] Gamma transformation: Normalize each channel of the BGR image (divide by 255), take the gamma parameter as 1.5, and perform gamma transformation to adjust the color of each channel of the image.

[0059] Contrast / Brightness Adjustment: The contrast and brightness of the input image are adjusted using Python's `random.uniform` function, which takes a value in the range (0.7, 1.3) as the contrast parameter `alpha`, and `random.randint` function, which takes a value in the range (-50, 50) as the brightness parameter `beta`. OpenCV's `convertScaleAbs` function is then used to adjust the contrast and brightness of the input image.

[0060] Random padding: Using Python's random.uniform function, take a parameter in the range (0.1, 0.8) to get the expansion ratio of the image width and height, and randomly select a value between (0, 255) as the background color value for the expansion to expand the image.

[0061] Feature extraction and normalization The Dino-v2 multimodal model was used to extract features from the augmented image, obtaining 768-dimensional features and normalizing them. Download the pre-trained model weights of the Dino-v2 multimodal model trained on massive multimodal data from Hugging Face. Process the original and augmented images of various diseased plants using the Dino-v2 image preprocessor to convert them into a data format that the Dino-v2 multimodal model can process. Then, use the Dino-v2 model to extract vector features and divide the features by the L2 norm of the features to normalize the features to the (0, 1) interval.

[0062] 1-3 are stored in a vector database (this embodiment uses the Milvus database, but is not limited to this vector database).

[0063] 2. Image recognition The process for classifying plant categories and searching for pests and diseases in user-uploaded images is as follows (e.g., Figure 2 ).

[0064] 2-1. Plant identification First, a multimodal large model is used (in this embodiment, the qwen-vl-max multimodal large model is selected, but it is not limited to this model). The uploaded image is analyzed to determine whether it belongs to a plant. If it does not belong to a plant, it is directly returned as "not Panax notoginseng".

[0065] Qwen-VL-Max is a large-scale visual-language multimodal model launched by Alibaba's Qwen team. It is a high-performance version of the Qwen-VL series, focusing on handling complex image and text interaction tasks. It supports visual question answering (VQA).

[0066] In this embodiment, the Qwen-VL-Max API provided by Alibaba Cloud is used to perform semantic understanding of user-uploaded images by inputting prompts.

[0067] First, the user-uploaded image is converted to a base64 image. Then, the prompt is entered: "Analyze whether there are plants in the image," and the result is returned in the following format: {Plant: Yes / No}. This achieves the identification of whether it is a plant. For those that do not belong to the plant category, they can be further determined to be non-Panax notoginseng.

[0068] Plants and non-plants are relatively easy to distinguish, and the accuracy of Qwen-VL-Max identification is high, reaching 100% on the test set (50 plants and 50 non-plants). However, the accuracy of Qwen-VL-Max identification decreases within plants, especially among plants with similar phenotypic characteristics and among diseased plants, requiring more refined identification methods.

[0069] 2-2. Foreground Extraction For images identified as plants, a foreground extraction model (this embodiment selects the SAM model, but is not limited to the SAM model) is used to extract the foreground region of the image, as shown below: For images identified as plants, the image format is converted to BGR. The SAM2 (Segment Anything Model 2) model from Ultralytics is used to segment the foreground in the image, extract foreground instances, and further extract the minimum bounding rectangles of the foreground instances. After removing the minimum bounding rectangles of non-plant categories, the union of the remaining rectangles is taken as the ROI (Region of Interest) of the foreground.

[0070] The foreground Region of Interest (ROI) is cropped from the original image to reduce the interference of the background on feature extraction and similarity matching. Then, the multimodal model Dino-v2 is used for feature extraction and normalization. Next, cosine similarity is used to retrieve the image's label and cosine similarity value from the vector database. The top-10 similarity categories and their corresponding cosine similarity values ​​are selected. Based on this, the category with the highest frequency among the top-10 is chosen as the matching category. If the proportion of matched categories is lower than a specified threshold (set to 0.6 in this embodiment) or the mean cosine similarity of matched categories is lower than a specified threshold (set to 0.5 in this embodiment), the user-uploaded image is determined to be non-37.

[0071] 2-3. Negative Sample Resolution Since the pre-trained multimodal large model is trained on massive image and text data, it has a good representation ability for features and the relationships between features. For user-uploaded images, the multimodal large model extracts features based on Dinov2 and performs feature normalization. Then, it calculates the cosine distance between the features of positive class (Panax notoginseng and diseased Panax notoginseng) samples in the Milvus vector database. When the similarity is lower than a certain threshold (set to 0.5 in this embodiment) or the distance between the input sample and the Panax notoginseng category in the vector database is relatively dispersed (the number of categories with the highest similarity is less than the threshold of 5), it can be considered that the input sample is far away from the samples in the vector database, and the input sample does not belong to any category in the database. It is regarded as non-Panax notoginseng, that is, a negative sample. This solves the problem of missing negative samples when constructing the disease and pest sample database to a certain extent.

[0072] Meanwhile, since the multimodal large model is trained on massive amounts of data, the model has a better ability to represent features, which also avoids the problem of model generalization when training a small model with few sample data to identify diseases and pests of Panax notoginseng.

[0073] Example 3 Based on the same principles as the aforementioned methods, a method for recovering falsely detected meridians based on spatial topology and location reasoning is also proposed. See [link to relevant documentation]. Figure 4 An embodiment of this disclosure provides a device 100 for identifying diseases and pests in Chinese medicinal materials based on positive samples, comprising: The positive sample feature library construction module 110 is used to obtain normalized feature vectors based on positive sample Chinese medicinal plant images through balance augmentation and feature extraction, and generate a feature vector database. The image purification and feature extraction module 120 is used to identify Chinese medicinal plant images based on a multimodal large model. It obtains the purified foreground region through foreground extraction, performs feature extraction and normalization, and generates a feature vector of Chinese medicinal plant. The feature matching and negative sample discrimination module 130 is used to judge the feature vector of Chinese medicinal plants based on the feature vector database by using a similarity threshold and a frequency threshold of the highest similarity category, to obtain and verify negative sample Chinese medicinal plants, and to achieve closed-loop iteration.

[0074] As an optional implementation of this application, the feature matching and negative sample discrimination module 130 may further include: The feature similarity calculation module 131 is used to receive the feature vector to be identified and perform batch comparison calculations with the feature vector database. The dual threshold negative discrimination module 132 is used to discriminate negative samples from the similarity calculation results according to the preset dual threshold rules; The manual verification and iteration module 133 is used to handle misjudgment cases, receive feedback from users and experts, and realize a closed-loop process of system self-optimization.

[0075] Obviously, those skilled in the art should understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the control methods described above. The modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, the present invention is not limited to any specific hardware and software combination.

[0076] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the control methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0077] Example 4 Furthermore, this application proposes an electronic device, characterized in that it is used to implement any of the described methods for identifying diseases and pests of Chinese medicinal materials based on positive samples, comprising: The processor is used to perform all computational tasks and implement a method for identifying diseases and pests in Chinese medicinal materials based on positive samples of Chinese medicinal materials. Memory is used to store processor-executable instructions and statically stored data.

[0078] The electronic device of this disclosure includes a processor and a memory for storing processor-executable instructions. The processor is configured to implement, when executing the executable instructions, any of the preceding methods for identifying diseases and pests in Chinese medicinal materials based on positive samples.

[0079] It should be noted that the number of processors can be one or more. Furthermore, the electronic device in this embodiment may also include input devices and output devices. The processor, memory, input devices, and output devices can be connected via a bus or other means, without specific limitations herein.

[0080] The memory, serving as a computer-readable storage medium for the method of identifying diseases and pests of Chinese medicinal materials based on positive samples, can be used to store software programs, computer-executable programs, and various modules, such as the program or module corresponding to the method of identifying diseases and pests of Chinese medicinal materials based on positive samples in this disclosure. The processor executes various functional applications and data processing of the electronic device by running the software program or module stored in the memory.

[0081] Input devices can be used to receive input digital numbers or signals. These signals can be key signals related to user settings and function control of the device / terminal / server. Output devices can include display devices such as screens.

[0082] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for identifying diseases and pests in Chinese medicinal materials based on positive samples, characterized in that, include: Based on positive sample images of medicinal plants, normalized feature vectors are obtained through balance augmentation and feature extraction. Generate a feature vector database; Based on the multimodal large model for recognizing Chinese medicinal plant images, the foreground region is obtained by foreground extraction, and features are extracted and normalized to generate feature vectors of Chinese medicinal plants. Based on the feature vector database, the feature vectors of Chinese medicinal plants are judged by similarity threshold and the frequency threshold of the highest similarity category, and negative sample Chinese medicinal plants are obtained and verified to achieve closed-loop iteration.

2. The method for identifying diseases and pests of medicinal herbs based on positive samples of medicinal herbs as described in claim 1, characterized in that, The normalized feature vector is obtained from the positive sample images of medicinal plants through balance augmentation and feature extraction. Generate a feature vector database, including: Based on the health and pest / disease categories of Chinese medicinal plants, the images of the remaining categories are randomly sampled using the minimum number of samples for each category to generate original positive sample images for each category. Based on the positive sample images of each category, image augmentation is performed to obtain augmented positive sample images of each category; Based on augmented positive sample images of each category and original positive sample images, vector features are extracted using a multimodal model and normalized data is generated to form a feature vector database.

3. The method for identifying diseases and pests of Chinese medicinal materials based on positive samples of Chinese medicinal materials as described in claim 1, characterized in that, The method for recognizing medicinal plant images based on a multimodal large model involves obtaining a purified foreground region through foreground extraction, performing feature extraction and normalization, and generating a feature vector for the medicinal plant, including: Uploaded images are analyzed using a multimodal large model to identify medicinal plant plants and obtain images that are identified as plants. A foreground extraction model is used to segment the foreground of images identified as plants and remove non-plant content in the foreground to obtain a purified foreground region. Based on the purification prospect area, vector features are extracted through a multimodal model and normalized data is generated to produce the feature vector of Chinese medicinal plants.

4. The method for identifying diseases and pests of Chinese medicinal materials based on positive samples of Chinese medicinal materials as described in claim 1, characterized in that, The process involves judging the feature vectors of medicinal plants based on a feature vector database using a similarity threshold and a frequency threshold for the highest similarity category, obtaining and verifying negative sample medicinal plants, and achieving closed-loop iteration, including: Based on a multimodal model, cosine similarity is generated by comparing the feature vectors of Chinese medicinal plants with the feature vectors of positive samples in the feature vector database. Based on the cosine similarity, the plant is compared with a preset similarity threshold and a threshold for the frequency of occurrence of the highest similarity category to identify negative samples. The negative sample plants were verified through user feedback and human expert review. Images with incorrect identification were obtained, and the images were augmented and normalized features were extracted. The corrected labels and augmented features were then stored in the feature vector database.

5. The method for identifying diseases and pests of Chinese medicinal materials based on positive samples of Chinese medicinal materials as described in claim 2, characterized in that, The step of image augmentation based on positive sample images of each category to obtain augmented positive sample images of each category includes: For positive sample images of each category, geometric transformations such as rotation, flipping, and center cropping are used to simulate the diversity of plant image perspectives and compositions. For each category of positive sample images, HSV channel enhancement and gamma transformation are used to perturb the saturation, brightness, and color distribution of plant images, simulating different lighting and environmental conditions. For each category of positive sample images, the overall visual effect and background range of the image are adjusted by contrast / brightness adjustment and random expansion.

6. The method for identifying diseases and pests of Chinese medicinal materials based on positive samples of Chinese medicinal materials as described in claim 3, characterized in that, Using the foreground extraction model, perform foreground segmentation on the image recognized as a plant, and剔除 non-plant content in the foreground to obtain a purified foreground area, including: Generate masks and bounding boxes for all foreground objects in the image through the foreground extraction model, and perform segmentation to obtain foreground instances; Perform semantic understanding and recognition on each foreground instance, screen and剔除 non-target plant instance boxes in the foreground instances, and obtain instance boxes determined to be plants; Generate a purified foreground area by obtaining the union of the contents of all instance boxes determined to be plants.

7. The method for identifying diseases and pests of Chinese medicinal materials based on positive samples of Chinese medicinal materials as described in claim 4, characterized in that, According to the cosine similarity, compare it with the preset similarity threshold and the highest similarity category occurrence frequency threshold to实现 negative sample judgment of plants, including: The similarity threshold is set to N, the highest similarity category occurrence frequency threshold is set to M, the cosine similarity is set to n, and the highest similarity category occurrence frequency of the cosine similarity is set to m; When n≥N and m≥M, the plant characteristics are similar to multiple similar samples in the database, and the feature space distribution is concentrated, belonging to normal samples, and add a positive sample label to the plant; When n<N and m≥M, the plant characteristics are not similar to multiple similar samples in the database, and the feature space distribution is concentrated, belonging to unseen abnormal samples, and add a negative sample label to the plant; When n≥N and m<M, the plant characteristics are similar to multiple similar samples in the database, and the feature space distribution is dispersed, belonging to unseen abnormal samples, and add a negative sample label to the plant; When n<N and m<M, the plant characteristics are not similar to multiple similar samples in the database, and the feature space distribution is dispersed, belonging to unseen abnormal samples, and add a negative sample label to the plant.

8. A device for identifying diseases and pests in Chinese medicinal materials based on positive samples, characterized in that, The device includes: A positive sample feature library construction module, which is used to obtain normalized feature vectors based on positive sample traditional Chinese medicine plant images through balanced augmentation and feature extraction, and generate a feature vector database; An image purification and feature extraction module, which is used to recognize traditional Chinese medicine plant images based on a multi-modal large model, obtain a purified foreground area through foreground extraction, perform feature extraction and normalization, and generate traditional Chinese medicine plant feature vectors; A feature matching and negative sample discrimination module, which is used to judge traditional Chinese medicine plant feature vectors according to the feature vector database through a similarity threshold and the highest similarity category occurrence frequency threshold, obtain and verify negative sample traditional Chinese medicine plants, and实现 a closed-loop iteration.

9. The medicinal herb disease and pest identification device based on positive samples of medicinal herbs according to claim 8, characterized in that, The feature matching and negative sample discrimination module further includes: A feature similarity calculation module, which is used to receive the feature vector to be recognized and perform batch comparison calculations with the feature vector database; A dual-threshold negative judgment module, which is used to perform negative sample discrimination on the similarity calculation results according to the preset dual-threshold rule; An artificial verification and iteration module, which is used to process misjudgment cases, receive feedback from users and experts, and实现 a closed-loop process of system self-optimization.

10. An electronic device, characterized in that, For implementing the traditional Chinese medicine pest and disease identification method based on positive samples of traditional Chinese medicine described in any one of claims 1 to 7, including: A processor, which is used to execute all computational tasks and实现 the traditional Chinese medicine pest and disease identification method based on positive samples of traditional Chinese medicine; A memory, which is used to store processor-executable instructions and static stored data.

Images