A veterinary medicinal material sorting and impurity removal device and method based on visual perception
The veterinary medicinal material sorting and impurity removal device uses a multi-scale representation learning model with multi-task prediction to address the inefficiencies of manual sorting, achieving accurate and efficient quality control by integrating prior knowledge and capturing long-range dependencies in veterinary medicinal material images.
Patent Information
- Application Number
- CN202411147515.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2044-08-21
AI Technical Summary
Traditional manual sorting and impurity removal of veterinary medicinal materials are subjective, inefficient, and fail to accurately assess subtle differences in appearance quality, leading to inconsistent quality control in the veterinary medicinal materials supply chain.
A veterinary medicinal material sorting and impurity removal device and method based on visual perception, utilizing a multi-scale representation learning model that integrates prior knowledge and employs multi-task learning to predict defect types and quality scores, enhancing the model's ability to capture long-range dependencies and correlations within images.
The method provides rapid, objective, and traceable evaluation of veterinary medicinal material quality, improving accuracy and efficiency in sorting and impurity removal by leveraging a robust, generalized model trained on a high-quality dataset with integrated prior knowledge.
Smart Images
Figure CN118663572B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of veterinary medicinal material detection, and in particular relates to a veterinary medicinal material sorting and impurity removal device and method based on visual perception. Background Art
[0002] Veterinary medicinal materials are the raw materials for preparing veterinary Chinese medicine preparations, and their quality is directly related to the safety and effectiveness of animal medicines. Veterinary medicinal materials must not contain harmful ingredients such as mold, deterioration, insect bites, and impurities, and quantitative or qualitative testing requirements are put forward for the quality attributes of different veterinary medicinal materials such as color and impurity content. For example, astragalus veterinary medicinal materials should be cylindrical, with a gray-yellow or light brown surface, hard and tough texture, and fibrous cross-section; while burdock veterinary medicinal materials must be oval or oblong, with a dark brown or black surface, reticular wrinkles, and a white cross-section. However, in the actual production and circulation of veterinary medicinal materials, affected by factors such as the natural environment, harvesting and processing, transportation and storage, veterinary medicinal materials are often difficult to fully meet quality standards. Common problems include:
[0003] (1) Appearance and shape defects: such as broken, deformed, surface cracks, etc., which affect the integrity and appearance quality of veterinary medicinal materials.
[0004] (2) Abnormal color or smell: If veterinary medicinal materials turn black, change color, or become moldy, they may lose their therapeutic effects.
[0005] (3) Adulteration: For example, mixing in foreign matter such as stones, sand, metal, etc., or passing inferior goods off as good ones or passing off fake goods as the real ones.
[0006] (4) Insects and rat bites: Insects and rats gnaw on veterinary medicinal materials, causing internal damage and loss of nutrients.
[0007] (5) Excessive moisture: If veterinary medicinal materials are damp or moldy, the moisture content exceeds the limit and bacteria are easily bred.
[0008] Traditional veterinary medicinal materials sorting and removal of impurities mainly rely on manual selection. Workers need to rely on certain knowledge and experience in identifying veterinary medicinal materials to inspect veterinary medicinal materials piece by piece, identify their appearance defects, and remove spoiled or mixed veterinary medicinal materials. However, manual sorting and removal of impurities have many limitations: first, the quality judgment standards of veterinary medicinal materials are vague and rely largely on personal experience. The test results vary from person to person and lack objectivity; second, it is difficult for the human eye to detect subtle differences in color and texture of veterinary medicinal materials, and it is easy to miss some hidden defects; third, manual picking is inefficient and difficult to adapt to the rapid development of the veterinary medicinal materials supply chain. In order to improve the quality control level of veterinary medicinal materials, it is necessary to introduce an automated visual inspection system to achieve rapid, accurate, and traceable evaluation of the appearance quality of veterinary medicinal materials through image processing and pattern recognition technology. Summary of the invention
[0009] The first is the acquisition and labeling of veterinary medicinal material image data. We collected images of Chinese veterinary medicinal materials recorded in Part II of the Veterinary Pharmacopoeia of the People's Republic of China (2020 edition), and several experts labeled the types and degrees of quality defects of the images, and used the majority voting method to determine the final labels. Each image of veterinary medicinal materials is labeled as one of the four defect types: intact, abnormal color, mixed and adulterated, and insect-infested and mildewed. Experts also refer to the veterinary drug quality grading standards and give comprehensive scores to samples in terms of color, impurity content, degree of crushing, etc. The scores are between 0 and 5, and the higher the score, the better the quality.
[0010] Next is the preprocessing of the veterinary medicinal material image data. We scale the image so that the short side is no less than 224 pixels, and then crop the largest square image containing the main area of the veterinary medicinal material. In order to improve the generalization of the model, we use random horizontal flipping, random rotation (-10° to 10°), random cropping (0.8-1.0 times) and other methods to expand the sample. At the same time, the three RGB channels of the image are standardized with a mean of 0 and a variance of 1 to accelerate the convergence of the model.
[0011] In order to further expand the sample, we refer to the veterinary medicinal material identification and inspection procedures to artificially synthesize veterinary medicinal material images reflecting different defect characteristics. Different degrees of browning and graying filters are applied to intact samples to obtain samples with abnormal color; sand and stone impurities are randomly added to simulate inclusions and adulteration; insect eye-shaped spots are added locally to generate insect-damaged samples.
[0012] After the above image data collection, annotation, preprocessing, and enhancement, we obtained a veterinary medicinal material image dataset for training and testing models. Next, we built and trained a veterinary medicinal material visual perception model based on this dataset. .
[0013] Will Seamless integration into of In the decoding end, multi-task learning is used to simultaneously predict the defect type and quality score of veterinary medicinal materials.
[0014] Specifically, First, a multi-scale representation is constructed for the input veterinary medicinal material image to make full use of visual information at different scales. The original image is downsampled at three scales to obtain images of three resolutions, which are used to extract overall shape, local texture and detail features respectively.
[0015] At each scale, the image is divided into a grid of fixed size and flattened into a vector. In addition to the image features at three scales, in order to introduce the quality knowledge of veterinary medicinal materials, we defined a shape dictionary and a color dictionary based on the veterinary medicinal material standards and pharmacopoeias, which contain words and their embedded vectors that describe the shape and color of veterinary medicinal materials. The word embedding is spliced with the multi-scale grid features of the image to obtain a multi-scale veterinary medicinal material representation that integrates prior knowledge, and a more semantically rich veterinary medicinal material representation is obtained from the two perspectives of veterinary medicinal material images and prior knowledge.
[0016] Next, the model Input multi-scale grid features of , learning the correlation between local regions. The output is then alternately passed through N and Capturing long-term sequence dependencies on veterinary drug quality.
[0017] The output of The decoder receives this vector and uses two independent , respectively predicting the defect category and quality score of veterinary medicinal materials.
[0018] A multi-task learning strategy is adopted to jointly optimize the two tasks of defect type and quality score.
[0019] Finally, the loss of the defect type task and the loss of the quality scoring task are weighted and summed to obtain the overall loss of multi-task learning. By jointly optimizing this overall loss, the model improves the performance of both veterinary drug defect classification and quality scoring tasks.
[0020] In addition, we have also proposed a veterinary medicinal material sorting and impurity removal device based on visual perception, including: the veterinary medicinal material sorting and impurity removal device based on visual perception includes an electronic storage device and a multi-core electronic computing and processing device, the storage device stores a computer running program for executing the above-mentioned veterinary medicinal material sorting and impurity removal method based on visual perception, and the multi-core electronic computing and processing device can be used to parallelly calculate the computer running program of the veterinary medicinal material sorting and impurity removal method based on visual perception stored in the storage device.
[0021] The veterinary medicinal material sorting and impurity removal device and method based on visual perception proposed by the present invention have the following beneficial technical effects:
[0022] First, the present invention constructs a veterinary medicinal material image dataset that integrates prior knowledge, collects images of various types of Chinese veterinary medicinal materials recorded in the "Pharmacopoeia of the People's Republic of China", and multiple experts annotate the images for defects and quality. The dataset covers a variety of defect conditions such as intact, abnormal color, adulteration, insect infestation, and mildew, and introduces professional terms and word embeddings that describe the shape and color of veterinary medicinal materials. Through operations such as image preprocessing and data enhancement, the richness and distribution balance of the data are further improved. The high-quality dataset lays the foundation for training a robust and generalized veterinary medicinal material quality assessment model.
[0023] Second, the present invention innovatively proposes The model is used for visual perception of veterinary medicinal materials. of Enhanced Variants , solving the pure In the process of capturing the characteristics of veterinary medicinal materials, the model can fully explore the correlation characteristics between different regions and scales, and form a global understanding of veterinary medicinal materials. On the other hand, the model integrates visual information of different granularities through multi-scale representation learning; on the At the same time, a multi-task strategy is used to simultaneously predict defect categories and quality scores. The appearance quality of veterinary medicinal materials is fully and finely characterized, greatly improving the accuracy of sorting and impurity removal. Compared with traditional manual sorting and single visual feature methods, this invention realizes rapid, objective and traceable evaluation of the quality of veterinary medicinal materials. DETAILED DESCRIPTION
[0024] The key to the task of sorting and removing impurities from veterinary medicinal materials is to build a quality assessment model that can objectively quantify the appearance characteristics of veterinary medicinal materials. ,in Indicates The RGB image of a sample of veterinary medicinal materials. The sorting and impurity removal process can be regarded as extracting discriminative representations from the veterinary medicinal materials image. , and then sort out the veterinary medicinal materials that meet the requirements by determining their categories (such as normal, mixed, moldy, etc.) and quality scores (such as excellent, good, unqualified, etc.).
[0025] Preferably, the present invention proposes a method for sorting and removing impurities from veterinary medicinal materials based on visual perception, and the specific method is as follows:
[0026] Get the veterinary medicinal material image dataset, specifically:
[0027] Veterinary medicinal materials image data collection and annotation, including:
[0028] Images of Chinese veterinary medicinal materials recorded in Part II of the Veterinary Pharmacopoeia of the People's Republic of China (2020 edition) were collected as veterinary medicinal material images. After obtaining the veterinary medicinal material images, experts annotated the types and degrees of quality defects of the above images. Each image was independently judged by multiple experts, and its label was determined by majority voting. For controversial samples, all experts reached a consensus through discussion. In the end, each veterinary medicinal material picture was labeled as one of the following four defect types: intact, abnormal color, mixed and adulterated, and insect-infested and mildewed. At the same time, experts referred to the veterinary drug quality grading standards and gave a comprehensive score to the veterinary medicinal material samples in terms of color, impurity content, degree of crushing, etc. The score range is [0, 5]. The higher the score, the better the quality of the veterinary medicinal material.
[0029] Veterinary medicinal materials image data preprocessing, including:
[0030] (1) Scaling and cropping: Scale the image proportionally to a size with the short side no less than 224 pixels, and then crop out the largest square image containing the main area of the veterinary medicinal material.
[0031] (2) Image enhancement: Use random horizontal flipping, random rotation (-10°-10°), random cropping (0.8-1.0 times), etc. to expand the samples and improve the generalization of the model.
[0032] (3) Normalization: The three channels of RGB of the image are normalized to a mean of 0 and a variance of 1 to accelerate model convergence.
[0033] Veterinary medicinal material image data enhancement, including:
[0034] Referring to the identification and inspection procedures for veterinary medicinal materials, artificially synthesize images of veterinary medicinal materials that reflect different defect characteristics. Apply different degrees of browning and graying filters to intact veterinary medicinal material samples to obtain samples with abnormal color; randomly add sand and stone impurities to veterinary medicinal materials to simulate inclusion and adulteration; add insect eye-shaped spots to the parts of veterinary medicinal materials to generate insect-damaged samples.
[0035] After the above-mentioned veterinary medicinal material image data collection and annotation, veterinary medicinal material image data preprocessing and veterinary medicinal material image data enhancement, a veterinary medicinal material image data set is obtained.
[0036] Build on Visual perception model of veterinary medicinal materials .
[0037] In this model, (Encoder) Seamless integration into of In the , the model's ability to capture the long-range dependence of veterinary medicinal materials quality is enhanced; the decoder simultaneously completes the prediction of veterinary medicinal materials defect type and quality score through multi-task learning. Specifically include:
[0038] Multi-scale representation learning for veterinary medicinal material images:
[0039] In order to make full use of the visual information of veterinary medicinal materials images at different scales, the present invention first constructs a multi-scale representation of the input image. Specifically, the original RGB image is downsampled at three scales to obtain images with three resolutions: , , The images at these three scales are used to extract the overall shape, local texture and detail features of veterinary medicinal materials. At each scale, the image is divided into Each grid is normalized to a fixed size and flattened into a 256-dimensional vector. Thus, the veterinary medicinal material image is represented as a set of grid sequences , Indicates that the image is obtained by downsampling at the first scale; Indicates that the image is obtained by downsampling at the second scale; Indicates that the image is obtained by downsampling at the third scale.
[0040] In addition, in order to introduce a priori knowledge of the quality of veterinary medicinal materials, the present invention defines a shape dictionary based on veterinary medicinal material standards and pharmacopoeias and a color dictionary , respectively contain words describing the shape (such as "cylindrical", "oval") and color (such as "yellow-brown", "off-white") of veterinary medicinal materials, and the corresponding embedding vectors , , The number of shapes of medicinal materials for veterinary use, The word embedding vector is concatenated with the multi-scale grid features of the veterinary medicinal material image in the channel dimension to obtain a multi-scale veterinary medicinal material representation that incorporates prior knowledge. :
[0041] ;
[0042] in represents vector concatenation, is the total length of the veterinary medicinal material image sequence.
[0043] Will Integrate into of middle:
[0044] The present invention innovatively Embed in of In order to enhance the model's ability to capture the long-range dependence of veterinary medicinal materials quality. Two enhanced Variant composition:
[0045] (1) :Introduce exponential nonlinear gating function to enhance the ability of memory units to capture long-range information. Time step The update formula is:
[0046] ;
[0047] in represents the candidate memory state, is the current memory cell state, through the forget gate Control the previous moment memory The degree of preservation, input gate Controlling new memories The writing degree is finally updated. . is the current hidden state, through the output gate and exponential function on the memory cell state Transform to obtain. represents element-wise multiplication, and They are hyperbolic tangent and sigmoid activation functions respectively.
[0048] (2) : Using matrix Replacing the scalar memory unit, introducing the memory update in the form of outer product, greatly improves the information storage capacity. Stores a pair of vectors (key Sum ) is updated as:
[0049] ;
[0050] in is the query vector, is the vector dimension. To avoid The present invention also introduces a normalized state For scaling hidden state:
[0051] ;
[0052] Based on the above , The calculation process is as follows:
[0053] (1) Multi-scale grid features of veterinary medicinal materials images enter of , learning the correlation between local regions:
[0054] ;
[0055] in is the hidden layer dimension, Indicates multiple attention.
[0056] (2) Output Through N Layer (alternately used and ), capturing the long sequence dependency of veterinary drug quality:
[0057] ;
[0058] Alternate use and The progressive coding of the layers can make full use of two The complementary advantages of variants can be used to obtain a more comprehensive representation of the characteristics of veterinary medicinal materials.
[0059] Multi-task decoding for defect classification and quality scoring of veterinary medicinal materials:
[0060] Output go through Aggregate into a global veterinary medicine representation vector The decoder receives , through two independent , respectively predict the defect category of veterinary medicinal materials and quality rating :
[0061] ;
[0062] in is the weight matrix and bias for defect classification, is the weight vector and bias of the quality score, is the total number of defect categories. Since the quality assessment of veterinary medicinal materials needs to take into account the two dimensions of defect category and quality score, the present invention adopts a multi-task learning strategy to jointly optimize the following , is the factor used to balance the defect type classification task and the quality score regression task:
[0063] ;
[0064] For the defect type classification task, we use the following :
[0065] ;
[0066] in is the number of training samples, is the total number of defect categories, For the The samples belong to The predicted probability of class defect, is the corresponding true label (the value is 0 or 1).
[0067] For the quality score regression task, we use the following :
[0068] ;
[0069] in For the The prediction quality score of samples, is the corresponding true quality score.
[0070] In summary, the present invention proposes a veterinary medicinal material sorting and impurity removal device and method based on visual perception. The method first constructs a rich, balanced, and prior knowledge-integrated veterinary medicinal material image dataset, and on this basis proposes an innovative A visual perception model for veterinary medicinal materials. As the backbone network, the enhanced Variants Integrated into The method captures the long-term sequence features of veterinary medicinal materials, uses multi-scale representation learning to fully mine visual information of different granularities, and realizes the joint prediction of defect classification and quality score of veterinary medicinal materials through multi-task learning. The invention overcomes the subjectivity and inefficiency of traditional manual sorting, and also makes up for the shortcomings of the existing single visual feature method, providing a fast and accurate automated solution for the quality control of veterinary medicinal materials.
Claims
1. A method for sorting and removing impurities from veterinary medicinal materials based on visual perception, characterized in that: The method includes acquiring and annotating a veterinary medicinal material image data set, preprocessing the veterinary medicinal material image data, enhancing the veterinary medicinal material image data, and constructing and training a veterinary medicinal material visual perception model. , the construction and training of a veterinary medicine visual perception model Specifically include The model is seamlessly integrated into In the encoder of the model, at the decoding end, multi-task learning is used to simultaneously predict the defect type and quality score of veterinary medicinal materials; Specifically, The model first constructs a multi-scale representation of the input veterinary medicinal material image, making full use of visual information at different scales, including downsampling the original image at three scales to obtain images of three resolutions, which are used to extract overall shape, local texture and detail features respectively; At each scale, the image is divided into The veterinary medicine image is represented as a set of grid sequences. ; Indicates that the image is obtained by downsampling at the first scale; Indicates that the image is obtained by downsampling at the second scale; Indicates that the image is obtained by downsampling at the third scale; Based on the shape dictionary and color dictionary defined in veterinary medicinal material standards and pharmacopoeias, we introduce prior knowledge of veterinary medicinal material quality, including words describing the shape and color of veterinary medicinal materials and their embedded vectors. We then concatenate the word embeddings with the multi-scale grid features of the image to obtain a multi-scale veterinary medicinal material representation that incorporates prior knowledge. Next, The encoder of the model inputs multi-scale grid features Model , learning the correlation between local regions, The output of and Capturing the long-term sequence dependence of veterinary drug quality; for The multi-head self-attention layer of the model, and for Variants of the model, exist Based on the introduction of index gating and storage mixing; Will The model's memory unit is expanded from a scalar to a matrix, and a covariance update rule is introduced; The output of the encoder passes through The decoder receives this vector and uses two independent , respectively predict the defect category and quality score of veterinary medicinal materials, the is a densely connected layer; A multi-task learning strategy is used to jointly optimize the two tasks of veterinary medicinal material defect types and veterinary medicinal material quality scores; Finally, the loss of the veterinary medicine defect type task and the loss of the veterinary medicine quality scoring task are weighted and summed to obtain the overall loss of multi-task learning by jointly optimizing the overall loss: ;in Loss function for defect type classification task; Loss function for quality score regression task; is the number of training samples, is the total number of defect categories, For the The samples belong to The predicted probability of class defect, is the corresponding true label; For the The prediction quality score of samples, is the corresponding true quality score, is a factor used to balance the defect type classification task and the quality score regression task.
2. A method for sorting and removing impurities from veterinary medicinal materials based on visual perception as claimed in claim 1, characterized in that: The veterinary medicinal material image data preprocessing includes: scaling the veterinary medicinal material image in proportion so that the short side is not less than 224 pixels, then cropping out the largest square image containing the main area of the veterinary medicinal material, using random horizontal flipping, random rotation from -10° to 10°, and random cropping 0.8-1.0 times to expand the sample, and standardizing the three RGB channels of the image with a mean of 0 and a variance of 1.
3. A method for sorting and removing impurities from veterinary medicinal materials based on visual perception as claimed in claim 2, characterized in that: The multi-scale veterinary medicinal material representation integrating prior knowledge is specifically as follows: a shape dictionary is defined based on veterinary medicinal material standards and pharmacopoeias. and a color dictionary , respectively contain words describing the shape and color of veterinary medicinal materials, and the corresponding embedding vectors , , The number of shapes of medicinal materials for veterinary use, is the number of colors of veterinary medicinal materials; the word embedding vector is spliced with the multi-scale grid features of the veterinary medicinal material image in the channel dimension to obtain a multi-scale veterinary medicinal material representation that integrates prior knowledge : ； in represents vector concatenation, is the total length of the veterinary medicinal material image sequence.
4. A veterinary medicinal material sorting and impurity removal device based on visual perception, characterized in that: The veterinary medicinal material sorting and impurity removal equipment based on visual perception includes an electronic storage device and a multi-core electronic computing and processing device. The storage device stores a computer running program. The multi-core electronic computing and processing device can be used to parallelly calculate the computer running program stored in the storage device to implement the method described in any one of claims 1-3.