A deep learning-based multispectral fusion traditional Chinese medicine quality grading method and system

Abstract

The application discloses a kind of multi-spectral fusion traditional Chinese medicine quality grading method and system based on deep learning, including the X-ray image of the sample to be measured is collected, the X-ray image of the sample to be measured is input into first model to obtain the traditional Chinese medicine and impurity information in the sample to be measured;Visible light image of the sample to be measured is collected, the visible light image of the sample to be measured is time-stamped marked, based on the visible light image of the sample to be measured after marking and the traditional Chinese medicine and impurity information in the sample to be measured, the position data of traditional Chinese medicine and impurity in the sample to be measured is calculated;Based on the position data of traditional Chinese medicine and impurity in the sample to be measured, the impurity in the sample to be measured is eliminated;Near infrared hyperspectral image of the sample to be measured is collected, and the near infrared hyperspectral image of the sample to be measured is input into second model to obtain the quality grading of the sample to be measured, and the second model is the deep learning model trained by inputting near infrared hyperspectral image and its corresponding artificial marking quality grading.

Description

Technical Field

[0001] This invention relates to the field of traditional Chinese medicine identification, and in particular to a method and system for quality grading of traditional Chinese medicine based on deep learning and multispectral fusion. Background Technology

[0002] Traditional Chinese medicine (TCM) decoction pieces are TCM raw materials that have been processed according to TCM theory and processing methods and can be directly used in clinical practice. The quality of TCM decoction pieces directly affects the treatment effect. TCM comes from a wide range of sources, and the impurities it contains have a significant impact on quality. Therefore, by using positioning technology and multi-spectral fusion technology to locate impurities on the conveyor belt, it is easier to remove impurities in subsequent processes.

[0003] The difference in grayscale values ​​of X-ray images is related to the degree of X-ray attenuation. X-rays are attenuated when passing through traditional Chinese medicine, and the degree of attenuation depends on the thickness and density of the medicine. X-ray images can quickly and non-destructively obtain information about the internal structure of an object.

[0004] Near-infrared hyperspectral images are three-dimensional data. Near-infrared hyperspectral image data includes the spatial relationship corresponding to each wavelength in the near-infrared band. It combines imaging technology and spectral detection technology. While imaging the spatial features of the target being analyzed, it forms tens or even hundreds of narrow bands for each pixel through dispersion to achieve continuous spectral coverage. The image information can reflect the external quality characteristics of the sample, such as size, shape, and defects, while the spectral information can fully reflect the internal structure of the sample, such as molecular composition and quality components.

[0005] In order to quickly identify and locate impurities in samples and remove them for further quality grading of traditional Chinese medicine, this invention combines X-ray images, visible light images, and near-infrared hyperspectral images to analyze samples and proposes a multispectral fusion method for quality grading of traditional Chinese medicine based on deep learning. Summary of the Invention

[0006] This invention provides a method and system for quality grading of traditional Chinese medicine based on deep learning and multispectral fusion. By inputting X-ray images into an X-ray image model to calculate the information of traditional Chinese medicine and impurities in the sample to be tested, and combining the output results of the X-ray image model with a time-stamped visible light image to locate impurities, the quality grading is then performed through deep learning of near-infrared hyperspectral images, thereby realizing the removal of impurities and quality grading of the sample to be tested.

[0007] According to a first aspect of the present invention, a multispectral fusion traditional Chinese medicine based on deep learning is provided.

[0008] The quality grading method involves acquiring X-ray images of the sample to be tested, inputting these images into a first model to calculate information on the traditional Chinese medicine and impurities in the sample, and acquiring visible light images of the sample, recording the time when the sample arrives at the visible light image acquisition point as... Based on the visible light image of the sample to be tested and the information of the traditional Chinese medicine and impurities in the sample, the location data of the traditional Chinese medicine and impurities in the sample to be tested are calculated. The distance between the X-ray image acquisition point and the visible light image acquisition point is... The error distance is , The speed of the conveyor belt transporting the sample to be tested is Let the error time be denoted as Based on the location data of the Chinese medicine and impurities in the sample to be tested, impurities in the sample to be tested are removed, including: recording the time when the impurities in the sample to be tested pass through the visible light image acquisition point. After Arriving at the removal point in seconds, among which, , To remove impurities by eliminating the distance from the point of removal to the X-ray image acquisition point, near-infrared hyperspectral images of the sample to be tested are acquired. These images are then input into a second model to calculate the quality grade of the sample. The second model is a deep learning model trained on the input near-infrared hyperspectral images and their corresponding manually labeled quality grades. This model uses the output of a visible light image with timestamps combined with an X-ray image to identify and locate traditional Chinese medicine and impurities in the sample. Impurities are then removed based on their location information. Subsequently, deep learning of the near-infrared hyperspectral images is used to grade the quality of the traditional Chinese medicine in the sample. The traditional Chinese medicine is then classified based on its location and the quality grade results. It should be noted that there are many models in the field of deep learning, such as commonly used deep learning models and architectures including CNN, DBN, RNN, RNTN, autoencoders, and GAN. This invention only requires selecting a deep learning model that can input a training set with quality grade labels and output quality grade results after training. The training process of the deep learning model and architecture will not be elaborated here.

[0009] In some implementations, the deep learning model of near-infrared hyperspectral images can not only achieve quality grading based on the quality of artificially labeled materials, but also be trained based on artificially labeled Chinese medicines and impurities to achieve further identification of Chinese medicines and impurities.

[0010] In some implementations, the first model is an X-ray image model, which includes convolutional layers, activation functions, batch normalization layers, pooling layers, and discard layers.

[0011] In some implementations, the Rectified Linear Unit (ReLU) is used as the activation function in the X-ray image model, the Softmax activation function is used to solve the binary classification problem, and the sparse classification cross-entropy loss function is used to train the network. Compared with traditional activation functions, the ReLU is more efficient in gradient descent and backpropagation: it avoids the gradient explosion and gradient vanishing problems, and simplifies the calculation process: it is free from the influence of other complex activation functions such as exponential functions; at the same time, the dispersion of activity reduces the overall computational cost of the neural network.

[0012] In some embodiments, the X-ray image model is trained using the following method: acquiring X-ray images of several training samples, including traditional Chinese medicine samples and impurity samples; performing geometric transformations on the X-ray images of the training samples to obtain an X-ray image dataset; and inputting the X-ray image dataset into the X-ray image model for training iterations. The geometric transformations can include translation, scaling, rotation, affine transformation, and perspective transformation of the X-ray images. The purpose is to expand the X-ray image dataset based on a certain number of training samples. It should be noted that the size of the dataset affects the training accuracy and training time. Those skilled in the art can select the number of training samples according to the requirements of complexity, training time, and accuracy.

[0013] In some implementations, the X-ray image dataset is divided into a training set, a validation set, and a test set according to a predetermined ratio.

[0014] In some implementations, inputting the X-ray image dataset into the X-ray image model for training iterations further includes performing three convolutions on the X-ray images in the X-ray image dataset in convolutional layers to obtain convolutional feature maps. Specifically: the X-ray image is convolved through a first convolutional layer to obtain a first convolutional feature map; the first convolutional feature map is convolved through a second convolutional layer to obtain a second convolutional feature map; and the second convolutional feature map is convolved through a third convolutional layer to obtain a third convolutional feature map. The convolution process of the X-ray image is to perform convolution operations on the input image using a set of learnable filters (also called convolutional kernels) to extract local features of the image. Each filter slides across the entire image and performs convolution operations on local regions to obtain a feature map. By using different filters, the three convolutions mainly consider the width, height, and depth of the X-ray image of the sample to be tested. Therefore, for traditional Chinese medicine materials, three convolutions of the X-ray image are sufficient to extract feature maps of the X-ray image of the sample to be tested that express feature information.

[0015] In some implementations... It takes 2ms.

[0016] According to a second aspect of the present invention, a deep learning-based multispectral fusion traditional Chinese medicine quality grading system is provided, comprising a back-end processing device, a conveyor belt, and an X-ray detector, a visible light acquisition device, and a hyperspectral acquisition device arranged sequentially along the transport direction of the sample to be tested, as well as a rejection device and a classifier. The back-end processing device is used to execute the method described above. The X-ray detector, the visible light acquisition device, and the hyperspectral acquisition device are respectively connected to the back-end processing device via data lines. The rejection device is used to remove impurities from the sample to be tested. The classifier classifies the traditional Chinese medicine in the sample to be tested based on the quality grading result output by the near-infrared hyperspectral image model.

[0017] In some embodiments, the hyperspectral acquisition device includes a line-scan hyperspectral camera and a light source positioned below the line-scan hyperspectral camera.

[0018] In some embodiments, the rejection device includes a first rejection device disposed between the visible light acquisition device and the hyperspectral acquisition device, used to reject the test sample that is simultaneously identified as an impurity by both the X-ray detector and the visible light device. The rejection device also includes a second rejection device disposed along the conveyor belt conveying direction after the hyperspectral device, used to reject impurities in the test sample. It should be noted that the second rejection device is only needed when the near-infrared hyperspectral image is also used to determine the traditional Chinese medicine and impurities in the test sample.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] This invention uses a combination of X-ray and visible light images to calculate information about traditional Chinese medicine (TCM) and impurities in a sample in a dual-channel manner. It also locates the TCM and impurities, which is then used for subsequent impurity removal. Furthermore, it enables quality grading of the remaining TCM based on deep learning from near-infrared hyperspectral images. This method and system accurately identify and locate TCM and impurities, ensuring a high impurity removal rate in the sample, while also classifying the remaining TCM according to quality, thus improving the speed and quality of TCM raw material screening. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the deep learning-based multispectral fusion method for grading the quality of traditional Chinese medicine provided by the present invention.

[0022] Figure 2 This is a schematic diagram of the structure of the deep learning-based multispectral fusion traditional Chinese medicine quality grading system provided by the present invention.

[0023] Figure label:

[0024] 1. Back-end processing unit; 2. X-ray detector; 3. Visible light acquisition equipment; 4. Spectral image acquisition equipment; 5. Light source; 6. Conveyor belt. Detailed Implementation

[0025] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “described,” and “the” as used in the invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0027] It should be understood that although the terms "first," "second," "third," etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another.

[0028] Unless otherwise specified, the methods and equipment used in this invention are all conventional reagents, methods and equipment in this technical field.

[0029] In existing technologies, the identification of impurities in Chinese medicinal materials is generally based on the analysis of X-ray images or near-infrared hyperspectral images. However, existing technologies cannot achieve both precise removal of impurities and quality grading simultaneously. Therefore, to address the shortcomings of existing technologies, this invention proposes a method and system for quality grading of Chinese medicinal materials based on deep learning and multispectral fusion. To facilitate understanding of the embodiments of this invention, the deep learning-based multispectral fusion system for quality grading of Chinese medicinal materials provided by this invention will be introduced first, followed by a further introduction of the deep learning-based multispectral fusion method for quality grading of Chinese medicinal materials in conjunction with the system.

[0030] The present invention will now be described in further detail with reference to the accompanying drawings.

[0031] Combination Figure 2This invention provides a deep learning-based multispectral fusion traditional Chinese medicine quality grading system. The deep learning-based multispectral fusion traditional Chinese medicine quality grading system includes a back-end processing device 1, a conveyor belt 6, and X-ray detector 2, visible light acquisition device 3 and hyperspectral acquisition device 4 arranged sequentially along the sample transport direction, as well as a rejection device (not shown) and a classifier (not shown).

[0032] The back-end processing device 1 is used to execute a deep learning-based multispectral fusion method for grading the quality of traditional Chinese medicine. Specifically, this method is as follows:

[0033] An X-ray image of the sample to be tested is acquired, and the X-ray image of the sample to be tested is input into a first model to calculate the information of traditional Chinese medicine and impurities in the sample to be tested.

[0034] A visible light image of the sample to be tested is acquired, and the visible light image of the sample to be tested is timestamped. Based on the timestamped visible light image of the sample to be tested and the information of traditional Chinese medicine and impurities in the sample to be tested, the position data of traditional Chinese medicine and impurities in the sample to be tested is calculated. Based on the position data of traditional Chinese medicine and impurities in the sample to be tested, the impurities in the sample to be tested are removed.

[0035] Near-infrared hyperspectral images of the sample to be tested are acquired. The near-infrared hyperspectral images of the sample to be tested are input into a second model to calculate the quality grade of the sample to be tested. The second model is a deep learning model trained by inputting the near-infrared hyperspectral images and their corresponding artificially labeled quality grades.

[0036] The X-ray detector 2 is used to acquire X-ray images of the sample (not shown) on the conveyor belt 6, the visible light acquisition device 3 is used to acquire visible light images of the sample on the conveyor belt 6, and the hyperspectral acquisition device 4 is used to acquire near-infrared hyperspectral images of the sample on the conveyor belt 6.

[0037] The X-ray detector 2, visible light acquisition device 3, and hyperspectral acquisition device 4 are respectively connected to the back-end processing device 1 via data cable 7. The rejection device is used to remove impurities in the sample to be tested. The classifier classifies the traditional Chinese medicine in the sample to be tested based on the quality grading results output by the near-infrared hyperspectral image model.

[0038] This deep learning-based multispectral fusion traditional Chinese medicine (TCM) quality grading system comprehensively acquires X-ray, visible light, and near-infrared hyperspectral images of the sample under test using three image acquisition devices. The back-end processing device 1 uses a dual-channel approach to identify TCM and impurities in the sample. Furthermore, it calculates the location of TCM and impurities using timestamps and computational methods, providing locational data support for subsequent impurity removal and TCM quality grading. This enables rapid identification and removal of TCM and impurities, while simultaneously achieving quality grading.

[0039] In a preferred embodiment of the present invention, the hyperspectral acquisition device 4 includes a line-scan hyperspectral camera and a light source 5 disposed under the line-scan hyperspectral camera.

[0040] In a preferred embodiment of the present invention, the rejection device includes a first rejection device disposed between the visible light acquisition device 3 and the hyperspectral acquisition device 4, used to reject the sample to be tested that is simultaneously identified as an impurity by both the X-ray detector and the visible light device. Of course, the first rejection device can also be disposed after the hyperspectral acquisition device 4. The rejection device can also include a second rejection device disposed along the conveyor belt direction after the hyperspectral device, used to reject impurities in the sample to be tested. It should be noted that the second rejection device is only needed when the near-infrared hyperspectral image is also used to determine the traditional Chinese medicine and impurities in the sample to be tested. The dual rejection device can further ensure the rejection rate of impurities.

[0041] Next, combined Figure 1 This paper introduces a deep learning-based multispectral fusion method for quality grading of traditional Chinese medicine. The method specifically includes:

[0042] S100, acquires X-ray images of the sample to be tested;

[0043] In practical applications, the sample to be tested can be placed on the conveyor belt 6 of the deep learning-based multispectral fusion traditional Chinese medicine quality grading system. When the sample to be tested arrives at the acquisition point of the X-ray detector 2, the X-ray detector is activated to acquire the X-ray image of the sample to be tested.

[0044] S200, input the X-ray image of the sample to be tested into the first model to calculate the information of traditional Chinese medicine and impurities in the sample to be tested;

[0045] Specifically, in this embodiment, the first model is an X-ray image model, which includes a convolutional layer, an activation function, a batch normalization layer, a pooling layer, and a discard layer; the number of output categories of the X-ray image model is set to 2, and the output values ​​are 0 representing traditional Chinese medicine and 1 representing impurities.

[0046] S300, acquires visible light images of the sample to be tested;

[0047] In practical applications, when the sample to be tested is transported by the conveyor belt 6 to the collection point of the visible light collection device 3, the visible light collection device is activated to collect the visible light image of the sample to be tested.

[0048] S400, timestamp the visible light image of the sample to be tested, and record the time when the sample to be tested arrives at the visible light image acquisition point as... Based on the visible light image of the sample to be tested and the information of traditional Chinese medicine and impurities in the sample to be tested, the position data of traditional Chinese medicine and impurities in the sample to be tested are calculated.

[0049] Specifically, while acquiring visible light images of the sample to be tested, the visible light images are timestamped. By recognizing the contours of the visible light images and corresponding with the X-ray images, it can be determined whether the sample to be tested at that time is a traditional Chinese medicine or an impurity. At the same time, the position data of the sample to be tested corresponding to the visible light image at that time is calculated by combining the timestamp.

[0050] S500, based on the location data of Chinese medicine and impurities in the sample to be tested, remove the impurities in the sample to be tested;

[0051] Specifically, when impurities in the sample to be tested are transported to the rejection point via conveyor belt 6, the rejection device performs a rejection action on the impurities.

[0052] S600, acquires near-infrared hyperspectral images of the sample to be tested;

[0053] Specifically, when the sample to be tested arrives at the acquisition point of the hyperspectral acquisition device via conveyor belt 6, the hyperspectral acquisition device is activated to acquire a near-infrared hyperspectral image of the sample to be tested.

[0054] S700, The near-infrared hyperspectral image of the sample to be tested is input into the second model to calculate the quality grade of the sample to be tested. The second model is a deep learning model trained by inputting the near-infrared hyperspectral image and its corresponding artificially labeled quality grade.

[0055] It should be noted that there are many models in the field of deep learning, such as commonly used deep learning models and architectures including CNN, DBN, RNN, RNTN, autoencoders, GAN, etc. This invention only requires selecting a deep learning model that can take a training set with quality grading labels as input and output quality grading results after training. The training process of deep learning models and architectures will not be elaborated here. The quality grading labels are based on different Chinese medicines, which are labeled and graded according to the components and their contents associated with the quality grade. Deep learning analyzes the components in the hyperspectral image based on the labels to achieve the final grading effect. Those skilled in the art should understand which components are used to grade the quality of different Chinese medicines, which will not be elaborated here.

[0056] In a preferred embodiment of the present invention, some specific settings of the X-ray image model in S200 include: a rectified linear unit (ReLU) is used as the activation function in the X-ray image model; a softmax activation function is used to solve the binary classification problem; and a sparse classification cross-entropy loss function is used to train the network. Compared with traditional activation functions, the rectified linear unit is more efficient in gradient descent and backpropagation, avoiding gradient explosion and gradient vanishing problems. In addition, it simplifies the calculation process by eliminating the influence of other complex activation functions such as exponential functions. At the same time, the dispersion of activity reduces the overall computational cost of the neural network.

[0057] In a preferred embodiment of the present invention, the X-ray image model in S200 is trained using the following method: X-ray images of several training samples are acquired, including traditional Chinese medicine samples and impurity samples; geometric transformations are performed on the X-ray images of the training samples to obtain an X-ray image dataset; the X-ray image dataset is input into the X-ray image model for training iterations. The geometric transformations can include translation, scaling, rotation, affine transformation, and perspective transformation of the X-ray images. The purpose is to expand the X-ray image dataset based on a certain number of training samples. It should be noted that the size of the dataset affects the training accuracy and training time. Those skilled in the art can select the number of training samples according to the requirements of complexity, training time, and accuracy.

[0058] In a preferred embodiment of the present invention, the above-mentioned X-ray image dataset is divided into a training set, a validation set, and a test set in a ratio of 7:2:1. Of course, this ratio can also be changed by those skilled in the art according to the requirements of complexity, training time, and accuracy.

[0059] In a preferred embodiment of this invention, the above-mentioned inputting the X-ray image dataset into the X-ray image model for training iterations further includes performing three convolutions on the X-ray images in the X-ray image dataset in a convolutional layer to obtain convolutional feature maps. Specifically: the X-ray images are convolved through a first convolutional layer to obtain a first convolutional feature map; the first convolutional feature map is convolved through a second convolutional layer to obtain a second convolutional feature map; and the second convolutional feature map is convolved through a third convolutional layer to obtain a third convolutional feature map. The convolution process of the X-ray images involves using a set of learnable filters (also called convolutional kernels) to perform convolution operations on the input image, thereby extracting local features of the image. Each filter slides across the entire image and performs convolution operations on local regions to obtain a feature map. By using different filters, the three convolutions mainly consider the width, height, and depth of the X-ray image of the sample to be tested. Therefore, for traditional Chinese medicine materials, three convolutions of the X-ray image are sufficient to extract feature maps of the X-ray image of the sample to be tested that express feature information.

[0060] In a preferred embodiment of the present invention, the method for obtaining the positional data of traditional Chinese medicine and impurities in the sample to be tested in step S400 is as follows: The speed of the conveyor belt transporting the sample to be tested is denoted as v; the time when the sample to be tested arrives at the X-ray image acquisition point is denoted as t1; the time when the sample to be tested arrives at the visible light image acquisition point is denoted as t2; and the distance between the X-ray image acquisition point and the visible light image acquisition point is denoted as s. Since there may be slight slippage between the sample to be tested and the conveyor belt during transport, an error distance needs to be denoted as s. Let the error time be denoted as Among them, let If it is 2ms, then The location data of the Chinese medicine and impurities in the sample when they arrive at the visible light image acquisition point. ;

[0061] In a preferred embodiment of the present invention, the method for removing impurities from the sample to be tested in step S500 is as follows: Let m be the distance from the removal point to the X-ray image acquisition point. When the impurity in the sample to be tested arrives at the removal point x seconds after passing the visible light image acquisition point, the impurity is removed. It should be noted that the time x for the impurity to reach the rejection point can be calculated based on t1 or t2.

[0062] In a preferred real-time mode of this invention embodiment, step S700 further includes preprocessing the acquired infrared hyperspectral image, including black and white correction, and extracting spectral features from the spectrum of overlapping absorption peaks formed by the background and the labeled components using chemometric methods.

[0063] This invention provides a deep learning-based multispectral fusion method and system for quality grading of traditional Chinese medicine (TCM). It uses X-ray images combined with visible light images to calculate information about TCM and impurities in the sample in a dual-channel format, and locates the TCM and impurities. This location is used for subsequent impurity removal, and the remaining TCM is graded based on deep learning using near-infrared hyperspectral images. This method and system accurately identify and locate TCM and impurities, ensuring a high impurity removal rate in the sample, while also classifying the remaining TCM according to quality, thus improving the screening speed and quality of TCM raw materials.

[0064] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep learning-based multispectral fusion traditional Chinese medicine quality grading method, characterized in that, X-ray images of a to-be-tested sample are collected, and information of traditional Chinese medicine and impurities in the to-be-tested sample is calculated by inputting the X-ray images of the to-be-tested sample into a first model; A visible light image of the sample to be tested is collected, and the time when the sample to be tested reaches the visible light image collection point is recorded as The position data of the traditional Chinese medicine and impurities in the sample to be tested is calculated based on the visible light image of the sample to be tested and the information of the traditional Chinese medicine and impurities in the sample to be tested , wherein the distance between the X-ray image collection point and the visible light image collection point is , the error distance is , The transmission speed of the transmission belt of the sample to be tested is , and the error time is ; The impurities in the sample to be tested are removed based on the position data of the traditional Chinese medicine and the impurities in the sample to be tested, comprising: recording the time when the impurities in the sample to be tested pass through the visible light image acquisition point seconds later reach the removal point, wherein , is the distance from the removal point to the X-ray image acquisition point, and the impurities are removed; near-infrared hyperspectral images of the to-be-tested sample are collected, and quality grading of the to-be-tested sample is calculated by inputting the near-infrared hyperspectral images of the to-be-tested sample into a second model, wherein the second model is a deep learning model trained by inputting near-infrared hyperspectral images and corresponding artificial labeled quality grading.

2. The deep learning-based multi-spectral fusion traditional Chinese medicine quality grading method according to claim 1, characterized in that, The first model is an X-ray image model, and the X-ray image model comprises a convolution layer, an activation function, a batch normalization layer, a pooling layer and a discard layer.

3. The deep learning-based multi-spectral fusion traditional Chinese medicine quality grading method according to claim 2, characterized in that, The first model is trained by the following method: collecting X-ray images of a plurality of training samples, wherein the plurality of training samples comprise traditional Chinese medicine samples and impurity samples, performing geometric transformation processing on the X-ray images of the plurality of training samples to obtain an X-ray image dataset, and inputting the X-ray image dataset into the X-ray image model for training iteration.

4. The deep learning-based multi-spectral fusion traditional Chinese medicine quality grading method according to claim 3, characterized in that, The X-ray image dataset is divided into a training set, a validation set and a test set according to a predetermined proportion.

5. The deep learning-based multi-spectral fusion traditional Chinese medicine quality grading method according to claim 3, characterized in that, The X-ray image dataset is input into the X-ray image model for training iteration, which further comprises performing three convolutions on the X-ray images in the X-ray image dataset in the convolution layer to obtain a convolution feature map.

6. The deep learning-based multi-spectral fusion traditional Chinese medicine quality grading method according to claim 1, characterized in that, Error time is 2 ms.

7. A deep learning-based multispectral fusion traditional Chinese medicine quality grading system, characterized in that, It comprises a backend processing device, a conveyor belt, and an X-ray detector, a visible light acquisition device and a hyperspectral acquisition device arranged in sequence along the conveying direction of the to-be-tested sample, and a rejection device and a classifier, wherein the backend processing device is used to execute the method of any one of claims 1-6, the X-ray detector, the visible light acquisition device and the hyperspectral acquisition device are respectively connected to the backend processing device through data lines, the rejection device is used to reject impurities in the to-be-tested sample, and the classifier classifies traditional Chinese medicine in the to-be-tested sample based on the quality grading result output by the near-infrared hyperspectral image model.

8. The deep learning-based multi-spectral fusion traditional Chinese medicine quality grading system according to claim 7, characterized in that, The hyperspectral acquisition device comprises a line-scan hyperspectral camera and a light source arranged below the line-scan hyperspectral camera.

9. The deep learning-based multi-spectral fusion traditional Chinese medicine quality grading system according to claim 7, characterized in that, The rejection device comprises a first rejection device arranged between the visible light acquisition device and the hyperspectral acquisition device, which is used to reject the to-be-tested sample confirmed as impurities by the X-ray detector and the visible light device in the double channel, and the rejection device further comprises a second rejection device arranged behind the hyperspectral device along the conveying direction of the conveyor belt, which is used to reject impurities in the to-be-tested sample.

Images