A baizhu softening endpoint discrimination method and system based on multi-modal data fusion and deep learning

Abstract

The application discloses a white atractylodes rhizome softening endpoint discrimination method and system based on multi-modal data fusion and deep learning, adopts a Fourier transform near-infrared spectrometer, a low-field nuclear magnetic resonance instrument and a texture analyzer to perform multi-time point synchronous collection on a white atractylodes rhizome softening process, acquires data, and performs pretreatment after time sequence alignment and cleaning. After the above data is extracted and features are spliced into a combined feature vector, the combined feature vector is input into a convolutional neural network branch to extract a first feature vector; the nuclear magnetic resonance image is input into a residual neural network to extract a second feature vector; after the two feature vectors are fused, feature reweighting is performed through an SE-Block attention module, and finally, a regression layer is input to output a softening index S. The application eliminates false softening signals of single dimension discrimination, realizes a change from a traditional Chinese medicine processing black box operation to transparent production, and provides an objective technical means for traditional Chinese medicine decoction piece quality control.

Description

Technical Field

[0001] This invention relates to the field of traditional Chinese medicine quality control technology, specifically to a method and system for determining the softening endpoint of Atractylodes macrocephala based on multimodal data fusion and deep learning. Background Technology

[0002] Atractylodes macrocephala Koidz., a commonly used traditional Chinese medicine, is widely used in the preparation of prepared Chinese medicines and the processing of processed Chinese medicinal herbs. During the processing and clinical handling of Atractylodes macrocephala, it is usually necessary to thoroughly moisten or soften it to reduce its hardness, improve its slicing properties, and thus ensure that the slices are of uniform thickness and have intact cross-sections, meeting the requirements of subsequent processing or preparation techniques.

[0003] During the softening process of Atractylodes macrocephala, as external moisture gradually penetrates into the herb, its tissue structure and mechanical properties undergo significant changes. The internal moisture gradually transforms from a bound state to a semi-bound and free state, while the intercellular spaces widen, resulting in a decrease in overall hardness. This softening process has distinct stages; insufficient softening leads to difficulty in cutting and easy cracking; excessive softening can cause tissue collapse, loss of active ingredients, or increased processing losses.

[0004] However, in current TCM processing practices, there is a lack of unified and objective evaluation standards for the softening endpoint of Atractylodes macrocephala. The relevant judgment mainly relies on the experience of operators, making it difficult to achieve quantitative control. When making manual judgments, relying solely on the surface moisture state or the feel of the slices can easily lead to a false softening phenomenon of "soft on the outside and hard on the inside," resulting in a dry center and a high rate of broken pieces during cutting. Some studies use single detection methods, such as using only near-infrared spectroscopy or only texture analysis to establish a discriminant model. However, due to the large differences in the origin, specifications, and density of Atractylodes macrocephala, the model has poor generalization ability across different batches and requires frequent recalibration. Furthermore, there is a lack of multi-dimensional data fusion and dynamic tracking mechanisms for the softening process, making it impossible to provide real-time feedback on moisture migration and tissue changes. Summary of the Invention

[0005] To address the shortcomings of existing technologies mentioned above, this invention provides a method and system for determining the softening endpoint of Atractylodes macrocephala based on multimodal data fusion and deep learning, thereby overcoming the aforementioned technical problems in related existing technologies.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: A method for determining the softening endpoint of Atractylodes macrocephala based on multimodal data fusion and deep learning includes the following steps: S1: At multiple time points during the softening process of Atractylodes macrocephala, Fourier transform near-infrared spectrometer, low-field nuclear magnetic resonance spectrometer and texture analyzer were used to detect the Atractylodes macrocephala samples, and near-infrared spectral data, transverse relaxation time data, nuclear magnetic resonance imaging data and texture characteristic data were acquired respectively, and time sequence alignment was performed, and the aligned data were cleaned. S2: The near-infrared spectral data, transverse relaxation time data, and texture property data after cleaning are preprocessed. Features are extracted from the preprocessed data and spliced ​​to obtain a combined feature vector. The nuclear magnetic resonance imaging data is preprocessed to obtain a nuclear magnetic resonance image. S3: Construct and train a trimodal deep learning fusion model. The trimodal deep learning fusion model includes a convolutional neural network branch and a residual neural network. The convolutional neural network branch is used to process the combined feature vectors and output the first feature vector, and the residual neural network is used to process the MRI images and output the second feature vector. S4: The first feature vector and the second feature vector are fused together, and the fused feature vector is input into the attention module for feature reweighting to obtain a weighted feature vector; S5: Input the weighted feature vector into the regression layer and output the softening index S. When S is greater than the set threshold, it is determined that Atractylodes macrocephala has reached the softening endpoint.

[0007] Preferably, in step S1, the acquisition of near-infrared spectral data includes: placing the Atractylodes macrocephala sample to be tested under a diffuse reflectance probe, and setting the spectral scanning range to 4000-12000 cm⁻¹. -1 The resolution is 8cm. -1 Each spectrum was scanned a total of 32 times, and each sample was measured in parallel 3 times and the average value was taken. The acquisition of transverse relaxation time data included: moving the Atractylodes macrocephala sample to be tested into the center of the MRI probe coil, acquiring proton signals using the Carr-Purcell-Meiboom-Gill sequence, with an echo number of 8000, a signal receiving bandwidth of 100kHz, a waiting time of 5000ms, 4 scans, and an echo time of 0.30ms; after acquiring the transverse relaxation decay data, the transverse relaxation time spectrum was obtained by analyzing the SIRT mathematical inversion model. The acquisition of the nuclear magnetic resonance imaging data included: obtaining magnetic resonance images of the cross-section of Atractylodes macrocephala based on a multi-slicepinechoes sequence, with a slice width of 2.0 mm, a slice spacing of 1.0 mm, a slice number of 5, a time echo of 20 ms, a readout size of 256, a phase size of 192, a repetition time of 1000 ms, a flip angle of 90°, a refocusing flip angle of 180°, an average value of 4, an RG of 20 dB, a PRG of high, and a FOV of 100 mm × 100 mm.

[0008] Preferably, in step S1, the acquisition of textural property data includes: using an analyzer to puncture the Atractylodes macrocephala sample to be tested, with a trigger point load of 5g, a weighing sensor of 10000g, a return speed of 1mm / s, a test speed of 1mm / s, a puncture depth of 6mm, a data frequency of 50 points / s, and taking the average value after three consecutive measurements.

[0009] Preferably, in step S2, the preprocessing includes: truncating the near-infrared spectral data to 4000-10000 cm⁻¹. -1 After band normalization, the hardness, cohesion, elasticity and resilience parameters of the texture property data are extracted and normalized to the 0-1 range; the relaxation time distribution characteristics of the transverse relaxation time data are extracted; and the nuclear magnetic resonance imaging data are adjusted to a grayscale image of 224×224×1.

[0010] Preferably, the formula for concatenating the combined feature vectors is: V combined =Concat(V NIR V TPA V T2 ) Among them, V NIR V represents the near-infrared spectral eigenvector. TPA V represents the eigenvector of texture properties. T2 This represents the lateral relaxation time feature vector, and Concat represents the concatenation operation in the channel dimension.

[0011] Preferably, in step S3, the convolutional neural network branch includes: an input layer, a first convolutional layer, a ReLU activation layer, a first pooling layer, a second convolutional layer, a ReLU activation layer, a second pooling layer, a third convolutional layer, a ReLU activation layer, a global average pooling layer, a fully connected layer, and an output layer. The first convolutional layer has 32 kernels, a size of 11, a stride of 2, and the same padding parameter; the second convolutional layer has 64 kernels, a size of 5, a stride of 1, and the same padding parameter; the third convolutional layer has 128 kernels, a size of 3, and a stride of 1; the first and second pooling layers both have a pooling size of 2 and a stride of 2; and the fully connected layer has 128 neurons. The residual neural network includes: an input layer, a convolutional layer, a normalization layer, a ReLU activation layer, a pooling layer, four residual modules, a global average pooling layer, a fully connected layer, and an output layer. The convolutional layer has 64 kernels, a size of 7×7, a stride of 2, and padding of 3. The pooling layer has a pooling size of 3×3 and a stride of 2. The number of channels in the four residual modules are 64, 128, 256, and 512, respectively. The number of neurons in the fully connected layer is 128.

[0012] Preferably, in step S4, the attention module adopts the SE-Block mechanism, including the following steps: taking the spliced fused feature vector as the global descriptor; reducing the feature dimension to 1 / 16 of the original dimension through the first fully connected layer, with the activation function being ReLU; restoring the feature dimension to the original dimension through the second fully connected layer, with the activation function being Sigmoid, to generate the weight vector W; multiplying the weight vector W and the fused feature vector element by element to obtain the weighted feature vector F final , and the calculation formula is: F final =F concat ×W; Among them, F concat represents the fused feature vector.

[0013] Preferably, in step S3, the training process of the three-modal deep learning fusion model includes: constructing a training dataset, which contains the Atractylodes macrocephala samples data collected at different softening time points and the corresponding label values. The label values are determined by combining the expert sensory evaluation and the quality inspection of the cut products. The expert sensory evaluation scores according to the hand feeling, nail indentation and cross-section moisture. The quality inspection of the cut products includes measuring the percentage of the weight of the complete cut pieces in the total weight and the proportion of the weight of the debris with a diameter less than 2 mm. After normalizing the expert sensory evaluation and the quality inspection results of the cut products, they are weighted and summed to obtain a continuous value of 0.0~1.0 as the label value; using the mean square error loss function combined with L2 regularization as the loss function, and using the AdamW optimizer for model training, with the initial learning rate being 1e-4 and the weight decay coefficient being 1e-2.

[0014] Preferably, in step S5, the softening index S is a continuous value of 0~1. The softening state level is defined according to the S value range: when 0.0≤S<0.75, it is determined as insufficient softening; when 0.75≤S≤0.95, it is determined as suitable for cutting; when 0.95<S≤1.0, it is determined as over-softening.

[0015] Another aspect of the present invention provides a real-time monitoring system for the softening end point of Atractylodes macrocephala, including a data acquisition unit, a sample transmission and holding unit, a central processing unit and a control unit: Among them, the data acquisition unit includes a Fourier transform near-infrared spectroscopy acquisition module, a low-field nuclear magnetic resonance analysis module and a full texture analysis measurement module, which are used to detect Atractylodes macrocephala samples and obtain near-infrared spectroscopy data, transverse relaxation time data, nuclear magnetic resonance imaging data and texture characteristic data respectively; The sample transmission and holding unit includes a sample box and an automated robotic arm, which are used to transfer Atractylodes macrocephala samples between the infrared spectroscopy acquisition module, the low-field nuclear magnetic resonance analysis module and the full texture analysis measurement module; The central processing unit includes an industrial control computer, pre-installed with data preprocessing algorithms and a multimodal fusion model, used to receive and process the detection data acquired by the data acquisition unit; The control unit includes a PLC control system, which is used to control the data acquisition process and sample transmission scheduling; The output unit is used to display the softening index S, the discrimination result, and the moisture distribution cloud map in real time.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention establishes a collaborative discrimination scheme that deeply couples chemical fingerprinting, microscopic moisture distribution, and macroscopic mechanical properties, eliminating false softening signals generated by single-dimensional discrimination and ensuring a high degree of consistency between softening endpoint discrimination and actual cutting suitability. Simultaneously, it automatically extracts deep nonlinear features from multidimensional heterogeneous data using deep learning algorithms, significantly improving the model's robustness and generalization ability, and solving the problem of traditional models failing with different batches of medicinal materials. Furthermore, the system can output continuous softening index values ​​and moisture distribution cloud maps in real time, realizing a shift from a black-box operation in traditional Chinese medicine processing to transparent production, providing an objective technical means for the quality control of prepared Chinese medicine slices. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the principle framework of the present invention; Figure 2 This is a schematic diagram of the overall structure and flow of the present invention; Figure 3 This is a schematic diagram of the model architecture of the present invention; Figure 4 This is a schematic diagram of MRI at different times according to the present invention; Figure 5 This is a schematic diagram of the transverse relaxation time T2 at different times according to the present invention; Figure 6 A schematic diagram showing the strength of the wetted texture at different times. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, features defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0020] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0021] Please see the appendix Figure 1-6 A method for determining the softening endpoint of Atractylodes macrocephala based on multimodal data fusion and deep learning includes the following steps: S1: At multiple time points during the softening process of Atractylodes macrocephala, Fourier transform near-infrared spectrometer, low-field nuclear magnetic resonance spectrometer and texture analyzer were used to detect the Atractylodes macrocephala samples, and near-infrared spectral data, transverse relaxation time data, nuclear magnetic resonance imaging data and texture characteristic data were acquired respectively, and time sequence alignment was performed, and the aligned data were cleaned. In step S1, the acquisition of near-infrared spectral data includes: placing the Atractylodes macrocephala sample to be tested under a diffuse reflectance probe, and setting the spectral scanning range to 4000-12000 cm⁻¹. -1 The resolution is 8cm. -1 Each spectrum was scanned a total of 32 times, and each sample was measured in parallel 3 times. The average spectrum was obtained using spectral acquisition software, namely OMNIC software. The acquisition of transverse relaxation time data included: moving the Atractylodes macrocephala sample to be tested into the center of the MRI probe coil, with a magnet temperature of 32±0.1℃, an RF diameter of 60mm, a P1 pulse duration of 10μs, and a P2 pulse duration of 19.04μs, with P1 and P2 pulses supplemented every 1ms. Proton signals were acquired using the Carr-Purcell-Meiboom-Gill spin echo sequence, with an echo number of 8000, a signal receiving bandwidth of 100kHz, a waiting time of 5000ms, 4 scans, and an echo time of 0.30ms. After acquiring the transverse relaxation decay data, the transverse relaxation time spectrum was obtained by analyzing the SIRT mathematical inversion model. The acquisition of the nuclear magnetic resonance imaging data included: obtaining magnetic resonance images of the cross-section of Atractylodes macrocephala based on a multi-slicepinechoes sequence, with a slice width of 2.0 mm, a slice spacing of 1.0 mm, a slice number of 5, a time echo of 20 ms, a readout size of 256, a phase size of 192, a repetition time of 1000 ms, a flip angle of 90°, a refocusing flip angle of 180°, an average value of 4, an RG of 20 dB, a PRG of high, and a FOV of 100 mm × 100 mm.

[0022] The acquisition of textural properties data included: puncturing the Atractylodes macrocephala samples using a Brookfield CT3 full texture analyzer, selecting a TA-9 probe (needle-shaped, 1.0 mm in diameter, 43 mm in length), a trigger point load of 5 g, a weighing sensor of 10000 g, a return speed of 1 mm / s, a testing speed of 1 mm / s, a puncture depth of 6 mm, and a data frequency of 50 points / s. Three consecutive measurements were taken and the average value was calculated. The measured parameters included hardness, compression work cycles, viscosity, viscousity, and elasticity.

[0023] It should be noted that, to address the asynchronous nature of data along the timeline, this invention employs synchronous acquisition and time-series alignment processing of multi-source heterogeneous data. Specifically, the sampling time and period setting strategy is as follows: the system sampling period ΔT is set to 30 minutes, meaning that acquisition commands are triggered at times t=0.5 hours, 1.0 hours, and 1.5 hours. This sampling period can be adjusted according to the total softening time of Atractylodes macrocephala. Since low-field NMR acquisition takes the longest, the start time of low-field NMR acquisition is set as the time anchor point t of this period. anchor ; The timing control process is as follows: when the system clock reaches t... anchor At that time, the robotic arm grasped the Atractylodes macrocephala sample; first, a full texture test was performed, which took about 60 seconds, and the data was recorded. TPA Then, a near-infrared scan was performed, which took approximately 10 seconds. Three consecutive scans were performed, and the average value was recorded as data D. NIRFinally, the data was sent to the MRI chamber for imaging and relaxation analysis, which took approximately 180 seconds, and the data was recorded. NMR During the softening process of Atractylodes macrocephala, which lasts for several hours, the changes in physical properties within this time difference are negligible. The system uniformly labels the above three sets of data as the "joint eigenvector of time t".

[0024] The pairing and fusion of data sequences are accomplished through the following steps: A master data table based on the Pandas DataFrame data processing table framework is established, with the table header containing the batch number and softening time point. The system only extracts the most recent valid data within ±5 minutes of the softening time point for pairing. If data is missing at a certain moment, the system uses linear interpolation to fill in the missing data based on the data from the two time points before and after the missing data point.

[0025] Data cleaning and processing include: removing abnormal acquired data, such as data in near-infrared spectroscopy with baseline drift exceeding the normal range, image distortion data in nuclear magnetic resonance imaging caused by sample position shift, and data with abnormal triggering force in texture testing; and completing missing data using linear interpolation.

[0026] S2: The near-infrared spectral data, transverse relaxation time data, and texture property data after cleaning are preprocessed. Features are extracted from the preprocessed data and spliced ​​to obtain a combined feature vector. The nuclear magnetic resonance imaging data is preprocessed to obtain a nuclear magnetic resonance image. In step S2, the preprocessing includes: truncating the near-infrared spectral data to 4000-10000 cm⁻¹. -1 After banding, standardization was performed; the standard scaler standard deviation method was used, and the output dimension was (1, 1557); after extracting hardness, cohesion, elasticity and resilience parameters from the texture property data, they were normalized to the 0-1 interval using the Min-Max normalization method, and the output dimension was (1, 4); relaxation time distribution features were extracted from the transverse relaxation time data, and the inverted T2 relaxation time distribution data was extracted, with an output dimension of (1, 100); the nuclear magnetic resonance imaging data were adjusted to a grayscale image of size 224×224×1.

[0027] The formula for concatenating the combined feature vectors is: V combined =Concat(V NIR V TPA V T2 ) Among them, V NIR V represents the near-infrared spectral eigenvector. TPA V represents the eigenvector of texture properties. T2 This represents the lateral relaxation time feature vector, and Concat represents the concatenation operation in the channel dimension.

[0028] S3: Construct and train a trimodal deep learning fusion model. The trimodal deep learning fusion model includes a convolutional neural network branch and a residual neural network. The convolutional neural network branch is used to process the combined feature vectors and output the first feature vector, and the residual neural network is used to process the MRI images and output the second feature vector. In step S3, the convolutional neural network branch includes: an input layer, a first convolutional layer, a ReLU activation layer, a first pooling layer, a second convolutional layer, a ReLU activation layer, a second pooling layer, a third convolutional layer, a ReLU activation layer, a global average pooling layer, a fully connected layer, and an output layer. The first convolutional layer has 32 kernels, a size of 11, a stride of 2, and the same padding parameter. The second convolutional layer has 64 kernels, a size of 5, a stride of 1, and the same padding parameter. The third convolutional layer has 128 kernels, a size of 3, and a stride of 1. No pooling layer follows the third convolutional layer to preserve local semantic features. The first and second pooling layers both have a pooling size of 2 and a stride of 2, using MaxPooling 1D one-dimensional max pooling. The fully connected layer has 128 neurons, and the activation function is ReLU. The input layer receives an input of size (N, 1), where N is the length of the combined feature vector. This branch processes one-dimensional vector data, extracting deep features from the combined feature vector through layer-by-layer convolution and pooling operations, and outputting a first feature vector F with dimensions (1, 128). seq .

[0029] The residual neural network includes: an input layer, a convolutional layer, a normalization layer, a ReLU activation layer, a pooling layer, four residual modules, a global average pooling layer, a fully connected layer, and an output layer. The convolutional layers have 64 kernels, a size of 7×7, a stride of 2, and padding of 3. The pooling layers have a pooling size of 3×3 and a stride of 2. The four residual modules have 64, 128, 256, and 512 channels respectively. Each residual module contains two basic residual blocks. The first residual module has a stride of 1, and the second, third, and fourth residual modules have a stride of 2. The fully connected layers have 128 neurons, and the activation function is ReLU. The input layer receives a single-channel grayscale image with an input size of (224, 224, 1). This branch uses an improved ResNet-18 residual network with an 18-layer lightweight architecture to process MRI images. It avoids the gradient vanishing problem of deep networks through the residual connection mechanism, and extracts topological features reflecting the spatial distribution of water within Atractylodes macrocephala in the image. The output is a second feature vector F with dimensions of (1, 128). img .

[0030] The training process of the trimodal deep learning fusion model includes: constructing a training dataset, which contains Atractylodes macrocephala sample data collected at different softening time points and their corresponding label values. The training dataset is named BZ-SoftDB Atractylodes macrocephala softening database, and a total of 1200 valid data samples were collected, covering Atractylodes macrocephala medicinal materials from 5 different producing areas and 3 different grades (large / medium / small). The dataset is randomly divided into a training set of 960 groups, a validation set of 120 groups, and a test set of 120 groups in a ratio of 8:1:1.

[0031] like Figure 4 As shown in the MRI images, red areas correspond to higher moisture content, green areas to medium moisture content, and blue areas to lower moisture content. The results indicate that during the soaking process of Atractylodes macrocephala, the increase in moisture content was not significant within 0–16 h, and was mainly concentrated in the outer layer of the herb; however, during 24–48 h, the moisture content increased significantly and gradually penetrated into the inner layer. This phenomenon may be related to the chrysanthemum-like structure of Atractylodes macrocephala, where water initially has difficulty penetrating, but once the structural barrier is overcome, water rapidly penetrates and tends towards saturation.

[0032] Depend on Figure 5 It can be seen that there are two phases of water in the T2 relaxation spectrum of Atractylodes macrocephala: T21 (0.01–10 ms) belongs to water molecules with strong hydrogen bonds (monolayer water), mainly related to water in the cell wall; T22 (10–1000 ms) corresponds to water molecules with weaker bonds to monolayer water (multilayer water), including water in the cytoplasm and extracellular interstitial space. The peak integral area (A21, A22) reflects the relative content of water in each phase. As wetting progresses, the T2 relaxation time increases significantly and shifts towards higher relaxation, indicating that the degree of freedom of water inside the herb gradually increases, and a large amount of external water enters the interstitial space and is detected by the instrument. Before thorough wetting, T2 is mainly concentrated in the 0.1–10 ms range. At this time, although water has entered the herb, it still exists in the form of "bound water" due to limited water absorption. After wetting, the T2 range shifts to 10–1000 ms, indicating that a large amount of water has penetrated in, and the herb is becoming more saturated with water inside and outside.

[0033] Figure 6 The curve height shows a decreasing trend within the first 32 hours, consistent with the hardness change pattern in Table 2, indicating that the medicinal material gradually softens during the soaking process. After thorough soaking, the curve height tends to stabilize, indicating that the medicinal material has a uniform texture inside and out at this point, making it easy to cut.

[0034] Table 1 As shown in Table 1, the peak areas of both free water and bound water generally increased during the soaking process of Atractylodes macrocephala, with the increase in free water being particularly significant. The percentage of free water increased from the initial 0.45% to the final 74.86%, while the percentage of bound water decreased from 99.55% to 25.14%. At the time of thorough soaking (32 h), the proportions of free water and bound water were 42.95% and 57.05%, respectively.

[0035] Table 2 Table 2 shows that in the initial stage of Atractylodes macrocephala processing (before 16 hours), the high hardness of the herb prevented puncture testing; therefore, 16 hours was used as the first measurable point. During the processing, hardness, compression work cycles, viscosity, and stickiness gradually decreased, while elasticity and elasticity showed no significant changes. Significant changes in mechanical parameters occurred between 16 and 32 hours, with the rate of change slowing down between 32 and 48 hours. This may be because after 32 hours, the herb had reached a state of complete moisture penetration, with a balanced internal and external moisture distribution and a more uniform texture. This result corroborates the aforementioned processing kinetics, objectively demonstrating that small-sized Atractylodes macrocephala can achieve complete moisture penetration under a load below 500 g. The negative values ​​in the figure are due to the pressure exerted by the herb on the puncture needle during its return stroke.

[0036] The label value was determined by combining expert sensory evaluation with the quality inspection of the sliced ​​product. The expert sensory evaluation was based on the feel of the slices, the marks left by fingernails, and the smoothness of the cross-section. Three pharmacists with more than 5 years of experience in processing Chinese medicinal herbs conducted a blind evaluation, with a score range of 0-100 points. This part accounted for 40% of the weight. The quality inspection of the sliced ​​product included measuring the percentage of whole slices in the total weight and the percentage of fragments with a diameter of less than 2 mm. The Atractylodes macrocephala samples were immediately sliced ​​and statistically analyzed after the data was collected. This part accounted for 60% of the weight. The expert sensory evaluation scores and the quality inspection results of the cut finished products were normalized and then weighted and summed to obtain a continuous value from 0.0 to 1.0, which was used as the label value Y. true For example, a sample with a fragmentation rate of less than 3% and a completeness rate of more than 95% has a Y... true The value is marked as 1.0, indicating the ideal endpoint. A mean squared error loss function combined with L2 regularization (i.e., weight decay regularization) is used as the loss function to prevent overfitting. The AdamW adaptive moment estimation weight decay optimizer is used for model training, with an initial learning rate of 1e-4 and a weight decay coefficient of 1e-2. The learning rate adjustment strategy employs cosine annealing with warm restarts.

[0037] Specifically, the learning rate decays periodically according to a cosine function, with each restart cycle consisting of 50 epochs. After the initial cycle, the learning rate decays to its minimum value of 1×10⁻⁶. -6Then immediately restart at 50% of the initial learning rate, i.e., 5 × 10. -5 A new decay cycle begins. This strategy helps the model escape local optima and improves convergence quality. The training process employs an early stopping strategy, stopping training when the validation set loss does not decrease for 20 consecutive epochs. Finally, the model weights with the best performance on the validation set are selected as the final model.

[0038] Model evaluation metrics include: the coefficient of determination R for regression accuracy. 2 The mean absolute error (MAE) is used to calculate the accuracy of endpoint determination. For accuracy, the S-value is binarized according to a threshold, and the accuracy is calculated along with the F1-Score as a comprehensive evaluation index. Through the above training process, the model can automatically learn the complex nonlinear mapping relationship between multimodal data and the softening index during the softening process of Atractylodes macrocephala. Compared with traditional linear regression models, this deep learning model can automatically filter out nonlinear interference noise caused by differences in Atractylodes macrocephala's origin, size, and density, significantly improving the model's adaptability to different batches of medicinal materials.

[0039] S4: The first feature vector and the second feature vector are fused together, and the fused feature vector is input into the attention module for feature reweighting to obtain a weighted feature vector; In step S4, the attention module employs the SE-Block mechanism, including the following steps: using the concatenated fused feature vector as a global descriptor; reducing the feature dimension to 1 / 16 of the original dimension through a first fully connected layer, with ReLU as the activation function; restoring the feature dimension to the original dimension through a second fully connected layer, with Sigmoid as the activation function, generating a weight vector W; and multiplying the weight vector W element-wise with the fused feature vector to obtain a weighted feature vector F. final The calculation formula is: F final =F concat ×W; Among them, F concat This represents the fused feature vector. This attention mechanism enables the model to automatically adjust the importance of each modal data (near-infrared spectroscopy, nuclear magnetic resonance, and textural properties) in the final judgment at different stages of softening. For example, in the early stage of softening when water only wets the surface, the model automatically assigns higher weight to spectral features that reflect surface chemical changes; in the later stage of softening when water has penetrated the interior, the model automatically assigns higher weight to nuclear magnetic resonance or textural features that reflect the softness and hardness of the internal tissue, thereby achieving adaptive focusing on key features of the softening process.

[0040] S5: Input the weighted feature vector into the regression layer and output the softening index S. When S is greater than the set threshold, it is determined that Atractylodes macrocephala has reached the softening endpoint.

[0041] In step S5, the weighted feature vector F final is input into the regression layer. The regression layer is a fully connected layer with 1 neuron, the activation function is Sigmoid, and the output is the softening index S. The softening index S is a continuous value between 0 and 1, reflecting the softening degree of Atractylodes macrocephala Koidz. According to the S value range, the softening state level is defined: when 0.0 ≤ S < 0.75, it is determined as insufficient softening, with characteristics of high central hardness, easy chipping during cutting, and large machine load; when 0.75 ≤ S ≤ 0.95, it is determined as suitable for cutting, with characteristics of moderate hardness, the highest integrity rate, and less loss of volatile oil; when 0.95 < S ≤ 1.0, it is determined as over-softening, with characteristics of tissue collapse, slice adhesion, and serious loss of active ingredients.

[0042] The selection basis for setting the threshold is as follows: The predicted value S of 1200 groups of samples is fitted with their actual cutting integrity rate. Through the correlation analysis of the ROC (Receiver Operating Characteristic) curve and cutting quality statistics, when the S value reaches 0.85, the powder rate of Atractylodes macrocephala Koidz slices shows a cliff-like drop, from 12% to less than 3%, and the integrity rate is stable above 90%. Therefore, S = 0.85 is selected as the best balance point that takes into account both production efficiency and finished product quality. The selection of this threshold takes into account both production efficiency and finished product quality, provides core algorithm support for the construction of a digital workshop for Atractylodes macrocephala Koidz processing, effectively reduces the powder rate of sliced products, and ensures the retention rate of components such as volatile oil.

[0043] A real-time monitoring system for the softening end point of Atractylodes macrocephala Koidz includes a data acquisition unit, a sample transmission and holding unit, a central processing unit, and a control unit: Among them, the data acquisition unit includes a Fourier transform near-infrared spectroscopy acquisition module, a low-field nuclear magnetic resonance analysis module, and a full texture analysis measurement module, which are used to detect Atractylodes macrocephala Koidz samples and obtain near-infrared spectral data, transverse relaxation time data, nuclear magnetic resonance imaging data, and texture characteristic data respectively; The sample transmission and holding unit includes a sample box and an automated robotic arm, which are used to transfer Atractylodes macrocephala Koidz samples among the infrared spectroscopy acquisition module, the low-field nuclear magnetic resonance analysis module, and the full texture analysis measurement module; The central processing unit includes an industrial control computer, pre-installed with data preprocessing algorithms and a multi-modal fusion model, which is used to receive and process the detection data obtained by the data acquisition unit; The control unit includes a PLC control system, which is used to control the data acquisition process and sample transmission scheduling; An output unit, which is used to display the softening index S, the discrimination result, and the moisture distribution cloud map in real time.

[0044] This invention establishes a collaborative discrimination scheme that deeply couples chemical fingerprinting, microscopic moisture distribution, and macroscopic mechanical properties, eliminating false softening signals generated by single-dimensional discrimination and ensuring a high degree of consistency between softening endpoint discrimination and actual cutting suitability. Simultaneously, by automatically extracting deep nonlinear features from multidimensional heterogeneous data using deep learning algorithms, the robustness and generalization ability of the model are significantly improved, solving the problem of traditional models failing with different batches of medicinal materials.

[0045] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for determining the softening endpoint of Atractylodes macrocephala based on multimodal data fusion and deep learning, characterized in that, Includes the following steps: S1: At multiple time points during the softening process of Atractylodes macrocephala, Fourier transform near-infrared spectrometer, low-field nuclear magnetic resonance spectrometer and texture analyzer were used to detect the Atractylodes macrocephala samples, and near-infrared spectral data, transverse relaxation time data, nuclear magnetic resonance imaging data and texture characteristic data were acquired respectively, and time sequence alignment was performed, and the aligned data were cleaned. S2: The near-infrared spectral data, transverse relaxation time data, and texture property data after cleaning are preprocessed. Features are extracted from the preprocessed data and spliced ​​to obtain a combined feature vector. The nuclear magnetic resonance imaging data is preprocessed to obtain a nuclear magnetic resonance image. S3: Construct and train a trimodal deep learning fusion model. The trimodal deep learning fusion model includes a convolutional neural network branch and a residual neural network. The convolutional neural network branch is used to process the combined feature vectors and output the first feature vector, and the residual neural network is used to process the MRI images and output the second feature vector. S4: The first feature vector and the second feature vector are fused together, and the fused feature vector is input into the attention module for feature reweighting to obtain a weighted feature vector; S5: Input the weighted feature vector into the regression layer and output the softening index S. When S is greater than the set threshold, it is determined that Atractylodes macrocephala has reached the softening endpoint.

2. The method for determining the softening endpoint of Atractylodes macrocephala based on multimodal data fusion and deep learning according to claim 1, characterized in that, In step S1, the acquisition of near-infrared spectral data includes: placing the Atractylodes macrocephala sample to be tested under a diffuse reflectance probe, and setting the spectral scanning range to 4000-12000 cm⁻¹. -1 The resolution is 8cm. -1 Each spectrum was scanned a total of 32 times, and each sample was measured in parallel 3 times and the average value was taken. The acquisition of transverse relaxation time data included: moving the Atractylodes macrocephala sample to be tested into the center of the MRI probe coil, acquiring proton signals using the Carr-Purcell-Meiboom-Gill sequence, with an echo number of 8000, a signal receiving bandwidth of 100kHz, a waiting time of 5000ms, 4 scans, and an echo time of 0.30ms; after acquiring the transverse relaxation decay data, the transverse relaxation time spectrum was obtained by analyzing the SIRT mathematical inversion model. The acquisition of the nuclear magnetic resonance imaging data included: obtaining magnetic resonance images of the cross-section of Atractylodes macrocephala based on a multi-slicepinechoes sequence, with a slice width of 2.0 mm, a slice spacing of 1.0 mm, a slice number of 5, a time echo of 20 ms, a readout size of 256, a phase size of 192, a repetition time of 1000 ms, a flip angle of 90°, a refocusing flip angle of 180°, an average value of 4, an RG of 20 dB, a PRG of high, and a FOV of 100 mm × 100 mm.

3. The method for determining the softening endpoint of Atractylodes macrocephala based on multimodal data fusion and deep learning according to claim 1, characterized in that, In step S1, the acquisition of textural property data includes: using an analyzer to puncture the Atractylodes macrocephala sample to be tested, with a trigger point load of 5g, a weighing sensor of 10000g, a return speed of 1mm / s, a test speed of 1mm / s, a puncture depth of 6mm, a data frequency of 50 points / s, and taking the average value after three consecutive measurements.

4. The method for determining the softening endpoint of Atractylodes macrocephala based on multimodal data fusion and deep learning according to claim 1, characterized in that, In step S2, the preprocessing includes: truncating the near-infrared spectral data to 4000-10000 cm⁻¹. -1 After band normalization, the hardness, cohesion, elasticity and resilience parameters of the texture property data are extracted and normalized to the 0-1 range; the relaxation time distribution characteristics of the transverse relaxation time data are extracted; and the nuclear magnetic resonance imaging data are adjusted to a grayscale image of 224×224×1.

5. The method for determining the softening endpoint of Atractylodes macrocephala based on multimodal data fusion and deep learning according to claim 4, characterized in that, The formula for concatenating the combined feature vectors is: V combined =Concat(V NIR V TPA V T2 ) Among them, V NIR V represents the near-infrared spectral eigenvector. TPA V represents the eigenvector of texture properties. T2 This represents the lateral relaxation time feature vector, and Concat represents the concatenation operation in the channel dimension.

6. The method for determining the softening endpoint of Atractylodes macrocephala based on multimodal data fusion and deep learning according to claim 1, characterized in that, In step S3, the convolutional neural network branch includes: an input layer, a first convolutional layer, a ReLU activation layer, a first pooling layer, a second convolutional layer, a ReLU activation layer, a second pooling layer, a third convolutional layer, a ReLU activation layer, a global average pooling layer, a fully connected layer, and an output layer. Among them, the number of convolutional kernels in the first convolutional layer is 32, the size is 11, the stride is 2, and the padding parameter is same. The number of convolutional kernels in the second convolutional layer is 64, the size is 5, the stride is 1, and the padding parameter is same. The number of convolutional kernels in the third convolutional layer is 128, the size is 3, and the stride is 1. The pooling sizes of the first pooling layer and the second pooling layer are both 2, the stride is 2, and the number of neurons in the fully connected layer is 128; The residual neural network includes: an input layer, a convolutional layer, a normalization layer, a ReLu activation layer, a pooling layer, four residual modules, a global average pooling layer, a fully connected layer, and an output layer; Among them, the number of convolutional kernels in the convolutional layer is 64, the size is 7×7, the stride is 2, and the padding is 3. The pooling size of the pooling layer is 3×3, and the stride is 2. The number of channels of the four residual modules are 64, 128, 256, and 512 in sequence, and the number of neurons in the fully connected layer is 128.

7. The method for determining the softening endpoint of Atractylodes macrocephala based on multimodal data fusion and deep learning according to claim 1, characterized in that, In step S4, the attention module employs the SE-Block mechanism, including the following steps: using the concatenated fused feature vector as a global descriptor; reducing the feature dimension to 1 / 16 of the original dimension through a first fully connected layer, with ReLU as the activation function; restoring the feature dimension to the original dimension through a second fully connected layer, with Sigmoid as the activation function, generating a weight vector W; and multiplying the weight vector W element-wise with the fused feature vector to obtain a weighted feature vector F. final The calculation formula is: F final =F concat ×W; Among them, F concat This represents the fused feature vector.

8. The method for determining the softening endpoint of Atractylodes macrocephala based on multimodal data fusion and deep learning according to claim 1, characterized in that, In step S3, the training process of the three-modal deep learning fusion model includes: constructing a training data set, which contains Atractylodes macrocephala samples data collected at different softening time points and corresponding label values. The label values are determined by combining expert sensory evaluation and cut finished product quality inspection. The expert sensory evaluation scores according to the hand-squeezing feeling, nail indentation, and cross-section moisture. The cut finished product quality inspection includes measuring the percentage of the weight of intact cut pieces in the total weight and the proportion of the weight of debris with a diameter less than 2 mm. After normalizing the expert sensory evaluation and the cut finished product quality inspection results, they are weighted and summed to obtain a continuous value of 0.0~1.0 as the label value. The mean square error loss function combined with L2 regularization is used as the loss function, and the AdamW optimizer is used for model training. The initial learning rate is 1e-4, and the weight decay coefficient is 1e-2.

9. The method for determining the softening endpoint of Atractylodes macrocephala based on multimodal data fusion and deep learning according to claim 8, characterized in that, In step S5, the softening index S is a continuous value of 0~1. The softening state level is defined according to the S value range: when 0.0≤S<0.75, it is determined as insufficient softening; when 0.75≤S≤0.95, it is determined as suitable for cutting; when 0.95<S≤1.0, it is determined as over-softening.

10. A real-time monitoring system for the softening endpoint of Atractylodes macrocephala, used to implement the steps of the Atractylodes macrocephala softening endpoint discrimination method based on multimodal data fusion and deep learning as described in any one of claims 1 to 9, characterized in that, It includes a data acquisition unit, a sample transmission and holding unit, a central processing unit, and a control unit: Among them, the data acquisition unit includes a Fourier transform near-infrared spectroscopy acquisition module, a low-field nuclear magnetic resonance analysis module, and a full texture analysis measurement module, which are used to detect Atractylodes macrocephala samples and obtain near-infrared spectroscopy data, transverse relaxation time data, nuclear magnetic resonance imaging data, and texture characteristic data respectively; The sample transmission and holding unit includes a sample box and an automated robotic arm, which are used to transfer Atractylodes macrocephala samples between the infrared spectroscopy acquisition module, the low-field nuclear magnetic resonance analysis module, and the full texture analysis measurement module; The central processing unit includes an industrial control computer, pre-installed with data preprocessing algorithms and a multi-modal fusion model, which is used to receive and process the detection data obtained by the data acquisition unit; The control unit includes a PLC control system, which is used to control the data acquisition process and sample transmission scheduling; An output unit, which is used to display the softening index S, the discrimination result, and the moisture distribution cloud map in real time.

Images