Multi-source data intelligent fusion medicinal material identification system and medicinal material identification method

Abstract

The invention relates to the technical field of medicinal material identification. The invention discloses a multi-source data intelligent fusion medicinal material identification system and method, and the system comprises a data collection module which is used for collecting various information of medicinal materials, a data preprocessing module which is used for processing the data collected by the data collection module, and a feature extraction module. The feature extraction module is used for carrying out feature extraction on information processed by the data preprocessing module, the feature fusion module is used for fusing features extracted by the feature extraction module to form a comprehensive feature vector, the medicinal material recognition model module is used for recognizing medicinal materials through the comprehensive feature vector and determining the types of the medicinal materials, and the recognition result output module is used for outputting recognition results. The system is used for outputting and storing various information of medicinal materials. According to the multi-source data intelligent fusion medicinal material identification system and method, image data can be utilized, smell data, infrared spectrum data and other information sources can be combined, and the accuracy of medicinal material identification is comprehensively improved through multi-modal data analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of medicinal material identification, and in particular to a medicinal material identification system and a medicinal material identification method for intelligent fusion of multi-source data. Background Art

[0002] In the pharmaceutical process, the identification of medicinal materials is a key step to ensure the quality and safety of medicinal materials. Traditionally, many pharmaceutical factories rely on manual methods to identify and classify medicinal materials, which is not only time-consuming and labor-intensive, but also easily affected by human factors, resulting in low recognition accuracy. With the expansion of production scale and technological advancement, a small number of companies have begun to introduce automated medicinal material identification systems to improve production efficiency and recognition accuracy. However, existing medicinal material identification systems mainly rely on a large number of pictures for training, and determine the authenticity of medicinal materials by comparing the medicinal material images collected on site with a large number of medicinal material images in the database. This analysis method based on a single data source, namely images, has many limitations. In order to achieve high-accuracy medicinal material identification, the system needs to store a large number of medicinal material images, which not only increases the storage burden of the system, but also makes data management and maintenance complicated. Especially for some scarce medicinal materials, due to the limited number of samples, it is difficult to obtain enough pictures for training, which significantly increases the difficulty of identifying these medicinal materials. In addition, image recognition systems often perform poorly when dealing with similar or unique medicinal materials, because it is difficult to distinguish certain medicinal material varieties that are very similar in appearance based on visual features alone. For example, some Chinese medicinal materials are very similar in shape, color, etc., and it is difficult to accurately determine their types based on images alone, which affects the reliability and efficiency of recognition. Summary of the invention

[0003] In order to solve the deficiencies of the prior art, the present invention discloses a medicinal material identification system and a medicinal material identification method that can combine multi-source data to identify medicinal materials, with significant identification effect and high identification accuracy.

[0004] The present invention discloses a medicinal material identification system with intelligent fusion of multi-source data, which comprises: Data collection module, used to collect various information of medicinal materials; A data preprocessing module is used to process the data collected by the data collection module; A feature extraction module is used to extract features from the information processed by the data preprocessing module; A feature fusion module is used to fuse the features extracted by the feature extraction module to form a comprehensive feature vector; The medicinal material identification model module is used to identify the medicinal materials through comprehensive feature vectors and determine the types of medicinal materials; The identification result output module is used to output and store various information of medicinal materials.

[0005] Further, the data acquisition module includes an image acquisition submodule, an odor acquisition submodule and an infrared acquisition submodule; The image acquisition submodule acquires the appearance information of the medicinal materials by acquiring images of the medicinal materials. The odor acquisition submodule captures the odor emitted by the medicinal materials through the electronic nose device and generates a digital signal pattern to reflect the volatile components of the medicinal materials. The infrared acquisition submodule obtains the infrared information of the medicinal materials through the infrared spectrum sensor. The data preprocessing module includes an image processing submodule, an odor processing submodule, and an infrared processing submodule; The image processing submodule is used to process the appearance information of medicinal materials to improve the appearance quality and consistency of medicinal materials. The odor processing submodule is used to extract effective feature vectors and enhance the stability of odor data. The infrared processing submodule is used to process infrared spectral features and extract key spectral features.

[0006] Furthermore, the image acquisition submodule uses the convolution neural network CNN to perform feature recognition and collect image data, the image processing submodule uses the principal component analysis method PCA to reduce the dimension of the collected image data, and the feature extraction module extracts the final image comprehensive feature vector F l ; The odor collection submodule collects odor data, the odor processing submodule uses principal component analysis PCA to reduce the dimension of the collected odor data, and the feature extraction module extracts the odor data feature vector F o ; The infrared acquisition submodule collects infrared data, the infrared processing submodule applies the continuous projection algorithm SPA to dimensionalize the collected infrared data, and the feature extraction module extracts the infrared feature vector F r .

[0007] Furthermore, the image acquisition submodule uses a convolution neural network (CNN) to perform feature recognition and extraction on the image data. The features to be recognized and extracted include the cross-sectional features of the medicinal material X section , shape feature X shape , surface texture features X texture , size feature X size And color feature X color ; The image processing submodule uses principal component analysis (PCA) to reduce the dimension of the identified features to a single value. After the cross-sectional feature is reduced, x1 is obtained, after the shape feature is reduced, x2 is obtained, after the surface texture feature is reduced, x3 is obtained, after the size feature is reduced, x4 is obtained, and after the color feature is reduced, x5 is obtained. The reduced features are combined, and the feature extraction module calculates the final image comprehensive feature vector F l : ; Among them, the cross-sectional feature part F eature1= ; sinh(x1): hyperbolic sine function, used to enhance the importance of cross-sectional features; α: Regularization parameter, used to adjust the influence of sinh(x1); tanh(·): Hyperbolic tangent function, used to keep the output within a certain range and further enhance the nonlinear effect; Shape and Surface Texture Features Section F eature2 = ; |x2| and |x3|: square roots of shape features and surface texture features, used for smoothing and reducing the impact of extreme values; β: regularization parameter, used to adjust the influence of |x2|+|x3|; log(1+e·): function used to process shape and surface texture features, ensuring that the output is non-negative and has a certain degree of smoothness; Size and color characteristics part F eature3 =x4+x5.

[0008] Furthermore, the feature fusion module is used to fuse the features to form a comprehensive feature vector. The comprehensive feature vector formula is: ; w l , w r , w o : They are image feature weight, infrared feature weight and odor feature weight respectively; : Use normalization function to process image features; : Use exponential decay function to process infrared features and enhance nonlinear effects; log(1.68+F o ): Use logarithmic function to process odor features, reducing the impact while maintaining expressiveness.

[0009] Furthermore, the machine learning model is used to obtain the machine learning term model weights w of image features, infrared features and odor features. l，ML , w r，ML , w o，ML ; The frequency weights w of image features, infrared features and odor features are obtained by frequency-based allocation l，GD , w r，GD , w o，GD ; Take the weighted average of the machine learning model weight and the frequency weight to get the final weight w l , w r , wo : ; α: parameter used to adjust the relative contribution of the machine learning model and the gradient descent method. The value range of α is [0, 1]. α represents the weight of the machine learning model, and 1-α represents the weight of the gradient descent method. α is obtained through Bayesian optimization.

[0010] Furthermore, the machine learning model is used to obtain the machine learning term model weights w of image features, infrared features and odor features. l，ML , w r，ML , w o，ML : For image feature F l 、Infrared spectrum characteristics F r and odor characteristics F o Evaluate and derive relative importance scores through feature importance analysis of machine learning models l，ml ,s r，ml ,s o，ml , normalize the weights by the score: ; in: s l，ml : Importance score of image features; s r，ml : importance score of infrared features; s o，ml : Importance score of odor characteristics.

[0011] Furthermore, the frequency weights w of image features, infrared features, and odor features are obtained by frequency-based allocation. l，GD , w r，GD , w o，GD : ; q1: frequency of image features, 55%≤q1≤65%; q2: frequency of infrared spectrum characteristics, 20%≤q2≤45%; q3: frequency of odor characteristics, 10%≤q3≤20%.

[0012] The present invention discloses a medicinal material identification method based on intelligent fusion of multi-source data, which comprises: S1: Data collection stage, the appearance information, smell information and infrared information of medicinal materials are obtained through the image collection submodule, smell collection submodule and infrared collection submodule respectively; S2: Data preprocessing stage, the image processing submodule processes the appearance information of the medicinal materials, the odor processing submodule processes the odor information of the medicinal materials, and the infrared processing submodule processes the infrared information of the medicinal materials; S3: Feature extraction stage, the feature extraction module extracts image feature information, odor feature information and infrared feature information; S4: weight calculation stage, the machine learning weights of image features, infrared features and odor features are obtained through the machine learning model, and the frequency weights of image features, infrared features and odor features are obtained through frequency-based allocation. The machine learning model weights and frequency weights are weighted averaged to obtain the final weights; S5: In the medicinal material identification stage, the system uses the comprehensive feature vector F through the medicinal material identification model module. combined Identify the medicinal materials and determine their types; S6: Result output stage: various information of the medicinal materials are output and stored through the recognition result output module, including but not limited to the name of the medicinal materials and recognition confidence information.

[0013] Furthermore, the data collection stage collects image data, odor data and infrared data, the data preprocessing stage processes and reduces the dimension of the image data, odor data and infrared data, and the feature extraction stage extracts image data feature vectors, odor data feature vectors and infrared data feature vectors.

[0014] Beneficial effects of the present invention: The present invention discloses a medicinal material identification system and method with intelligent fusion of multi-source data, which can not only utilize image data, but also combine multiple information sources such as odor data and infrared spectrum data, and comprehensively improve the accuracy of medicinal material identification through multimodal data analysis. The image acquisition submodule can obtain high-definition images of medicinal materials through a high-resolution camera, and capture the cross-sectional features, shape features, surface texture features, size features and color features of the medicinal materials; the odor acquisition submodule uses an electronic nose device to capture the odor emitted by the medicinal materials and generate a digital signal pattern to reflect the volatile components of the medicinal materials; the infrared acquisition submodule obtains the infrared information of the medicinal materials through an infrared spectrum sensor, which is used to analyze the chemical composition of the medicinal materials. After preprocessing, these multi-source data are respectively extracted to obtain effective feature vectors, and a comprehensive feature vector is formed through a feature fusion module. Finally, the medicinal material identification model module uses this comprehensive feature vector to identify the medicinal materials, which can improve the accuracy and reliability of identification.

[0015] The present invention discloses a medicinal material identification system and method with intelligent fusion of multi-source data. The method of multi-source data fusion is adopted to effectively make up for the shortcomings of a single data source, especially when dealing with scarce medicinal materials or similar medicinal materials. By comprehensively analyzing three different types of data, namely, images, odors and infrared spectra, the system can find the unique identification of medicinal materials in a wider feature space, thereby improving the accuracy and reliability of identification. Multi-source data fusion can also reduce the dependence on a large number of pictures, reduce the storage pressure and computational complexity of the system, and make the system more efficient and practical. A medicinal material identification system and method with intelligent fusion of multi-source data can not only significantly improve the accuracy and efficiency of medicinal material identification, but also provide pharmaceutical factories with more reliable means of raw material quality control, and promote the healthy development of the pharmaceutical industry.

[0016] The comprehensive feature vector F of the present invention combined By weighted fusion of image features, infrared features and odor features, the comprehensive feature vector F l It can fully capture the multi-faceted characteristics of medicinal materials. The multimodal information fusion method enhances the system's expressive ability, enabling the model to understand the characteristics of medicinal materials more comprehensively. l , w r , w o The importance of each feature can be flexibly adjusted according to the specific application scenario. If the image features of a certain type of medicinal material are particularly important, l This flexibility ensures that the system can optimize feature selection and weight assignment according to different types of medicinal materials, further improving recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a medicinal material identification system with intelligent fusion of multi-source data in an embodiment of the present application.

[0018] Figure 2 Another schematic diagram of the medicinal material identification system with intelligent fusion of multi-source data in the implementation manner of the present application.

[0019] Figure 3 Another schematic diagram of the medicinal material identification system with intelligent fusion of multi-source data in the implementation manner of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the specific implementation modes of the present invention will be clearly and completely described below.

[0021] The present invention discloses a multi-source data intelligent fusion medicinal material identification system, which includes a data acquisition module for collecting various information of medicinal materials, a data preprocessing module for processing the data collected by the data acquisition module, and a feature extraction module for extracting features from the information processed by the data preprocessing module.

[0022] The feature fusion module is used to fuse the features extracted by the feature extraction module to form a comprehensive feature vector, and to fuse the visual, odor and spectral features. A comprehensive medicinal material feature vector is formed through a weighted fusion strategy to give full play to the advantages of multi-source data. For example, for medicinal materials with similar appearance and odor but different chemical composition, spectral features can help to accurately distinguish them; and for medicinal materials with similar spectral features but different appearance or odor, visual and odor features can play a role in distinguishing, thereby comprehensively improving the accuracy of medicinal material identification.

[0023] The medicinal material identification model module is used to identify medicinal materials through comprehensive feature vectors, determine the types of medicinal materials, and build a classification model based on deep learning, a hybrid model combining the feature extraction module and deep learning. A large amount of labeled medicinal material sample data covering image, odor and spectral data is used to train the model and optimize the model parameters so that it can accurately classify and identify different medicinal materials. With the help of transfer learning technology, based on the pre-trained model parameters on large-scale image, odor and spectral related data sets, fine-tuning is performed for medicinal material identification tasks to accelerate model training and improve generalization capabilities, so as to better adapt to actual medicinal material identification scenarios.

[0024] The recognition result output module is used to output and store various information of medicinal materials, and to display the recognition results to users in an intuitive interface, including information such as medicinal material name, category, similarity score, etc. The recognition results are stored in the database to facilitate subsequent query and statistical analysis, and provide data support for the quality traceability and management of medicinal materials.

[0025] As an implementation method, the data acquisition module includes an image acquisition submodule, an odor acquisition submodule, and an infrared acquisition submodule. The image acquisition submodule acquires the appearance information of the medicinal material by acquiring images of the medicinal material. The odor acquisition submodule captures the odor emitted by the medicinal material through an electronic nose device to generate a digital signal pattern to reflect the volatile components of the medicinal material. The infrared acquisition submodule acquires the infrared information of the medicinal material through an infrared spectrum sensor. The data preprocessing module includes an image processing submodule, an odor processing submodule, and an infrared processing submodule.

[0026] The image acquisition submodule uses a high-resolution, multi-spectral camera to capture images of medicinal materials. It can collect multi-dimensional visual information such as the appearance color, texture, shape, etc. of the medicinal materials, and can adapt to shooting under different lighting conditions to ensure that the collected image quality is clear, stable and contains rich features. The odor acquisition submodule is equipped with an electronic nose device to capture the odor information emitted by the medicinal materials. The electronic nose detects the chemical components in the air through a set of broad-spectrum response gas sensors and generates a digital signal pattern representing a specific odor. The volatile odor components of medicinal materials are closely related to the variety, quality and degree of processing, providing a new dimension for identification and enhancing accuracy. The infrared acquisition submodule is equipped with a near-infrared spectrum sensor to obtain the near-infrared spectrum data of medicinal materials. The near-infrared spectrum can reflect the chemical composition information inside the medicinal materials. Different medicinal materials present different characteristics on the near-infrared spectrum due to differences in chemical composition. This provides a more comprehensive and in-depth data basis for medicinal material identification, especially helping to distinguish medicinal materials with similar appearance but different chemical compositions.

[0027] The image processing submodule performs cropping, normalization, denoising and other processing on the image data to improve the image quality and consistency for subsequent analysis. The odor processing submodule performs baseline correction, filtering and other preprocessing operations on the odor data collected by the electronic nose to extract effective odor feature vectors, eliminate noise and interference, and enhance data stability and analyzability. The infrared processing submodule performs baseline correction, spectral smoothing and other preprocessing on the near-infrared spectral data to extract key spectral features for subsequent analysis.

[0028] As an implementation method, the image acquisition submodule uses a convolution neural network CNN to perform feature recognition on image data and collect image data, the image processing submodule uses the principal component analysis method PCA to reduce the dimension of the collected image data, and the feature extraction module extracts the final image comprehensive feature vector F l ; The odor collection submodule collects odor data, the odor processing submodule uses principal component analysis PCA to reduce the dimension of the collected odor data, and the feature extraction module extracts the odor data feature vector F o ; The infrared acquisition submodule collects infrared data, the infrared processing submodule applies the continuous projection algorithm SPA to dimensionalize the collected infrared data, and the feature extraction module extracts the infrared feature vector F r .

[0029] 1. Image acquisition submodule: Convolutional neural network (CNN) is used to perform feature recognition and acquire image data.

[0030] Image processing submodule: Use principal component analysis (PCA) to reduce the dimension of image data.

[0031] Feature extraction module: extract the final image comprehensive feature vector Fl .

[0032] CNN is a powerful deep learning model that is particularly suitable for processing image data and can automatically extract complex features such as shape, texture, etc., thereby improving the accuracy of feature recognition.

[0033] PCA can effectively reduce the dimension of image data, remove redundant information, and retain the most important features, thereby speeding up subsequent processing and reducing computing resource consumption.

[0034] All important features are integrated through the feature extraction module to form a comprehensive feature vector F l , which is convenient for subsequent comprehensive analysis and classification.

[0035] 2. Odor collection submodule: collects odor data.

[0036] Odor processing submodule: Use principal component analysis (PCA) to reduce the dimension of odor data.

[0037] Feature extraction module: extract the odor data feature vector F o .

[0038] PCA reduces the dimension of odor data, removes noise and redundant information, and retains the data that best represents the odor characteristics, making subsequent analysis more accurate.

[0039] Through the feature extraction module, the odor data after dimension reduction is converted into a feature vector F o , which is convenient for fusion analysis with data from other modalities.

[0040] 3. Infrared acquisition submodule: collects infrared data.

[0041] Infrared processing submodule: Apply the continuous projection algorithm SPA to reduce the dimension of infrared data.

[0042] Feature extraction module: extract infrared feature vector F of the difference spectrum of chemical components of medicinal materials r .

[0043] SPA is an effective dimensionality reduction method, especially suitable for spectral data, which can better preserve the key information in the spectrum while reducing the data dimension.

[0044] Infrared spectral data can directly reflect the differences in the chemical composition of medicinal materials. Through SPA and feature extraction modules, more accurate chemical composition features can be obtained. r , which helps to distinguish different types or qualities of medicinal materials.

[0045] By combining data from three different modalities, image, odor, and infrared, the characteristics of medicinal materials can be described and analyzed more comprehensively. Each modality provides different perspectives and information, which can greatly improve the accuracy and reliability of detection. The comprehensive feature vector F generated by the feature extraction module l 、F o and F r , can more accurately capture the key features of medicinal materials, thereby improving the accuracy of detection. Data from different modalities provide complementary information, such as images can reflect changes in appearance, smell can reflect volatile components, and infrared spectra can reflect chemical components. The combination of information can more comprehensively evaluate the quality of medicinal materials.

[0046] As an implementation method, the image acquisition submodule uses a convolution neural network (CNN) to perform feature recognition and extraction on the image data. The features to be recognized and extracted include the cross-sectional features of the medicinal material X section , shape feature X shape , surface texture features X texture , size feature X size And color feature X color ; The image processing submodule uses principal component analysis (PCA) to reduce the dimension of the identified features to a single value. After the cross-sectional feature is reduced, x1 is obtained, after the shape feature is reduced, x2 is obtained, after the surface texture feature is reduced, x3 is obtained, after the size feature is reduced, x4 is obtained, and after the color feature is reduced, x5 is obtained. The reduced features are combined, and the feature extraction module calculates the final image comprehensive feature vector F l : ; Among them, the cross-sectional feature part F eature1 = ; sinh(x1): hyperbolic sine function, used to enhance the importance of cross-sectional features; α: Regularization parameter, used to adjust the influence of sinh(x1); tanh(·): Hyperbolic tangent function, used to keep the output within a certain range and further enhance the nonlinear effect; Shape and Surface Texture Features Section F eature2 = ; |x2| and |x3|: square roots of shape features and surface texture features, used for smoothing and reducing the impact of extreme values; β: regularization parameter, used to adjust the influence of |x2|+|x3|; log(1+e·): function used to process shape and surface texture features, ensuring that the output is non-negative and has a certain degree of smoothness; Size and color characteristics part F eature3 =x4+x5.

[0047] Using CNN to extract image features enables the system to efficiently extract the cross-section, shape, surface texture, size and color features of medicinal materials without manual annotation. PCA reduces the dimension of each feature set to a single value, which not only reduces the dimension of the data, but also simplifies the subsequent data processing and model training process. The reduced features x1, x2, x3, x4, x5 are combined into the final comprehensive feature vector F l , multiple aspects of medicinal materials can be considered in a unified framework. The multimodal feature fusion method enhances the system's expressive power, enabling the model to understand the characteristics of medicinal materials more comprehensively, thereby improving the accuracy and robustness of recognition. By combining CNN and PCA to extract and reduce the dimension of medicinal material images, the reduced-dimensional features are combined into a comprehensive feature vector F l The image acquisition submodule can not only efficiently extract and process various features in medicinal material images, but also significantly improve the performance and stability of the model, providing a reliable and efficient means for medicinal material identification, especially when processing complex and diverse medicinal material images.

[0048] Sectional feature part F eature1 = : The hyperbolic sine function sinh(x1) is used to enhance the importance of cross-sectional features. The regularization parameter α is used to adjust the influence of sinh(x1). The hyperbolic tangent function tanh(·) is used to keep the output within a certain range and further enhance the nonlinear effect.

[0049] The hyperbolic sine function sinh(x1) is an odd function with the property of exponential growth. It can amplify the changes in the input signal x1, especially when x1 is large, the value of sinh(x1) will increase rapidly. It helps to highlight the subtle differences in the cross-sectional features of medicinal materials, making the system more sensitive to the cross-sectional features of different medicinal materials, thereby improving the accuracy of recognition. The introduction of the regularization parameter α provides the ability to flexibly adjust the degree of influence of sinh(x1). By adjusting α, the optimal distribution can be found during the training process to ensure that the cross-sectional features are neither overly dominant nor neglected. This is very important for balancing the feature expression between different medicinal materials, especially when the cross-sectional features of some medicinal materials are particularly significant or weak, you can adjust α to accurately control F l, the specific value of α varies according to different medicinal materials. The hyperbolic tangent function tanh(·) ensures that the eigenvalue is not too large or too small, thereby avoiding numerical instability and gradient vanishing problems. At the same time, tanh(·) introduces additional nonlinearity, allowing the model to better capture complex patterns and relationships. This nonlinear transformation is very beneficial for processing complex structures in the cross-sectional features of medicinal materials, such as irregular shapes, textures, etc., and enhances the expressive power of the system. Through the above combination, the system is able to search for unique identifiers of medicinal materials in a wider feature space, especially when dealing with similar medicinal materials. For example, some Chinese medicinal materials are very similar in morphology, but their cross-sectional features are slightly different. Use The combination of can effectively capture these subtle differences and significantly improve the accuracy and robustness of recognition. This setting is not only suitable for common medicinal materials, but also can handle scarce or unique medicinal materials well. The system can flexibly deal with various types of medicinal material cross-sectional features, whether they are common or rare. This is particularly important for pharmaceutical companies, as it ensures that the system can maintain efficient and accurate recognition performance when facing a variety of medicinal materials.

[0050] Shape and Surface Texture Features Section F eature2 = : |x2| and |x3| are the square roots of shape features and surface texture features, used for smoothing and reducing the impact of extreme values. β: Regularization parameter, used to adjust the impact of |x2| + |x3|. The log(1+e·) function is used to process shape and surface texture features to ensure that the output is non-negative and has a certain degree of smoothness.

[0051] The square root operations |x2| and |x3| can effectively reduce the impact of extreme values, making the eigenvalues ​​more stable and smooth. This is especially important for shape and surface texture features, because these features will have large fluctuations or outliers. Exponential function It can amplify smaller feature differences, especially showing stronger nonlinear effects when the feature value is small, which helps to capture subtle differences in the shape and surface texture of medicinal materials, thereby improving the accuracy of recognition. It ensures that the output is always non-negative and has good smoothness, avoiding numerical instability and gradient vanishing problems, so that the model can effectively capture feature changes at different scales and enhance the stability of the system. The regularization parameter β can be adjusted according to the specific application scenario to optimize the recognition effect. For certain specific medicinal materials, certain features can be selected for key extraction to improve recognition accuracy.

[0052] The final image comprehensive feature vector F l : The cross-sectional features, shape and surface texture features, size and color features of the medicinal material are organically integrated to form a comprehensive feature vector. The multi-feature fusion method can more comprehensively describe the characteristics of the medicinal material and improve the accuracy and robustness of recognition. Using the hyperbolic sine function It can effectively enhance the importance of cross-sectional features and highlight their key role in medicinal material identification. The nonlinear characteristics of the function make the feature representation richer and more flexible. For shape and surface texture features, the square root and logarithmic functions are used for smoothing to reduce the influence of extreme values ​​and ensure that the feature values ​​are within a reasonable range. The introduction of regularization parameters α and β allows the model to flexibly adjust the degree of influence of each feature according to actual conditions to avoid overfitting or underfitting problems. The log(1+e·) function is used to ensure that the output of shape and surface texture features is non-negative and has a certain degree of smoothness, which improves the stability and reliability of the system and helps to obtain better performance in subsequent model training and prediction. The structure of this formula is concise and clear, easy to implement and optimize, and each part has a clear mathematical meaning and physical explanation, which is easy to understand and debug. In addition, this formula can reduce the computational complexity while ensuring the feature expression ability, and is suitable for large-scale data processing. This formula has strong adaptability and flexibility, and the regularization parameters α and β can be adjusted according to the specific application scenarios and the types of medicinal materials to optimize the recognition effect. For example, for certain specific medicinal materials, we can choose to focus on extracting certain features and adjust the values ​​of regularization parameters α and β to improve recognition accuracy.

[0053] As an implementation method, the feature fusion module is used to fuse features to form a comprehensive feature vector. The comprehensive feature vector formula is: ; w l , w r , w o : They are image feature weight, infrared feature weight and odor feature weight respectively. : Use normalization function to process image features. :Use exponential decay function to process infrared characteristics and enhance nonlinear effects. log(1.68+F o ): Use logarithmic function to process odor features, reducing the impact while maintaining expressiveness.

[0054] Comprehensive feature vector F combined By weighted fusion of image features, infrared features and odor features, the comprehensive feature vector F l It can fully capture the multi-faceted characteristics of medicinal materials. The multimodal information fusion method enhances the system's expressive ability, enabling the model to understand the characteristics of medicinal materials more comprehensively. l , w r , wo The importance of each feature can be flexibly adjusted according to the specific application scenario. If the image features of a certain type of medicinal material are particularly important, l This flexibility ensures that the system can optimize feature selection and weight distribution according to different types of medicinal materials, further improving recognition accuracy. By fusing multiple features, the comprehensive feature vector F l It can offset the noise or error of a single feature to a certain extent. For example, some features are affected by environmental conditions such as light and humidity, while other features are relatively stable. By combining the information of multiple features, the comprehensive feature vector can effectively improve the system's noise resistance and stability, ensuring more reliable recognition results. Feature vector F combined Through comprehensive fusion of multimodal information, we first construct F l , then F l With infrared characteristics F r and odor characteristics F o Fusion to form F combined The phased approach ensures that the features of each modality can be fully extracted and utilized, avoiding information loss or confusion caused by processing multiple complex features at one time. Moreover, image features are a very important part of medicinal material identification because they can directly reflect the appearance characteristics of medicinal materials. By first calculating the final image comprehensive feature vector F l , which can ensure that image features are fully valued in the early stage and provide a solid foundation for subsequent fusion, helping to enhance the system's feature expression capabilities and improve recognition accuracy. combined When F is adjusted, it can be adjusted flexibly according to the specific application scenario. l 、F r and F o For example, if the image features of a certain type of medicinal material are particularly important, we can assign F l The larger the weight, the more influence it has; and vice versa. Phased calculation makes the entire system more modular, facilitating independent development and debugging. Each stage can be optimized separately, thus simplifying the model training and inference process.

[0055] As an implementation method, the machine learning model weights w of the image features, infrared features and odor features are obtained through the machine learning model. l，ML , w r，ML , w o，ML ; The frequency weights w of image features, infrared features and odor features are obtained by frequency-based allocation l，GD , w r，GD , w o，GD ; Take the weighted average of the machine learning model weight and the frequency weight to get the final weight w l , w r , w o : ; α: parameter used to adjust the relative contribution of the machine learning model and the gradient descent method. The value range of α is [0, 1]. α represents the weight of the machine learning model, and 1-α represents the weight of the gradient descent method. α is obtained through Bayesian optimization.

[0056] The weight of the machine learning model is based on a data-driven approach, which can automatically learn the optimal weight from a large number of samples, with strong adaptability and good generalization ability. This method is particularly suitable for processing complex and diverse data. Frequency weights are based on frequency allocation, which usually contains the knowledge and experience of domain experts and can capture some information that is difficult to reflect in the data but is very important in practical applications. For example, some features have higher priority or stability in specific scenarios. By combining the two, both the powerful data processing capabilities of machine learning are utilized, and the experience and insights of domain experts are incorporated, making the final weight more comprehensive and reliable. The weight of the machine learning model is affected by the distribution of the training data, while the frequency weight is based on a wider range of statistical analysis and expert judgment. Through weighted averaging, the bias caused by a single method can be reduced and the robustness and stability of the model can be improved. Frequency weights can offset the noise or outliers in the weight of the machine learning model to a certain extent, especially when the data quality is not high, to ensure that the model does not rely too much on certain specific samples. Calculating weights by two different methods and performing weighted averaging can not only efficiently integrate the advantages of data-driven and domain knowledge, but also significantly improve the stability of the model.

[0057] As an implementation method, the machine learning model weights w of the image features, infrared features and odor features are obtained through the machine learning model. l，ML , w r，ML , w o，ML : For image feature F l 、Infrared spectrum characteristics F r and odor characteristics F o Evaluate and derive relative importance scores through feature importance analysis of machine learning models l，ml ,s r，ml ,s o，ml , normalize the weights by the score: ; in: s l，ml : Importance score of image features; s r，ml: importance score of infrared features; s o，ml : Importance score of odor characteristics.

[0058] Through the feature importance analysis of the machine learning model, the importance score of each feature can be automatically calculated, and the weight can be adjusted accordingly, which can adapt to different types of medicinal material data and ensure that the weight distribution is more reasonable and scientific. By automatically calculating the feature importance score through the machine learning model, the weight of each feature can be dynamically adjusted without manual intervention, which not only reduces human bias, but also improves the objectivity and scientificity of weight distribution. Different types of medicinal materials show different importance in different features. The machine learning model can adaptively adjust the weight according to the training data to ensure that the system can flexibly respond to various types of medicinal materials and improve the accuracy of recognition. By giving higher weights to features with high importance, the model can focus more on those features that have a greater impact on recognition accuracy, which helps to improve the model's expressiveness and generalization ability, especially when dealing with complex and diverse medicinal material data. The feature importance score reflects the actual contribution of each feature to the model's prediction results. Adjusting the weight in this way can make the model pay more attention to key features, thereby significantly improving recognition accuracy.

[0059] As an implementation method, the frequency weights w of the image features, infrared features, and odor features are obtained by frequency-based allocation. l，GD , w r，GD , w o，GD : ; q1: frequency of image features, 55%≤q1≤65%; q2: frequency of infrared spectrum characteristics, 20%≤q2≤45%; q3: frequency of odor characteristics, 10%≤q3≤20%.

[0060] The frequency weights of image features, infrared features and odor features are obtained through frequency-based allocation. This method can be determined based on existing historical experience, industry experts and the characteristics of the medicinal materials themselves. The weights of different features can be determined by determining the frequency ratio of different features.

[0061] Attachments of the present invention Figure 1 It is a data collection and input terminal of a multi-source data intelligent fusion medicinal material identification system, used to input known medicinal material information. Figure 2 This is another medicinal material information-related figure of a medicinal material identification system with multi-source data intelligent fusion in the present application, which is used to display the actual detailed information of the medicinal material, such as location, delivery person, delivery time, etc. Figure 3This application is a recognition output end of a multi-source data intelligent fusion medicinal material recognition system, which is used to output the recognized medicinal material information and Figure 1 The collected medicinal material information is compared with the event labels of the actual detected drug information, showing the actual function of the drug.

[0062] The present invention discloses a medicinal material identification method based on intelligent fusion of multi-source data, which comprises: S1: During the data collection stage, the appearance information, smell information and infrared information of the medicinal materials are obtained through the image collection submodule, the odor collection submodule and the infrared collection submodule respectively; during the data collection stage, a variety of information of the medicinal materials is collected to ensure comprehensive coverage of the medicinal materials characteristics and improve the accuracy of recognition.

[0063] S2: In the data preprocessing stage, the image processing submodule processes the appearance information of the medicinal materials, the odor processing submodule processes the odor information of the medicinal materials, and the infrared processing submodule processes the infrared information of the medicinal materials. In the data preprocessing stage, different types of information of the medicinal materials are preprocessed separately, which enhances the quality and consistency of the data and lays a solid foundation for subsequent feature extraction.

[0064] S3: Feature extraction stage, the feature extraction module extracts image feature information, odor feature information and infrared feature information; S4: In the weight calculation stage, the machine learning weights of image features, infrared features, and odor features are calculated through the machine learning model, and the frequency weights of image features, infrared features, and odor features are obtained through frequency-based allocation. The machine learning model weights and frequency weights are weighted averaged to obtain the final weights. The double weighted average method ensures that the weight distribution is more reasonable and scientific, avoiding the deviation caused by a single method.

[0065] S5: In the medicinal material identification stage, the system uses the comprehensive feature vector F through the medicinal material identification model module. combined Identify medicinal materials and determine their types; the material identification stage not only takes into account the comprehensive impact of multiple features, but also further improves the accuracy and stability of identification through technologies such as ensemble learning.

[0066] S6: Result output stage: various information of the medicinal materials are output and stored through the recognition result output module, including but not limited to the name of the medicinal materials and recognition confidence information.

[0067] The medicinal material identification method of the present invention with intelligent fusion of multi-source data can adaptively adjust the feature extraction and weight distribution strategies according to different medicinal material types, ensuring that the system can maintain efficient recognition performance when facing various medicinal materials, and has strong flexibility and adaptability. The medicinal material identification method with intelligent fusion of multi-source data can update model parameters in real time, conduct online learning based on newly collected data, and continuously optimize and improve the recognition model to ensure that it is always in the best state and adapt to changes in medicinal material types and new feature patterns. The medicinal material identification method with intelligent fusion of multi-source data uses a variety of preprocessing technologies and feature extraction methods, and the system can effectively reduce the influence of noise and redundant information, ensuring that reliable recognition results can be provided even in complex environments. The entire process of the medicinal material identification method with intelligent fusion of multi-source data is highly automated, reducing the need for manual intervention and improving work efficiency. Especially in large-scale medicinal material identification tasks, this method can significantly speed up processing and ensure consistency.

[0068] As an implementation method, the data collection stage collects image data, odor data and infrared data, the data preprocessing stage processes and reduces the dimension of the image data, odor data and infrared data, and the feature extraction stage extracts the feature vector of the image data, the feature vector of the odor data and the feature vector of the infrared data. Through multimodal data collection and precise feature extraction, the system can describe the characteristics of medicinal materials more comprehensively and accurately, and significantly improve the accuracy of recognition. Dimensionality reduction processing simplifies the data structure, reduces the computational complexity, speeds up the processing speed, reduces the demand for computing resources, and is suitable for efficient processing of large-scale data sets. The system can adaptively adjust the feature extraction and weight distribution strategies according to different types of medicinal materials to ensure that efficient recognition performance can be maintained when facing various medicinal materials, and has strong flexibility and adaptability. The system can perform online learning based on the newly collected data, continuously optimize and improve the recognition model, ensure that it is always in the best state, and adapt to changes in medicinal material types and new feature patterns.

[0069] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A medicinal material identification system based on intelligent fusion of multi-source data, characterized in that: include: Data collection module, used to collect various information of medicinal materials; A data preprocessing module is used to process the data collected by the data collection module; A feature extraction module is used to extract features from the information processed by the data preprocessing module; A feature fusion module is used to fuse the features extracted by the feature extraction module to form a comprehensive feature vector; The medicinal material identification model module is used to identify the medicinal materials through comprehensive feature vectors and determine the types of medicinal materials; The identification result output module is used to output and store various information of medicinal materials.

2. According to the multi-source data intelligent fusion medicinal material identification system described in claim 1, it is characterized by: The data acquisition module includes an image acquisition submodule, an odor acquisition submodule and an infrared acquisition submodule; The image acquisition submodule acquires the appearance information of the medicinal materials by acquiring images of the medicinal materials. The odor acquisition submodule captures the odor emitted by the medicinal materials through the electronic nose device and generates a digital signal pattern to reflect the volatile components of the medicinal materials. The infrared acquisition submodule obtains the infrared information of the medicinal materials through the infrared spectrum sensor. The data preprocessing module includes an image processing submodule, an odor processing submodule, and an infrared processing submodule; The image processing submodule is used to process the appearance information of medicinal materials to improve the appearance quality and consistency of medicinal materials. The odor processing submodule is used to extract effective feature vectors and enhance the stability of odor data. The infrared processing submodule is used to process infrared spectral features and extract key spectral features.

3. According to the multi-source data intelligent fusion medicinal material identification system of claim 2, it is characterized by: The image acquisition submodule uses the convolution neural network CNN to perform feature recognition and collect image data. The image processing submodule uses the principal component analysis method PCA to reduce the dimension of the collected image data. The feature extraction module extracts the final image comprehensive feature vector F l ; The odor collection submodule collects odor data, the odor processing submodule uses principal component analysis PCA to reduce the dimension of the collected odor data, and the feature extraction module extracts the odor data feature vector F o ; The infrared acquisition submodule collects infrared data, the infrared processing submodule applies the continuous projection algorithm SPA to dimensionalize the collected infrared data, and the feature extraction module extracts the infrared feature vector F r .

4. The medicinal material identification system based on multi-source data intelligent fusion according to claim 3 is characterized by: The image acquisition submodule uses the convolution neural network (CNN) to perform feature recognition and extraction on image data. The features to be recognized and extracted include the cross-sectional features of medicinal materials, X section , shape feature X shape , surface texture features X texture , size feature X size And color feature X color ; The image processing submodule uses principal component analysis (PCA) to reduce the dimension of the identified features to a single value. After the cross-sectional feature is reduced, x1 is obtained, after the shape feature is reduced, x2 is obtained, after the surface texture feature is reduced, x3 is obtained, after the size feature is reduced, x4 is obtained, and after the color feature is reduced, x5 is obtained. The reduced features are combined, and the feature extraction module calculates the final image comprehensive feature vector F l : ； Among them, the cross-sectional feature part F eature1 = ; sinh(x1): hyperbolic sine function, used to enhance the importance of cross-sectional features; α: Regularization parameter, used to adjust the influence of sinh(x1); tanh(·): Hyperbolic tangent function, used to keep the output within a certain range and further enhance the nonlinear effect; Shape and Surface Texture Features Section F eature2 = ; |x2| and |x3|: square roots of shape features and surface texture features, used for smoothing and reducing the impact of extreme values; β: regularization parameter, used to adjust the influence of |x2|+|x3|; log(1+e·): function used to process shape and surface texture features, ensuring that the output is non-negative and has a certain degree of smoothness; Size and color characteristics part F eature3 =x4+x5.

5. The medicinal material identification system based on multi-source data intelligent fusion according to claim 4 is characterized by: The feature fusion module is used to fuse features to form a comprehensive feature vector. The formula for the comprehensive feature vector is: ； w l , w r , w o : They are image feature weight, infrared feature weight and odor feature weight respectively; : Use normalization function to process image features; : Use exponential decay function to process infrared features and enhance nonlinear effects; log(1.68+F o ): Use logarithmic function to process odor features, reducing the impact while maintaining expressiveness.

6. The medicinal material identification system based on multi-source data intelligent fusion according to claim 5, characterized in that: The machine learning model is used to calculate the machine learning model weights w of image features, infrared features and odor features. l，ML , w r，ML , w o，ML ; The frequency weights w of image features, infrared features and odor features are obtained by frequency-based allocation l，GD , w r，GD , w o，GD ; Take the weighted average of the machine learning model weight and the frequency weight to get the final weight w l , w r , w o : ； α: parameter used to adjust the relative contribution of the machine learning model and the gradient descent method. The value range of α is [0, 1]. α represents the weight of the machine learning model, and 1-α represents the weight of the gradient descent method. α is obtained through Bayesian optimization.

7. The medicinal material identification system based on multi-source data intelligent fusion according to claim 6, characterized in that: The machine learning model is used to calculate the machine learning model weights w of image features, infrared features and odor features. l，ML , w r，ML , w o，ML : For image feature F l 、Infrared spectrum characteristics F r and odor characteristics F o Evaluate and derive relative importance scores through feature importance analysis of machine learning models l，ml ,s r，ml ,s o，ml , normalize the weights by the score: ； in: s l，ml : Importance score of image features; s r，ml : importance score of infrared features; s o，ml : Importance score of odor characteristics.

8. The medicinal material identification system based on multi-source data intelligent fusion according to claim 6, characterized in that: The frequency weights w of image features, infrared features and odor features are obtained by frequency-based allocation l，GD , w r，GD , w o，GD : ； q1: frequency of image features, 55%≤q1≤65%; q2: frequency of infrared spectrum characteristics, 20%≤q2≤45%; q3: frequency of odor characteristics, 10%≤q3≤20%.

9. A method for identifying medicinal materials by intelligent fusion of multi-source data, characterized in that: include: S1: Data collection stage, the appearance information, smell information and infrared information of medicinal materials are obtained through the image collection submodule, smell collection submodule and infrared collection submodule respectively; S2: Data preprocessing stage, the image processing submodule processes the appearance information of the medicinal materials, the odor processing submodule processes the odor information of the medicinal materials, and the infrared processing submodule processes the infrared information of the medicinal materials; S3: Feature extraction stage, the feature extraction module extracts image feature information, odor feature information and infrared feature information; S4: weight calculation stage, the machine learning weights of image features, infrared features and odor features are obtained through the machine learning model, and the frequency weights of image features, infrared features and odor features are obtained through frequency-based allocation. The machine learning model weights and frequency weights are weighted averaged to obtain the final weights; S5: In the medicinal material identification stage, the system uses the comprehensive feature vector F through the medicinal material identification model module. combined Identify the medicinal materials and determine their types; S6: Result output stage: various information of the medicinal materials are output and stored through the recognition result output module, including but not limited to the name of the medicinal materials and recognition confidence information.

10. The method for identifying medicinal materials by intelligent fusion of multi-source data according to claim 9, characterized in that: The data collection stage collects image data, odor data and infrared data; the data preprocessing stage processes and reduces the dimension of the image data, odor data and infrared data; the feature extraction stage extracts the feature vectors of the image data, the feature vectors of the odor data and the feature vectors of the infrared data.

Images