Traditional Chinese medicine abnormal state recognition method, system and equipment and storage medium

Abstract

The invention provides a traditional Chinese medicine abnormal state recognition method, system and device and a storage medium, and relates to the field of abnormal recognition. The method comprises the following steps: acquiring product information of a to-be-detected traditional Chinese medicine, and determining a data acquisition standard of the to-be-detected traditional Chinese medicine based on the product information of the to-be-detected traditional Chinese medicine and a preset detection rule; based on a data acquisition standard of the traditional Chinese medicine to be detected, inputting the image data into a preset image anomaly detection model to obtain a preliminary image anomaly region; performing feature fusion on the visual feature data and the spectral feature data to obtain an abnormal feature vector; and inputting the abnormal feature vector into a preset abnormal state judgment model, outputting abnormal state data, and performing abnormal state judgment based on the abnormal state data and a preset abnormal state judgment threshold to obtain an abnormal state detection result of the to-be-detected traditional Chinese medicine. Through the scheme, the abnormal state of the traditional Chinese medicine can be identified timely and accurately.

Description

Technical Field

[0001] The present application relates to the field of abnormality identification, and specifically to a method, system, device and storage medium for identifying abnormal states of traditional Chinese medicine. Background Art

[0002] Chinese medicine is a treasure of traditional Chinese medicine and plays an important role in disease prevention and treatment. However, due to the complexity and variability of the planting, harvesting, processing, and storage of Chinese medicinal materials, Chinese medicine products are prone to uneven quality, adulteration, and other problems. Therefore, quality testing and abnormal state identification of Chinese medicine products are of great significance to ensure the quality of Chinese medicine and clinical drug safety.

[0003] At present, the commonly used methods for testing the quality of traditional Chinese medicine mainly include microscopic examination, physical and chemical identification, chromatographic analysis, etc. Although these methods can reflect certain characteristics of traditional Chinese medicine from different angles, most of them rely on manual operation and empirical judgment, are highly subjective, difficult to achieve standardized and automated testing, and have a long testing cycle. Summary of the invention

[0004] The present application provides a method, system, device and storage medium for identifying abnormal status of traditional Chinese medicine, which can timely and accurately identify abnormal status of traditional Chinese medicine.

[0005] In a first aspect, the present application provides a method for identifying abnormal states of traditional Chinese medicine, the method comprising: Obtain product information of the Chinese medicine to be tested, and determine data acquisition standards for the Chinese medicine to be tested based on the product information of the Chinese medicine to be tested and preset testing rules; Based on the data acquisition standard of the Chinese medicine to be detected, standard image data and standard spectrum data of the Chinese medicine to be detected are obtained, and the standard image data and standard spectrum data are preprocessed to obtain image data and spectrum data; Input the image data and the spectral data into a preset image anomaly detection model to obtain an image anomaly area and a spectral anomaly area; Extract features from abnormal image regions to obtain visual feature data, and extract features from abnormal spectral regions to obtain spectral feature data; Perform feature fusion on visual feature data and spectral feature data to obtain abnormal feature vector; The abnormal feature vector is input into a preset abnormal state discrimination model, abnormal state data is output, and abnormal state judgment is performed based on the abnormal state data and a preset abnormal state judgment threshold to obtain an abnormal state detection result of the Chinese medicine to be detected.

[0006] By adopting the above technical scheme, the product information of the Chinese medicine to be detected is first obtained, and based on the product information of the Chinese medicine to be detected and the preset detection rules, the data acquisition standard of the Chinese medicine to be detected is determined, so that according to the characteristics of different Chinese medicines, a suitable data acquisition scheme can be formulated to improve data quality and detection accuracy. Then, based on the data acquisition standard of the Chinese medicine to be detected, the standard image data and standard spectral data of the Chinese medicine to be detected are obtained, and the standard image data and standard spectral data are preprocessed to obtain image data and spectral data. Using the two data forms of image and spectrum, the characteristic information of Chinese medicine can be more comprehensively characterized, and data noise and interference can be eliminated through preprocessing, thereby improving the consistency and comparability of data. Then, the image data and spectral data are input into the preset image anomaly detection model to obtain the image abnormal area and the spectral abnormal area. Using the preset anomaly detection model, the abnormal state of Chinese medicine can be automatically and intelligently discovered, avoiding the subjectivity and instability of manual detection. Feature extraction is performed on the image abnormal area to obtain visual feature data, and feature extraction is performed on the spectral abnormal area to obtain spectral feature data. Extracting the feature data of the abnormal area can highlight the key information of the abnormal state and reduce the feature dimension and calculation complexity. The visual feature data and spectral feature data are fused to obtain the abnormal feature vector. Through feature fusion, the two abnormal information of image and spectrum can be comprehensively utilized to enhance the discriminative power and robustness of the features, and improve the accuracy and generalization ability of abnormal recognition. Finally, the abnormal feature vector is input into the preset abnormal state discrimination model, the abnormal state data is output, and the abnormal state judgment is performed based on the abnormal state data and the preset abnormal state judgment threshold to obtain the abnormal state detection result of the Chinese medicine to be detected. The abnormal state discrimination model is used for classification decision-making, which can automatically identify the specific abnormal type of Chinese medicine, control the sensitivity of discrimination through the abnormal state judgment threshold, and improve the reliability and applicability of detection.

[0007] In a second aspect of the present application, a method system for identifying abnormal state of traditional Chinese medicine is provided, comprising: A data acquisition module is used to acquire product information of the Chinese medicine to be tested, and determine data acquisition standards for the Chinese medicine to be tested based on the product information of the Chinese medicine to be tested and preset testing rules; A preprocessing module, based on the data acquisition standard of the Chinese medicine to be detected, obtains standard image data and standard spectrum data of the Chinese medicine to be detected, and preprocesses the standard image data and standard spectrum data to obtain image data and spectrum data; A first preliminary abnormal region detection module, used for inputting image data and spectral data into a preset image abnormality detection model to obtain image abnormal regions and spectral abnormal regions; The second preliminary abnormal region detection module is used to extract features from abnormal image regions to obtain visual feature data, and to extract features from abnormal spectral regions to obtain spectral feature data; A feature fusion module is used to fuse the visual feature data and the spectral feature data to obtain an abnormal feature vector; The state determination module is used to input the abnormal feature vector into a preset abnormal state determination model, output abnormal state data, and perform abnormal state determination based on the abnormal state data and a preset abnormal state determination threshold to obtain the abnormal state detection result of the Chinese medicine to be detected.

[0008] In a third aspect of the present application, a computer storage medium is provided, wherein the computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the above method steps.

[0009] In the fourth aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the above method.

[0010] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The present application first obtains the product information of the Chinese medicine to be detected, and based on the product information of the Chinese medicine to be detected and the preset detection rules, determines the data acquisition standard of the Chinese medicine to be detected, so that according to the characteristics of different Chinese medicines, a suitable data acquisition scheme can be formulated to improve data quality and detection accuracy. Then, based on the data acquisition standard of the Chinese medicine to be detected, the standard image data and standard spectrum data of the Chinese medicine to be detected are obtained, and the standard image data and standard spectrum data are pre-processed to obtain image data and spectrum data. Using two data forms of image and spectrum, the characteristic information of Chinese medicine can be more comprehensively characterized, and data noise and interference can be eliminated by pre-processing, thereby improving the consistency and comparability of data.

[0011] 2. This application inputs image data and spectral data into a preset image anomaly detection model to obtain image abnormality areas and spectral abnormality areas. By using a preset anomaly detection model, the abnormal state of traditional Chinese medicine can be automatically and intelligently discovered, avoiding the subjectivity and instability of manual detection. Feature extraction is performed on the image abnormal area to obtain visual feature data, and feature extraction is performed on the spectral abnormal area to obtain spectral feature data. Extracting the feature data of the abnormal area can highlight the key information of the abnormal state and reduce the feature dimension and computational complexity. Feature fusion is performed on the visual feature data and the spectral feature data to obtain an abnormal feature vector. Through feature fusion, the two types of abnormal information, image and spectrum, can be comprehensively utilized to enhance the discriminability and robustness of the features and improve the accuracy and generalization ability of abnormal recognition.

[0012] 3. This application inputs the abnormal feature vector into a preset abnormal state discrimination model, outputs abnormal state data, and performs abnormal state judgment based on the abnormal state data and the preset abnormal state judgment threshold to obtain the abnormal state detection result of the Chinese medicine to be detected. The abnormal state discrimination model is used for classification decision-making, which can automatically identify the specific abnormal type of Chinese medicine, control the sensitivity of the discrimination through the abnormal state judgment threshold, and improve the reliability and applicability of the detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram of a flow chart of a method for identifying abnormal status of traditional Chinese medicine provided in an embodiment of the present application; Figure 2 An architectural diagram of a Chinese medicine abnormal state recognition system provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0014] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0015] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0016] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0017] In order to facilitate understanding of the method and system provided by the embodiments of the present application, before introducing the embodiments of the present application, the background of the embodiments of the present application is first introduced.

[0018] Chinese medicine is a treasure of traditional Chinese medicine and plays an irreplaceable and important role in disease prevention and treatment. However, due to the complexity of Chinese medicinal materials themselves, as well as the variability of planting, harvesting, processing, storage and other links, it is difficult to control the quality of Chinese medicine products, and it is easy to have problems such as uneven quality and adulteration. Therefore, it is of great significance to establish a scientific, efficient and accurate system for quality detection and abnormal state identification of Chinese medicine to ensure the quality of Chinese medicine, ensure the safety of clinical drug use, and promote the healthy development of Chinese medicine.

[0019] At present, the commonly used methods for quality inspection of traditional Chinese medicine mainly include microscopic inspection, physical and chemical identification, chromatographic analysis, etc. Among them, microscopic inspection identifies and evaluates the quality of traditional Chinese medicine by observing the external morphological characteristics of traditional Chinese medicine, such as color, texture, cross-sectional characteristics, etc.; physical and chemical identification detects certain physical and chemical properties of traditional Chinese medicine, such as water content, ash content, extract content, etc., through chemical reactions, physical constant determination and other methods; chromatographic analysis such as thin layer chromatography and high performance liquid chromatography can be used to separate and identify the chemical components of traditional Chinese medicine and conduct qualitative and quantitative analysis. These traditional methods have played an important role in the quality inspection of traditional Chinese medicine, but they also have some limitations. For example, most of these methods rely on manual operation and experience judgment, are greatly affected by the quality and subjective factors of the inspectors, and it is difficult to achieve standardization, automation and quantification of the inspection process and results; at the same time, the sample processing and inspection steps are cumbersome, time-consuming and labor-intensive, and it is difficult to meet the growing demand for rapid inspection, which restricts the efficiency and timeliness of traditional Chinese medicine quality control.

[0020] After the background introduction of the above content, those skilled in the art can understand the problems existing in the prior art. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0021] Based on the above background technology, please refer to Figure 1 , Figure 1 A flow chart of a method for identifying abnormal status of traditional Chinese medicine provided in an embodiment of the present application. The system can be implemented by a computer program or can be run as an independent tool application. Specifically, in an embodiment of the present application, the method can be applied on a server, but can also be applied on electronic devices such as a server. A method for identifying abnormal status of traditional Chinese medicine includes the following steps: S101, obtaining product information of the Chinese medicine to be detected, and determining a data acquisition standard for the Chinese medicine to be detected based on the product information of the Chinese medicine to be detected and a preset detection rule; Specifically, based on the product information of the Chinese medicine to be detected, the present invention determines the data acquisition standard of the Chinese medicine to be detected. The formulation of this standard takes full account of the diversity and complexity of Chinese medicinal materials, and for different Chinese medicine varieties, from the macroscopic and microscopic levels, stipulates the corresponding image acquisition parameters (such as light source, resolution, field size, etc.) and spectrum acquisition parameters (such as wavelength range, scanning interval, integration time, etc.), and strives to comprehensively collect the morphology, color, tissue structure, chemical composition and other aspects of information of Chinese medicine samples, and provide reliable data support for subsequent quality evaluation and abnormal identification. It is worth mentioning that the setting of data acquisition standards also takes full account of the balance between detection efficiency and cost. While meeting the necessary amount of information, try to choose a fast and simple collection method, reduce sample preparation and manual operation links, and improve detection throughput and automation level.

[0022] S102, based on the data acquisition standard of the Chinese medicine to be detected, acquiring standard image data and standard spectrum data of the Chinese medicine to be detected, and preprocessing the standard image data and standard spectrum data to obtain image data and spectrum data; Specifically, after determining the data acquisition standard of the Chinese medicine to be tested, the method of the present invention further collects the required image and spectral data based on the standard. Specifically, for the acquisition of image data, a high-resolution digital camera or microscope can be used to shoot the surface and cross-sectional morphological characteristics of the Chinese medicine sample from multiple angles under standard light source conditions to obtain standard image data reflecting information on its color, texture, tissue structure, etc. For the collection of spectral data, a spectral instrument suitable for the Chinese medicine to be tested, such as a near-infrared spectrometer, a Raman spectrometer, etc., can be selected, and the Chinese medicine sample is spectrally scanned according to the established parameter settings, such as wavelength range, number of scans, resolution, etc., to obtain standard spectral data containing its chemical composition and structural information.

[0023] It should be pointed out that the original collected images and spectral data often have noise, background interference and other factors, which make it difficult to use them directly for subsequent analysis, so they need to be preprocessed. The preprocessing of standard image data mainly includes image denoising, enhancement, segmentation, correction and other operations, the purpose of which is to remove image noise, improve contrast, segment the area of ​​interest, and eliminate the influence of factors such as shooting angle and lighting conditions, so as to make the image clearer and more standardized, and facilitate the extraction of key morphological features. The preprocessing of standard spectral data includes spectral denoising, baseline correction, spectral normalization, feature extraction and other steps, aiming to eliminate spectral noise and background drift, highlight key absorption peaks, and extract characteristic spectra that can reflect the chemical properties of traditional Chinese medicine, so as to prepare for spectral interpretation and quantitative analysis.

[0024] Based on the above embodiment, as an optional embodiment, before acquiring the standard image data of the Chinese medicine to be detected, the method further includes: S201, based on the data acquisition standard of the Chinese medicine to be detected, acquiring multi-view image data of the Chinese medicine to be detected, and preprocessing the multi-view image data to obtain a standardized image to be detected; Specifically, multiple high-resolution digital cameras or a rotatable camera platform are used, and reasonable shooting angles and paths are designed according to the shape characteristics of Chinese medicinal materials. Generally, for tuber Chinese medicinal materials, they can be photographed from different angles such as the front, back, side, top, and bottom; for strip-stem Chinese medicinal materials, they can be imaged from different positions along the axial and radial directions; for spherical or irregularly shaped Chinese medicinal materials, they can be rotated around their outer surfaces to take pictures at equal angles. During the acquisition process, by adjusting camera parameters such as aperture, shutter, white balance, etc., strive to obtain multi-view image data with uniform brightness, true colors, and clear details under standard light conditions. At the same time, key parameters such as shooting angle and distance should also be recorded as a basis for image registration and three-dimensional reconstruction.

[0025] The collected multi-view image data is usually inevitably affected by factors such as uneven illumination, lens distortion, and depth of field blur. Therefore, necessary image preprocessing is required to improve image quality and comparability. Specifically, it can be divided into the following steps: first, the multi-view image is distorted to eliminate the image distortion caused by radial and tangential distortion of the lens; then, image enhancement techniques such as histogram equalization and Gaussian filtering are used to improve image contrast and smooth noise; thirdly, threshold segmentation and morphological processing are used to segment the target area of ​​Chinese medicine from the background to eliminate impurity interference; finally, the segmented Chinese medicine image is calibrated with reference to a ruler of known size to achieve the correspondence between pixel distance and actual size. After the above preprocessing, a standardized image to be inspected with uniform viewing angle, standard color, and accurate size can be obtained.

[0026] Compared with the conventional single-view or a small number of random view shooting methods, the multi-view image acquisition and standardized preprocessing of the present invention can record the morphological characteristics of Chinese medicinal materials more comprehensively and finely, and reduce the information omission caused by improper angle selection. By fusing and optimizing the image information of different view angles, on the one hand, the data quality is improved, and the problems of large noise and low clarity of a single image are overcome. On the other hand, the unification of image scale and color is achieved, which is convenient for comparative analysis of images of Chinese medicines of different batches and varieties. In addition, appropriate preprocessing of multi-view images can also filter out the interference of background and noise, highlight the characteristic information of Chinese medicine itself, thereby improving the speed and accuracy of subsequent feature extraction and abnormal discrimination.

[0027] S202, using a preset image segmentation model to perform foreground extraction on the standardized image to be inspected to obtain standard image data.

[0028] Specifically, a preset image segmentation model is used, namely a semantic segmentation network based on deep learning. Different from traditional threshold segmentation, edge detection and other methods, the semantic segmentation network can fully explore the contextual associations between pixels in the image, and directly predict the semantic category (such as foreground, background) to which each pixel belongs through end-to-end feature learning and multi-scale reasoning, thereby realizing pixel-level labeling of the image. Specifically, the current leading semantic segmentation network architecture, such as SegNet, DeepLab series, etc., can be used, and trained and tuned on massive Chinese medicinal materials image data, so that it can learn robust and accurate Chinese medicine foreground extraction capabilities.

[0029] When using the image segmentation model, the standardized image to be tested is input into the trained semantic segmentation network. After downsampling by the encoder and upsampling by the decoder, a pixel category probability map of the same size as the original image is generated. By setting a certain confidence threshold, the probability map is converted into a binary mask, that is, the target area of ​​the Chinese medicine is marked as 1 and the background area is marked as 0. Finally, the mask is used to perform pixel-level AND operations on the original image to accurately segment the foreground area of ​​the Chinese medicinal materials, and obtain standard image data that eliminates background interference and only contains the Chinese medicine target.

[0030] Compared with manual annotation and traditional segmentation methods, the image segmentation model based on deep learning adopted by the present invention has significant advantages. First, the semantic segmentation network can accurately grasp the morphological boundaries of Chinese medicine targets from a global perspective through end-to-end feature extraction and contextual reasoning, overcoming the problems of insufficient contrast, blurred edges and other difficulties in segmentation in traditional methods. Secondly, since the network model can automatically learn the diversity characteristics of Chinese medicinal materials through big data, it has better adaptability and robustness for Chinese medicine samples of different varieties and forms, greatly reducing the manual debugging of segmentation parameters. In addition, the calculation process of image segmentation is highly parallel, and with the help of GPU acceleration, near-real-time extraction of Chinese medicine foreground can be achieved, providing an efficient solution for batch processing of massive Chinese medicine images.

[0031] By performing foreground extraction on the standardized image, redundant information in the background area is eliminated, and subsequent feature analysis and anomaly discrimination are focused on the key quality attributes of the Chinese medicine target. While improving computational efficiency, the accuracy and reliability of anomaly detection are further improved. It is worth mentioning that the standard image data obtained by the present invention not only contains the two-dimensional foreground area of ​​the Chinese medicinal material, but can also further reconstruct its three-dimensional morphological model. By spatially aligning and fusing the multi-view segmentation results, complete and accurate three-dimensional point cloud or grid data of Chinese medicine can be obtained, providing a new data basis for higher-level quality characterization and artificial intelligence analysis.

[0032] Based on the above embodiment, as an optional embodiment, before obtaining the standard spectrum data of the Chinese medicine to be detected, the method further includes: S301, based on the data acquisition standard of the Chinese medicine to be detected, obtaining the visible light-near infrared full-band spectrum data of the Chinese medicine to be detected, and preprocessing the visible light-near infrared full-band spectrum data to obtain a standardized spectrum curve; Specifically, the present application adopts a highly sensitive visible-near infrared spectrometer to perform full-band scanning of Chinese medicine samples under standard light sources and temperature control conditions. The scanning wavelength range generally covers 350-2500nm, covering the visible light region and near infrared one and two regions, and the resolution can reach 1-2nm to ensure the complete collection and fine characterization of the spectral characteristics of Chinese medicine. In the process of spectrum collection, it is necessary to strictly control instrument parameters such as the number of scans, integration time, slit width, etc., and use a standard whiteboard for reflectivity correction to eliminate the spectral differences introduced by instrument factors as much as possible. In addition, for Chinese medicine samples of different dosage forms and different forms, it is also necessary to optimize the spectrum collection mode and sampling method according to their physical properties, such as transmission method, diffuse reflection method, fiber optic probe method, etc., and strive to obtain original spectral data with high signal-to-noise ratio and good reproducibility.

[0033] The collected visible-near-infrared full-band spectral data are usually inevitably affected by factors such as scattering, noise, and baseline drift. Therefore, necessary spectral preprocessing is required to improve the spectral quality and comparability. Specifically, it can be divided into the following steps: first, the full-band spectrum is scatter corrected to eliminate the scattering effect caused by differences in particle size and surface state; second, the spectral baseline drift and amplification error are eliminated by using methods such as detrending and standard normal variables; third, the spectral signal-to-noise ratio is improved by using moving average smoothing and wavelet transform denoising, and the characteristic absorption peaks of active ingredients and indicator substances are highlighted; finally, the corrected spectral data is normalized to achieve the unification of the scale of spectral curves measured by different batches and different instruments. After the above preprocessing, a standardized visible-near-infrared full-band spectral curve can be obtained.

[0034] Compared with the conventional single-band or narrow-band spectrum acquisition method, the full-band spectrum acquisition and standardized preprocessing of the present invention can more comprehensively and finely characterize the chemical properties of Chinese medicinal materials, and reduce the information omission caused by improper spectrum acquisition conditions or band selection. By correcting and optimizing the full-band spectral data, on the one hand, the spectral quality is improved, and the problems of large noise and serious baseline drift of the original spectrum are overcome. On the other hand, the standardization of spectral measurements under different instruments and different conditions is achieved, which is convenient for the comparative analysis of spectra of Chinese medicinal materials from different batches and different origins. In addition, denoising and feature enhancement of full-band spectral data can also effectively reduce background interference, highlight the spectral characteristics of key effective components of Chinese medicinal materials, thereby providing more reliable data support for subsequent chemometric modeling and abnormal discrimination.

[0035] S302, based on the prior component information of the Chinese medicine to be detected, the standardized spectral curve is screened for characteristic wavelengths to obtain characteristic bands, and the spectral data of the characteristic bands are used as standard spectral data.

[0036] Specifically, when screening characteristic wavelengths, it is necessary to make full use of the prior component information of the Chinese medicine to be tested, that is, the known chemical composition and quality attributes. By consulting literature, quality standards and historical test data, the effective ingredients, indicator ingredients and characteristic ingredients that are prone to quality problems that play a decisive role in the quality of Chinese medicinal materials are summarized. On this basis, the physical and chemical properties and spectral response characteristics of these key ingredients are analyzed, and their characteristic absorption ranges in the visible-near infrared spectrum are preliminarily determined. Then, spectral analysis and correlation studies are carried out on the standardized spectral curve, and the sensitive characteristic bands and their width ranges of each component are further optimized through indicators such as spectral resolution, signal-to-noise ratio, and characteristic absorption peak intensity. After selecting several alternative characteristic bands, it is also necessary to use methods such as cross-validation and independent test sets to evaluate the actual prediction effect and modeling robustness of each band, and finally select the optimal characteristic band combination with good stability, good effect and strong representativeness. The spectral data of one or more selected characteristic bands are combined in sequence to obtain the standard spectral data input.

[0037] Compared with the use of full-band spectral data, the screening of characteristic wavelengths based on prior component information has obvious advantages. First, characteristic wavelength screening makes full use of the background knowledge of the object to be tested, converts empirical information and historical data into targeted spectral screening strategies, ensures the high correlation between the selected bands and key quality attributes, and reduces the introduction of invalid information. Secondly, through multiple evaluation and optimization criteria, characteristic wavelength screening reduces the data dimension while retaining the information content of spectral data to the greatest extent, overcomes the multicollinearity problem in the full-band spectrum, and improves the information density and modeling efficiency of the data. In addition, the dimension and combination method of the characteristic bands can be flexibly adjusted according to actual needs. Either a single global optimal band can be selected, or multiple local optimal bands can be combined to facilitate subsequent spectral analysis and quality factor analysis.

[0038] S103, inputting the image data and the spectral data into a preset image anomaly detection model to obtain an image anomaly region and a spectral anomaly region; Specifically, in the abnormal Chinese medicine state identification method provided by the present invention, it is a key step to input image data and spectral data into a preset image anomaly detection model to obtain image anomaly areas and spectral anomaly areas. Among them, image data refers to data obtained by preprocessing the standard image data of the Chinese medicine to be detected, reflecting the visual feature information of the Chinese medicine sample, such as color, shape, texture, etc. Spectral data refers to data obtained by preprocessing the standard spectral data of the Chinese medicine to be detected, reflecting the chemical composition and structural information of the Chinese medicine sample, such as absorption peaks, reflectivity, etc. The reason why both image data and spectral data are input into the anomaly detection model is that a single data source is often difficult to fully reflect the quality status of Chinese medicine and is easily affected by local anomalies or interference. By fusing the two data sources of image and spectrum, the characteristics of Chinese medicine can be characterized from different angles, complementing and verifying each other, and improving the reliability and accuracy of anomaly detection.

[0039] In specific implementation, a pre-trained deep learning model can be used as an image anomaly detection model, such as a convolutional neural network (CNN), an autoencoder (AE), etc. The image data and spectral data are respectively input into different branches of the model, and the multi-level features of the image and spectrum are extracted through operations such as convolution, pooling, and activation. Then, the anomaly score of the feature is calculated by the anomaly measurement function, and the normal area and the abnormal area are divided according to the preset threshold. Among them, the convolution operation can extract the local features of the image and spectrum, the pooling operation can reduce the feature dimension and calculation amount, and the activation function can increase the nonlinearity and expression ability of the feature. The anomaly measurement function can measure the degree to which the feature deviates from the normal sample. Commonly used ones are Euclidean distance, Mahalanobis distance, KL divergence, etc. The abnormal area refers to the image block or spectral segment whose anomaly score exceeds the threshold, reflecting the potential abnormal part or component of the traditional Chinese medicine sample.

[0040] S104, extracting features from the abnormal image area to obtain visual feature data, and extracting features from the abnormal spectrum area to obtain spectral feature data; Specifically, after obtaining the abnormal image area and the abnormal spectrum area, the abnormal Chinese medicine state identification method provided by the present invention also needs to extract features from the abnormal area to obtain visual feature data and spectral feature data. Among them, visual feature data refers to data extracted from the abnormal image area that reflects the visual properties of Chinese medicine, such as color histogram, local binary pattern, scale-invariant features, etc. Spectral feature data refers to data extracted from the abnormal spectrum area that reflects the properties of Chinese medicine ingredients, such as characteristic absorption peak position, peak intensity, peak width, etc. The reason why the abnormal area is to be feature extracted is that although the abnormal area reflects the potential abnormal parts or components of Chinese medicine, its original data has high dimensionality and information redundancy, which is not conducive to subsequent analysis and discrimination. Through feature extraction, the key information of the abnormal area can be condensed and refined to obtain a more discriminative feature representation, reduce storage and computing overhead, and improve the efficiency and accuracy of abnormal state recognition.

[0041] In specific implementation, the feature extraction of abnormal areas of images can use traditional image feature descriptors, such as color histograms to characterize the color distribution of abnormal areas, local binary patterns to characterize the texture structure of abnormal areas, and scale-invariant features to characterize the salient points and edges of abnormal areas. These features have certain invariance to image translation, rotation, scale changes, etc., which can improve the robustness of abnormal area representation. In addition, deep learning methods, such as convolutional neural networks, can be used to input abnormal areas into pre-trained feature extraction networks to automatically learn hierarchical visual features.

[0042] The feature extraction of abnormal spectral regions can adopt common spectral analysis techniques, such as characteristic absorption peak detection, baseline correction, spectral curve fitting, etc. Among them, the characteristic absorption peak is the local maximum point of the spectral curve, which corresponds to the specific chemical group or structural fragment of the substance to be tested and is closely related to the composition and content of the substance. By detecting the characteristic absorption peak position, peak intensity, peak width and other parameters in the abnormal area, the abnormal component properties can be revealed. Baseline correction and spectral curve fitting can eliminate background noise and defects in the spectral measurement process and improve the signal-to-noise ratio of characteristic absorption peak extraction.

[0043] S105, performing feature fusion on the visual feature data and the spectral feature data to obtain an abnormal feature vector; Specifically, when implementing feature fusion, a variety of strategies can be adopted. One idea is to first perform feature selection on the visual feature data and spectral feature data respectively, screen out several key features with the strongest discrimination, and then simply concatenate or linearly combine these features to obtain the fused feature vector. This method is simple to implement and has high computational efficiency, but there may be information redundancy between features. Another more advanced fusion strategy is to use machine learning methods such as multi-kernel learning and manifold learning to explore their intrinsic associations in a high-dimensional or low-dimensional space while maintaining the physical meaning of different features, forming a new unified feature representation. For example, the multi-kernel support vector machine (MK-SVM) extracts pattern information of different scales from visual and spectral features by setting different kernel functions, and performs optimal fusion on the classification boundary; multi-view manifold learning maps the original multi-source heterogeneous features to a common low-dimensional manifold space through nonlinear transformation, achieving dimensionality reduction compression and collaborative representation of features. After feature fusion, not only the scale of feature data is greatly reduced, the amount of subsequent calculations is reduced, but also the complementary and synergistic relationship between visual and spectral features is excavated, forming a more accurate and comprehensive characterization of the sample state.

[0044] Based on the above embodiment, as an optional embodiment, feature fusion is performed on the visual feature data and the spectral feature data to obtain an abnormal feature vector, including: S401, performing feature alignment processing on the visual feature data and the spectral feature data to obtain aligned visual feature data and aligned spectral feature data, and using a multi-view representation learning method to fuse the aligned visual feature data and the aligned spectral feature data to obtain a multi-view representation learning method. Figure 1 Somatic characteristics; Specifically, when implementing feature fusion, the present application first needs to align the visual feature data and the spectral feature data. Since the visual image and the spectral curve have differences in the acquisition process, signal source, data format, etc., it is necessary to perform necessary alignment before fusion, including pre-processing operations such as spatial registration, time synchronization, and data correspondence. Taking spatial registration as an example, since the acquisition viewing angle and resolution of the visual image and the spectral signal are usually inconsistent, it is necessary to correspond the spectral sampling points to the image pixels one by one through geometric transformation (such as projection transformation, affine transformation) and interpolation calculation to ensure the precise matching of the two types of data in space. Similarly, for data collected in different batches and at different times, batch correction and time alignment are also required. For another example, for the visual and spectral detection data of the same batch of samples, it is necessary to match one by one according to index information such as sample encoding to ensure the data correspondence of subsequent fusion analysis. Through a series of alignment processes, the aligned visual feature data and aligned spectral feature data of spatial registration and time synchronization can be obtained.

[0045] On this basis, a multi-view representation learning method is further used to fuse the aligned visual-spectral multi-source data. Multi-view learning is an effective way to fuse heterogeneous data. Its core idea is to map different view data to a common subspace, optimize the consistency and complementarity of different views in the subspace, and thus obtain a unified compact representation. Common multi-view learning methods include canonical correlation analysis (CCA), multi-view spectral embedding (MSE), and multi-view dictionary learning (MDL). Taking CCA as an example, by finding the linear transformation of different view features, the maximum correlation of visual-spectral features in the common space is constructed, thereby achieving effective alignment of heterogeneous features; for example, MSE, through manifold learning and local preservation projection, the visual-spectral features are mapped to a low-dimensional manifold space, and the neighbor relationship consistency constraint of different view samples is realized. In general, a variety of representation learning methods can be used in combination, and through strategies such as weighted fusion or cascade decomposition, the consistency and difference of visual-spectral data can be explored from a complementary perspective, and a multi-view representation learning method with scale normalization, consistent distribution, and complete information can be constructed. Figure 1 Expression of physical features.

[0046] Through feature alignment and multi-view representation learning, the present invention realizes the deep fusion of visual information and spectral information, and can stereoscopically characterize the quality status of traditional Chinese medicine samples from two dimensions: macroscopic morphology and microscopic composition. Compared with single visual detection or spectral detection, multi-source fusion can capture multiple aspects of traditional Chinese medicine quality issues and provide a more comprehensive and reliable comprehensive criterion. On the one hand, the alignment mechanism eliminates the redundancy of the original multi-source data, highlights the complementary correlation of signals from different views, and lays a solid foundation for data fusion; on the other hand, multi-view representation learning can adaptively extract unified fusion features from massive data, which not only retains the complementary information of different data sources, but also overcomes the blindness of artificially designed features, providing a low-redundancy, high-discrimination decision-making basis for anomaly detection. In addition, the aligned and fused multi-view Figure 1 The consistent scale and integrated distribution of the physical characteristics facilitate the quality comparison of different batches and varieties of Chinese medicine samples, and also provide data support for subsequent anomaly location and traceability analysis.

[0047] Based on the above embodiment, as an optional embodiment, a multi-view representation learning method is used to fuse the aligned visual feature data and the aligned spectral feature data to obtain a multi-view feature data. Figure 1 Physical characteristics include: S501, constructing a visual feature view based on the aligned visual feature data, and constructing a spectral feature view based on the aligned spectral feature data; Specifically, when implementing the construction of multi-view representation, independent feature views are constructed for the aligned visual feature data and spectral feature data respectively. For visual feature data, a set of visual original features, such as morphological features, color features, texture features, etc., are obtained through operations such as feature extraction and feature selection, and they are organized into a unified visual feature vector to form a visual feature view; similarly, for spectral feature data, a set of spectral original features, such as reflection peak position, absorption intensity, spectral curvature, etc., are obtained through operations such as spectral preprocessing and feature extraction, and they are organized into a unified spectral feature vector to form a spectral feature view. When constructing each view, in order to fully explore the discriminative information of the original data, a multi-level and multi-scale feature representation framework is usually adopted, that is, based on the pixel-level features, high-level semantic features at the regional and global levels are extracted, and view hierarchical features are constructed through methods such as feature combination optimization. In addition, considering that multi-view learning is sensitive to the balance of different view dimensions, it is usually necessary to appropriately screen and transform the original features when constructing the view so that the features of different views are as balanced as possible in terms of dimension and scale.

[0048] Multi-view representation provides a new perspective for the fusion of heterogeneous data. Compared with simple feature splicing, multi-view representation does not need to forcibly combine features from different sources, but implements feature mapping and association analysis in a higher-level semantic space, which helps to essentially eliminate the distribution differences and modal splits of heterogeneous features. On the one hand, by constructing visual, spectral and other feature views separately, the uniqueness of heterogeneous data can be fully reflected, and the discriminant information of different modalities can be fully mined; on the other hand, by realizing the fusion mapping of different views in a common semantic space, the complementarity and consistency of heterogeneous data can be fully utilized, and multi-source information can be interacted and integrated at a deeper level. It is worth mentioning that the method of the present invention makes full use of empirical knowledge and data-driven analysis when constructing multi-view representation. While exploring the intrinsic discriminant features of visual and spectral data, it can introduce expert knowledge in the field of traditional Chinese medicine to assist in multi-level and multi-scale feature extraction, so that the constructed feature views are more in line with the professional needs of Chinese herbal medicine quality assessment, and more targeted and interpretable.

[0049] S502, constructing a visual feature view and a spectral feature view, and applying a manifold learning method in the visual feature view and the spectral feature view to maintain a local neighborhood structure of features, to obtain a first visual feature view and a first spectral feature view; Specifically, when applying the manifold learning method to optimize each view, the manifold structure mining and preservation are performed for the visual feature view and the spectral feature view respectively. Taking the visual feature view as an example, by calculating the original similarity matrix between samples, such as Euclidean distance, angle cosine, etc., a neighbor graph reflecting the initial association between samples is constructed; on the basis of the neighbor graph, the geodesic distance between samples is reconstructed to correct the influence of nonlinear distortion on the similarity metric, and the manifold similarity matrix of the visual view is obtained; finally, the visual feature view reconstructs the features of each sample by optimizing the objective function, and realizes the enhancement of the discriminant structure within the view under the criteria of satisfying local linear embedding, Laplace eigenmapping, etc., and obtains the first visual feature view. Similarly, the above manifold learning steps are repeated in the spectral feature view to realize the preservation of the local neighborhood structure of the spectral view and obtain the first spectral feature view. In practical applications, representative manifold learning algorithms such as equal metric learning, isometric mapping, and local linear embedding can be comprehensively adopted to construct personalized manifold mapping criteria through collaborative optimization to improve the accuracy and robustness of feature learning.

[0050] By applying the manifold learning method in each view, the method of the present invention further strengthens the discriminability in each view on the basis of constructing different modal views, and lays a more solid feature foundation for subsequent fusion learning. On the one hand, manifold learning can extract deep intrinsic structural information from high-dimensional nonlinear data, correct the distortion and redundancy of the original feature space, and learn a more compact and smooth manifold representation, so that similar samples are more aggregated and heterogeneous samples are further separated, avoiding isolated points and short-circuit connection problems. In this way, subsequent fusion learning will be able to more accurately extract the common features of different views and effectively avoid nonlinear interference and overfitting risks. On the other hand, by optimizing the manifold of visual and spectral views respectively, the discriminability of different modal views is enhanced, and the differential features and detail information are further highlighted, creating conditions for mining the complementarity of multiple views. It can be foreseen that on the basis of the first visual feature view and the first spectral feature view after manifold optimization, multi-view fusion will achieve "1+1>2" aggregation enhancement, significantly improving the recall and precision of Chinese medicine anomaly detection.

[0051] S503, learning the collaborative representation of the first visual feature view and the first spectral feature view through a bilinear mapping function to obtain a second visual feature view and a second spectral feature view; Specifically, when implementing the collaborative representation learning of bilinear mapping, the first visual feature view and the first spectral feature view are used as input, and the construction of the joint semantic space is realized by optimizing the parameters of the bilinear model. First, the projection matrix of the bilinear mapping function is randomly initialized, and the visual view features and the spectral view features are respectively mapped to the shared subspace to obtain the initial collaborative representation; then, by minimizing the similarity error between the collaborative representations of different views, such as geodesic distance, manifold sorting error, etc., the parameters of the bilinear mapping function are optimized so that the collaborative representations of different views are as close as possible; at the same time, in order to avoid overfitting and enhance the generalization of the model, a regularization term is usually introduced in the optimization objective to constrain and smooth the parameters of the bilinear model; finally, by iteratively optimizing the bilinear model, the collaborative representation of the visual view and the spectral view converges to the optimal solution, and the second visual feature view and the second spectral feature view are obtained respectively. In practical applications, efficient algorithms such as alternating optimization and gradient descent can be used to solve the bilinear model, and strategies such as early stopping mechanism and multi-view joint loss are introduced to enhance the robustness and convergence speed of learning.

[0052] The method of the present invention realizes multi-view collaborative representation learning through bilinear mapping. On the basis of independent optimization of different modalities, it further establishes the association and interaction between visual and spectral views, and realizes the fusion of multi-source data at the semantic level. On the one hand, the bilinear model can learn the collaborative relationship between different views, transform the visual and spectral features into a common implicit space through mapping, so that different views obtain a consistent representation form, overcome the "heterogeneous gap", and realize the migration of multimodal features from a specific domain to a public domain. In the collaborative representation of each other's knowledge, the differences between modalities are smoothed, and the complementary characteristics are stimulated, which creates conditions for the subsequent feature-level fusion. On the other hand, the manifold structure information is incorporated into the bilinear mapping, and the discriminability within the view is maintained while learning the association between views. The collaborative representation not only inherits the advantages of the original view, but also learns new modal fusion characteristics, and has stronger semantic expression ability than a single view. Under the collaborative representation of the second visual feature view and the second spectral feature view, the abnormal characteristics of the Chinese herbal medicine sample will be more significant, the advantages of different modal data will be concentrated, and the efficiency of multi-view fusion will be fully exerted.

[0053] Based on the above embodiment, as an optional embodiment, learning the collaborative representation of the first visual feature view and the first spectral feature view through a bilinear mapping function to obtain the second visual feature view and the second spectral feature view includes: S601, initializing the parameters of the bilinear mapping function, and constructing a visual mapping matrix and a spectral mapping matrix based on the parameters of the bilinear mapping function; Specifically, when implementing the bilinear model parameter initialization and mapping matrix construction, firstly, according to the dimensions of the first visual feature view and the first spectral feature view, the weight matrix and bias vector and other parameters of the bilinear mapping function are initialized. Common initialization strategies include all-zero initialization, random initialization, Xavier initialization, etc. Among them, Xavier initialization can make the variance of each layer output as equal as possible, which is a more commonly used initialization method. After obtaining the initialization parameters, based on the weight matrix, combined with the original dimensions of the visual features and spectral features, the visual mapping matrix and the spectral mapping matrix are constructed respectively. Assuming that the feature dimension of the first visual feature view is m, the feature dimension of the first spectral feature view is n, and the target dimension of the collaborative representation space is d, the dimension of the visual mapping matrix is ​​m×d, and the dimension of the spectral mapping matrix is ​​n×d. The bilinear model realizes the linear transformation from the original feature space to the collaborative representation space by multiplying the visual features with the visual mapping matrix and the spectral features with the spectral mapping matrix. In practical applications, different parameter initialization strategies and mapping matrix construction methods can be flexibly selected, and combined with regularization, parameter sharing and other mechanisms to enhance the adaptability and generalization of bilinear model learning.

[0054] By properly initializing the bilinear model parameters and constructing feature mapping matrices for different views, the method of the present invention lays a solid foundation for collaborative representation learning. On the one hand, reasonable initial parameter values ​​can accelerate the optimization solution of the bilinear model, making it converge to the optimal solution faster and alleviating the difficulty of model training, which is particularly important for high-dimensional and sparse multimodal data of traditional Chinese medicine. Although the mapping matrix obtained by initialization may not be optimal, it can better respond to the original feature distribution of different views, providing a favorable starting point for subsequent parameter fine-tuning. On the other hand, the construction of the visual mapping matrix and the spectral mapping matrix is ​​directly related to the quality of multi-view collaborative representation. By multiplying the features of each view with the corresponding mapping matrix, the original visual and spectral features can be transformed into a common semantic space, realizing the convergence and fusion of different view discrimination information in the new space. Even in the initial stage, this feature transformation carried by the mapping matrix can alleviate the "heterogeneous gap" to a certain extent and create conditions for mining the complementarity between views. Therefore, with the mapping matrix constructed with the initialization parameters as a bridge, multi-view collaborative representation learning will be more efficient and robust, which lays a solid foundation for subsequent optimization of parameters and extraction of integrated fusion features.

[0055] S602, mapping the visual mapping matrix and the spectral mapping matrix from the original feature space to a common subspace, and solving the optimization problem of collaborative representation learning in the common subspace to obtain an optimal inter-view mapping matrix; Specifically, when implementing the mapping of the visual mapping matrix and the spectral mapping matrix to the common subspace and solving the collaborative representation learning optimization problem, the transformation mapping from the original space to the common subspace is learned by optimizing and minimizing the representation differences of the features after different view mappings based on the initially constructed visual mapping matrix and the spectral mapping matrix. Common optimization objectives include cross entropy loss, mean square error, etc., which are used to measure the semantic similarity of different view mapping features. Taking the mean square error as an example, by minimizing the reconstruction error between the visual mapping features and the spectral mapping features, they are made as close as possible in the common subspace. At the same time, by constraining the low rank of each view mapping matrix, a more compact collaborative representation can be learned. After constructing the optimization objective function, the gradient descent method is used to iteratively optimize and solve it, and the basis vectors of the visual mapping matrix, the spectral mapping matrix and the common representation space are updated alternately, and finally the optimal collaborative mapping matrix between views is obtained in the common subspace. It is worth mentioning that in order to further enhance the discriminability and generalization of collaborative representation, prior constraints such as regularization terms and multi-view label consistency can be introduced into the optimization objectives, and the convergence speed and generalization ability of model training can be controlled through strategies such as learning rate adjustment and early stopping.

[0056] By mapping the visual mapping matrix and the spectral mapping matrix to a common subspace and solving the optimization problem of collaborative representation learning in this space, the method of the present invention further strengthens the consistency of the discriminant information of different views on the basis of multi-view semantic alignment, and realizes the deep mining of multimodal complementary characteristics. On the one hand, the introduction of the common subspace solves the problem of large differences in the spatial distribution of the original visual and spectral features, and realizes the effective alignment of different views at the semantic level through feature transformation, providing a unified optimization platform for collaborative learning. The nonlinear mapping from the original space to the common subspace essentially realizes the consistent matching of the multi-view discriminant structure, which not only maintains the discriminative power of each view, but also alleviates the "heterogeneous gap" between views. On the other hand, under the framework of the common subspace, the representation difference of the multi-view mapping features is optimized by cross entropy loss, etc., which maximizes the mining of the complementary synergy of different views. The optimization process iteratively refines the semantic association between views, weakens the influence of view redundancy and noise, and realizes the "complementary" aggregation of discriminant information to a certain extent. Therefore, multi-view collaborative learning on the common subspace will make the extracted integrated features more complete in representation and more accurate in discrimination, thereby providing a more reliable basis for the precise detection of abnormalities in traditional Chinese medicine.

[0057] S603 : Map the first visual feature view and the first spectral feature view to a common subspace using an optimal inter-view mapping matrix to obtain a second visual feature view and a second spectral feature view.

[0058] Specifically, when the optimal inter-view mapping matrix is ​​used to map the first visual feature view and the first spectral feature view to a common subspace, the learned optimal visual mapping matrix and the optimal spectral mapping matrix are used as a bridge to realize the linear transformation from the original feature space to the common subspace through matrix multiplication. Suppose the number of samples of the first visual feature view is x, the feature dimension is d1, the number of samples of the first spectral feature view is y, the feature dimension is d2, and the dimension of the common subspace is k, then the dimension of the optimal visual mapping matrix is ​​d1×k, and the dimension of the optimal spectral mapping matrix is ​​d2×k. For each sample of the first visual feature view, its feature vector is multiplied by the optimal visual mapping matrix to transform it to a k-dimensional common subspace, thereby obtaining the representation of the sample in the second visual feature view; similarly, using the optimal spectral mapping matrix, the sample features of the first spectral feature view can be transformed to the common subspace to obtain the second spectral feature view. It is worth noting that in order to ensure the comparability of the common representations after mapping from different views, the original features of each view are usually normalized before mapping, such as z-score normalization, l2 normalization, etc., to eliminate the influence of feature dimension and scale differences.

[0059] By using the optimal inter-view mapping matrix to map the first visual feature view and the first spectral feature view to the common subspace, the second visual feature view and the second spectral feature view are obtained. The method of the present invention further realizes the effective fusion of multi-view features at the semantic level on the basis of personalized mining of view discriminative power. On the one hand, the mapping transformation of the common subspace realizes the consistency calibration of the discriminative information of different views, which not only retains the original view discriminative structure, but also bridges the "heterogeneous gap" between views. The common representation bridged by the optimal mapping matrix integrates the multi-source discriminative information of visual and spectral perspectives, which not only continues the inherent manifold structure of each view, but also learns a higher level of semantic representation, which will provide more comprehensive and more refined quality features for anomaly detection. On the other hand, the distribution of the second visual feature view and the second spectral feature view obtained by mapping different views in the common space will be more consistent, and the discriminative differences will be amplified, while the complementary characteristics will also be integrated to a certain extent. Compared with a single perspective, the multi-perspective fusion representation formed in the common space will have a more powerful semantic expression ability, providing a more reliable basis for describing the overall quality of Chinese medicinal materials and identifying local anomalies. Subsequent multi-view learning will achieve a leap forward from fusion to aggregation, and from consistency to complementarity on this basis.

[0060] S504, introduce multi-view Figure 1 Consistency constraints are used to optimize the consistency of the second visual feature view and the second spectral feature view to obtain a multi-view Figure 1 Physical characteristics.

[0061] Specifically, in the implementation of multi-view Figure 1In the consistency constraint optimization, based on the collaborative representation of the second visual feature view and the second spectral feature view, a unified multi-view feature map is constructed by introducing criterion functions such as inter-view similarity regularization and intra-view discrimination constraint. Figure 1 Consistency optimization model. Taking the inter-view similarity regularization as an example, by minimizing the difference in similarity matrices of collaborative representations of different views, such as matrix L2 norm, nuclear norm, etc., the semantic distance between visual view and spectral view is explicitly shortened to ensure that they have a consistent reflection of abnormal characteristics of traditional Chinese medicine; at the same time, within each view, through discriminant criterion functions such as intra-class aggregation and inter-class separation, the feature similarity of samples of the same class is maximized, and the feature similarity of samples of different classes is minimized, so as to further strengthen the discriminative power and mutual exclusivity of each view. Figure 1 In the consistency model, different criterion functions are weighted and combined to collaboratively mine the commonalities between views and the individualities within views, and the collaborative representation of each view is updated using an alternating iterative optimization strategy. Ultimately, the visual view and the spectral view are fully aligned in the semantic space, the redundant information is minimized, and the complementary characteristics are highlighted, thereby obtaining a unified multi-view image. Figure 1 In practical applications, we can flexibly use graph regularization, kernel space alignment, manifold reconstruction and other methods to construct personalized consistency constraints, and introduce strategies such as adaptive weighting and kernel function selection to enhance the adaptability and generalization of the model.

[0062] By introducing multi-view Figure 1 The consistency constraint optimizes the collaborative representation. The method of the present invention greatly enhances the discriminability and mutual exclusivity of multi-view fusion features, and realizes a comprehensive and accurate characterization of abnormal information of traditional Chinese medicine. On the one hand, the inter-view similarity regularization explicitly models the consistency of the discriminant features of different views. By shortening the distance between the visual and spectral views in the semantic space, the collaborative representation produces consistent discrimination for the same traditional Chinese medicine sample, overcoming the "semantic gap" of view representation and realizing the synchronous presentation of multi-view discriminant information. On the other hand, the intra-view discriminant constraint maximizes the individual characteristics of each view representation. Through intra-class aggregation and inter-class separation, it ensures that the fused features inherit the discriminant advantages of different perspectives such as vision and spectrum, and the complementary characteristics are fully reflected, and the details that are difficult to depict in a single view are captured. In a unified multi-view Figure 1 In the integrated features, the discriminant information from different modalities achieves "complementary advantages", data redundancy is minimized, while mutual exclusivity is maximized, and the fused feature representation is compact and robust, which can more accurately reflect the intrinsic quality of Chinese herbal medicines than a single view. Therefore, based on the integrated fusion features, the recall and precision rates of Chinese herbal medicine anomaly detection will be greatly improved, and the real difficulties such as missing labeled data and small samples will also be alleviated.

[0063] S402, after fusion of multiple views Figure 1The dimensionality reduction processing of the individualized features is performed to obtain a low-dimensional abnormal feature vector, and a feature selection method based on L1 regularization is adopted to automatically search for low-dimensional abnormal feature vectors by introducing a sparse penalty term to obtain a sparse abnormal feature vector representation; Specifically, when implementing feature dimensionality reduction and selection, we first use the classic dimensionality reduction method to reduce the multi-view Figure 1 The dimensionality reduction is carried out by reducing the dimension of the whole features. Common dimensionality reduction methods include principal component analysis, linear discriminant analysis, kernelization dimensionality reduction, etc. Taking principal component analysis as an example, this method maps the original high-dimensional features to a set of low-dimensional orthogonal bases through linear transformation, retains the main variance information of the sample, and filters out noise and redundant components, thereby obtaining a compact low-dimensional abnormal feature vector. In practical applications, principal component analysis decomposes the covariance matrix through eigenvalues, selects the eigenvectors corresponding to the first few largest eigenvalues ​​as the principal components, and realizes the linear mapping from high dimension to low dimension. It is worth mentioning that the present invention fully considers the category structure information of the sample when reducing the dimension, and introduces supervised information to guide the dimensionality reduction process, such as supervised principal component analysis, discriminant analysis, etc. Compared with unsupervised dimensionality reduction, supervised dimensionality reduction can maximize the separability of abnormal samples and normal samples in low-dimensional subspaces, which is more conducive to subsequent abnormal discrimination and classification decisions.

[0064] On this basis, the feature selection method based on L1 regularization is further used to search for the most discriminative sparse feature subset from the low-dimensional anomaly feature vector. L1 regularization is a structured sparse representation method based on the L1 norm. By introducing the L1 norm penalty term in the optimization objective, the weights of irrelevant features can be adaptively compressed to zero, thereby obtaining a sparse representation of key features. Common L1 regularization feature selection methods include Lasso regression and L1 regularized support vector machine. Taking Lasso regression as an example, when minimizing the empirical risk of the anomaly discrimination model, this method adds the L1 norm penalty term to make the learning algorithm prefer the sparse weight solution and automatically realize the feature screening function; when the regularization term coefficient is appropriate, only the weights corresponding to the few features most relevant to the anomaly judgment are non-zero, and the rest are compressed to zero. Specifically, Lasso regression solves the least squares problem containing the L1 regularization term and iteratively optimizes the weight coefficients of each feature until it converges to the sparse optimal solution. In this way, the optimal abnormal feature combination can be searched from the low-dimensional feature vector to achieve automatic extraction and sparse coding of key abnormal information.

[0065] Through dimensionality reduction compression and L1 regularization feature selection, the present invention realizes the automatic extraction and optimized representation of abnormal features from high dimension to low dimension, from redundant to concise. Compared with the original high-dimensional multi-view features, the sparse abnormal feature vector obtained not only has a greatly reduced dimension, but also eliminates a large amount of redundant information irrelevant to abnormal judgment, highlighting the key features that are most discriminative for abnormal samples. On the one hand, feature dimensionality reduction effectively controls the "dimensionality curse", making the subsequent training of abnormal detection models more convenient and efficient, while the supervised dimensionality reduction that incorporates category information further enhances the prominence of abnormal information. On the other hand, the introduction of L1 regularization embeds the feature selection process into the learning of the abnormal discrimination model, adaptively determines the optimal discriminant features by learning sparse representation, overcomes the blindness of artificially designed features, and provides a solid foundation for fast and accurate abnormal detection. In addition, the low-dimensional sparse features after dimensionality reduction and feature selection are more concise and efficient, greatly reducing the subsequent transmission, storage and calculation costs, which is conducive to the lightweight and real-time realization of the Chinese medicine abnormality detection system.

[0066] S403, using a supervised dimensionality reduction method based on the Fisher criterion, optimizing the inter-class divergence and intra-class divergence represented by the abnormal feature vector to obtain the abnormal feature vector.

[0067] Specifically, after obtaining the sparse abnormal feature vector representation, the abnormal Chinese medicine state identification method provided by the present invention further utilizes the supervised dimensionality reduction method based on the Fisher criterion to optimize the inter-class divergence and intra-class divergence of the abnormal feature vector representation, and obtain an abnormal feature vector with stronger discriminability. Among them, the Fisher criterion is a classic linear supervised dimensionality reduction method, which maximizes the separability of samples in the feature space by explicitly modeling the mean vectors and covariance matrices of different categories, and realizes the migration from the unsupervised feature subspace to the supervised discriminant subspace. The reason why the Fisher criterion is embedded on the basis of the sparse abnormal feature representation for supervised dimensionality reduction optimization is that although relying solely on unsupervised sparse representation can filter out most of the redundancy and noise and highlight the key patterns of abnormalities, it does not directly use the category information of the abnormal state to constrain and enhance the feature semantics. By introducing category priors through the Fisher criterion, the feature space can be reconstructed from the perspective of discrimination, so that the abnormal class and the normal class are separated as much as possible in statistical distribution, while being clustered as much as possible within each class, avoiding feature ambiguity caused by inter-class confusion and intra-class multimodality, and further improving the recognition and robustness of abnormal features.

[0068] In specific implementation, the optimal discriminant projection direction of the Fisher criterion can be solved by iterative optimization. Taking the current sparse abnormal feature vector as the initial feature, in each round of iteration, the mean vector and intra-class covariance matrix of the abnormal class and the normal class are first calculated to characterize the distribution of the two types of samples in the current feature space. Then, the Fisher discriminant objective function is constructed based on the Euclidean distance of the mean vector and the weighted trace of the covariance matrix. By maximizing the inter-class distance and minimizing the intra-class divergence, it is ensured that the abnormal class and the normal class are most separated in the projected feature space. Then, the generalized eigenvalue problem of the objective function is solved to obtain the eigenvector with the largest Fisher discriminant criterion, and the original sparse features are mapped to the optimal discriminant space defined by the vector to complete a supervised dimensionality reduction. This process is iterated multiple times until the abnormal feature vector converges, and the optimal discriminant abnormal feature representation is obtained.

[0069] S106, inputting the abnormal feature vector into a preset abnormal state discrimination model, outputting abnormal state data, and performing abnormal state judgment based on the abnormal state data and a preset abnormal state judgment threshold to obtain an abnormal state detection result of the Chinese medicine to be detected.

[0070] Specifically, the construction of the abnormal state discrimination model can adopt common machine learning classification algorithms such as support vector machine (SVM), random forest (RF), neural network (NN), etc. Taking SVM as an example, firstly, a batch of representative traditional Chinese medicine samples are labeled for abnormal state according to expert experience and quality standards as a training set. Then, the abnormal feature vectors of these samples are input into SVM, mapped into high-dimensional space through kernel function transformation, and the hyperplane with the maximum interval between samples of different categories is found as the optimal classification boundary. In this process, the key parameters such as kernel function type, penalty factor, slack variable of SVM are continuously optimized through methods such as cross-validation and grid search to improve the generalization performance of the model. After the training is completed, a stable and reliable abnormal state discrimination model is obtained. When a new sample to be tested is sent in, its abnormal feature vector is input into the model, and through a series of nonlinear transformations and classification decisions, the output data representing the abnormal state of the sample can be obtained.

[0071] In order to further clarify whether the sample is abnormal and the degree of abnormality, the present invention also introduces a preset abnormal state judgment threshold. By comparing the abnormal state data output by SVM with the size of the threshold, a more accurate abnormal judgment can be made on the sample. If the abnormal state data is greater than the threshold, it means that the abnormal characteristics of the sample exceed the range of normal samples and are judged to be abnormal; otherwise, it means that the various indicators of the sample are still within the normal range and are judged to be qualified. In addition, the abnormal state can be further quantified into different levels such as slight, moderate, and severe according to the degree of deviation between the abnormal state data and the threshold, providing an intuitive basis for quality grading control.

[0072] Through the dual control of abnormal state discrimination model and threshold, the objectivity and consistency of the test results are guaranteed, and the qualitative and quantitative description of the degree of abnormality is taken into account, which is an important improvement on the traditional empirical manual identification. Since the discrimination model makes full use of the prior knowledge of known abnormal samples, and summarizes and extracts the key feature patterns that distinguish between normal and abnormal through machine learning methods, it has a strong ability to summarize rules and make inferences and predictions for abnormal discrimination of unknown samples. At the same time, the overall discrimination based on fusion features can more accurately and comprehensively judge the comprehensive quality of traditional Chinese medicine than the threshold comparison of a single indicator, and reduce misjudgments caused by partial generalization and critical fuzziness. In addition, with the help of the automatic learning of the discrimination model and the dynamic adjustment function of the threshold, the judgment rules and boundaries can be continuously optimized as the abnormal sample library continues to accumulate and improve, so that the abnormal detection capability is continuously improved to adapt to the dynamic changes of different varieties, origins and even production processes.

[0073] Based on the above embodiment, as an optional embodiment, abnormal state determination is performed based on abnormal state data and a preset abnormal state determination threshold to obtain abnormal state detection results of the Chinese medicine to be detected, including: S701, performing confidence calculation on abnormal state data to obtain confidence scores of various abnormal states; Specifically, when implementing the confidence calculation of abnormal state data, the abnormal state can be mapped to a confidence score between 0 and 1 by designing a reasonable confidence calculation function based on the output probability or abnormal score of the classification model. Taking the Softmax classifier as an example, the output vector of the model represents the probability that the sample belongs to each category, and the sum of each dimension is 1. The probability value of the dimension corresponding to the abnormal state can be directly used as its confidence score. The higher the score, the more likely the sample belongs to the abnormal category. Considering the class imbalance characteristics of the Chinese medicine anomaly detection task, the category prior distribution can also be introduced when calculating the confidence, giving the sparse class a higher weight to correct the confidence deviation caused by the uneven distribution of samples. In addition, some detection models such as support vector machines do not directly output probabilities. At this time, the abnormal score can be converted into confidence through methods such as Platt calibration. Common calibration methods include Sigmoid transformation and Isotonic regression. It is worth mentioning that abnormal state data usually has structured characteristics, such as moldy often accompanied by color abnormalities, insect bites often cause shape changes, etc. Therefore, when calculating the anomaly confidence, methods such as graph-based reasoning and multi-instance learning can also be integrated to fully explore the correlation dependencies between anomalies, realize joint confidence calculation, and comprehensively improve the interpretability and accuracy of the judgment.

[0074] By calculating the confidence of abnormal state data, the confidence scores of various abnormal states are obtained. On the basis of accurately locating the abnormality of traditional Chinese medicine, the method of the present invention further realizes the quantitative judgment of the severity of abnormality, and provides an intuitive and explainable basis for quality grading decision-making. On the one hand, as a fine-grained description of the degree of abnormality, the abnormal confidence can more accurately reflect the continuous change characteristics of the quality of traditional Chinese medicine than discrete categories. Mapping the detected multidimensional abnormalities to a unified confidence score not only helps the comparability analysis between abnormalities, but also makes the abnormal judgment results easier for non-professionals to understand, and improves the usability of the detection model. On the other hand, combining the data-driven characteristics of deep learning with confidence calculation can fully mine the discriminant information contained in the model and reduce the subjectivity of manually setting thresholds. Classification probability, abnormal score, etc. indirectly reflect the degree of grasp of model judgment, and confidence calculation externalizes this uncertainty in a more intuitive form, providing a handle for automatically adjusting the detection threshold. In addition, abnormal confidence also provides convenience for introducing prior knowledge and integrating discrimination basis. Through methods such as prior distribution correction and multi-example constraints, the confidence estimation level in complex scenarios can be further improved. With its advantages of being intuitive, flexible, and explainable, anomaly confidence will play an increasingly important role in the quality grading of Chinese medicinal materials, pushing the detection model from experience-driven to data-driven, and ultimately achieving an intelligent upgrade from "discovering anomalies" to "managing risks."

[0075] S702, compare the confidence scores of various abnormal states with the preset abnormal state determination threshold. If the confidence score exceeds the preset abnormal state determination threshold, the abnormal state detection result of the Chinese medicine to be tested is determined to be an abnormal state; if the confidence scores of all abnormal states do not exceed the preset abnormal state determination threshold, the abnormal state detection result of the Chinese medicine to be tested is determined to be a normal state.

[0076] Specifically, when implementing the comparison between the confidence score of the abnormal state and the judgment threshold, the preset abnormal state judgment threshold is used as the benchmark, and the confidence score and the threshold are compared one by one through programming statements. Taking the mold abnormality as an example, assuming that its judgment threshold is set to 0.6, when the mold confidence score of a certain Chinese medicine to be tested is 0.8, the confidence score exceeds the preset threshold, and the final abnormal state detection result of the sample can be determined to be a mold abnormality; on the contrary, if the mold confidence of the sample is only 0.4, and the confidence scores of other abnormalities such as insect bites and impurities do not exceed the corresponding judgment threshold, the final state of the sample can be determined to be normal. Considering the diversity and complexity of Chinese herbal medicine abnormalities, it is necessary to personalize the judgment thresholds of various abnormalities in practical applications. For abnormalities with greater hazards such as mold, the threshold can be set lower to increase the risk detection rate; and for abnormalities with less hazards such as slight browning, the threshold can be appropriately increased to reduce the misjudgment rate. In addition, it is also necessary to adaptively adjust the abnormal thresholds of different Chinese medicinal materials based on prior knowledge. For example, for medicinal materials that are prone to insect infestation, the threshold for judging insect infestation abnormalities can be lowered, and for medicinal materials that are prone to mildew, the threshold for judging mildew abnormalities can be increased. In short, through personalized threshold setting, quality evaluation strategies can be formulated according to students' needs, fully considering the characteristics and usage requirements of Chinese medicinal materials, and finding a balance between the degree of automation and flexibility.

[0077] By comparing the confidence scores of various abnormal states with the preset abnormal state judgment threshold, the final abnormal state detection result of the Chinese medicine to be detected is obtained. The method of the present invention forms a set of automated and personalized Chinese medicinal materials quality grading strategies, which is a step further than simple abnormality identification and realizes the leap from "discovering problems" to "guiding decisions". On the one hand, the continuous confidence score is converted into a discrete abnormality judgment, and a clear quality judgment result is directly output, which not only retains the fine-grained advantage of confidence representation, but also greatly reduces the difficulty of automatic sorting and manual review, laying a foundation for the application of medicinal materials quality grading. On the other hand, the preset threshold is used as a bridge to integrate data-driven and knowledge-driven, which can fully absorb expert experience, realize personalized customization and real-time adjustment of detection strategies, and improve the adaptability of the system to complex application environments. Compared with the unified threshold, the personalized threshold scheme can finely balance the risk-benefit of different Chinese medicinal materials and different abnormal types, which is of great significance for improving the comprehensiveness and accuracy of the system. It is worth mentioning that threshold comparison is the last checkpoint for determining abnormalities, and its result is directly related to whether the medicinal materials can be inspected and stored and put into production. Therefore, the setting of thresholds needs to be based on the verification of a large number of real scenarios, and carefully balance the detection rate and false positive rate. If necessary, auxiliary mechanisms such as multi-level thresholds and confidence backtracking should be introduced to cope with more complex and changeable application requirements.

[0078] See also Figure 2 , Figure 2The present invention provides an architecture diagram of a Chinese medicine abnormal state recognition system, which may include: The data acquisition module 1 is used to acquire the product information of the Chinese medicine to be detected, and determine the data acquisition standard of the Chinese medicine to be detected based on the product information of the Chinese medicine to be detected and the preset detection rules; Preprocessing module 2, based on the data acquisition standard of the Chinese medicine to be detected, obtains standard image data and standard spectrum data of the Chinese medicine to be detected, and preprocesses the standard image data and standard spectrum data to obtain image data and spectrum data; A first preliminary abnormal region detection module 3, used for inputting image data and spectral data into a preset image abnormality detection model to obtain image abnormal regions and spectral abnormal regions; The second preliminary abnormal region detection module 4 is used to extract features from abnormal image regions to obtain visual feature data, and to extract features from abnormal spectral regions to obtain spectral feature data; A feature fusion module 5 is used to perform feature fusion on the visual feature data and the spectral feature data to obtain an abnormal feature vector; The state determination module 6 is used to input the abnormal feature vector into a preset abnormal state determination model, output abnormal state data, and perform abnormal state determination based on the abnormal state data and a preset abnormal state determination threshold to obtain an abnormal state detection result of the Chinese medicine to be detected.

[0079] It should be noted that: when the system provided in the above embodiment realizes its functions, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0080] Please refer to Figure 3 The application also discloses an electronic device. Figure 3 The electronic device 300 may include: at least one processor 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302 or end-to-end wireless communication.

[0081] The communication bus 302 is used to realize the connection and communication between these components.

[0082] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0083] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0084] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field~Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0085] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read~Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally be at least one storage system located away from the aforementioned processor 301. Refer to Figure 3 , the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module and an application program of a method for identifying abnormal status of traditional Chinese medicine.

[0086] exist Figure 3In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call the application program storing the abnormal state identification method of traditional Chinese medicine in the memory 305, and when executed by one or more processors 301, the electronic device 300 executes one or more methods in the above-mentioned embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application. In the above embodiments, the descriptions of each embodiment have their own emphasis, and the parts that are not described in detail in a certain embodiment can refer to the relevant descriptions of other embodiments.

[0087] In the several implementations provided in this application, it should be understood that the disclosed system can be implemented in other ways. For example, the system embodiments described above are only schematic, such as the division of modules, which is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, indirect coupling or communication connection of systems or modules, which can be electrical or other forms.

[0088] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] The present application also provides a computer storage medium that can store multiple instructions, which are suitable for being loaded and executed by a processor as described above. Figure 1 The method for identifying abnormal status of traditional Chinese medicine in the embodiment shown in the figure can be found in the specific implementation process. Figure 1 The specific description of the illustrated embodiment will not be repeated here.

[0090] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0091] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0092] The above are only exemplary embodiments of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0093] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for identifying abnormal status of traditional Chinese medicine, characterized in that: The method comprises: Acquire product information of the Chinese medicine to be detected, and determine data acquisition standards for the Chinese medicine to be detected based on the product information of the Chinese medicine to be detected and preset detection rules; Based on the data acquisition standard of the Chinese medicine to be detected, standard image data and standard spectrum data of the Chinese medicine to be detected are acquired, and the standard image data and standard spectrum data are preprocessed to obtain image data and spectrum data; Inputting the image data and the spectral data into a preset image anomaly detection model to obtain an image anomaly region and a spectral anomaly region; Extracting features from the abnormal image region to obtain visual feature data, and extracting features from the abnormal spectrum region to obtain spectral feature data; Performing feature fusion on the visual feature data and the spectral feature data to obtain an abnormal feature vector; The abnormal feature vector is input into a preset abnormal state discrimination model, abnormal state data is output, and abnormal state judgment is performed based on the abnormal state data and a preset abnormal state judgment threshold to obtain the abnormal state detection result of the traditional Chinese medicine to be detected.

2. The method according to claim 1, characterized in that Before obtaining the standard image data of the Chinese medicine to be detected, the method further includes: Based on the data acquisition standard of the Chinese medicine to be detected, multi-view image data of the Chinese medicine to be detected is acquired, and the multi-view image data is preprocessed to obtain a standardized image to be detected; The foreground of the standardized image to be inspected is extracted using a preset image segmentation model to obtain the standard image data.

3. The method according to claim 1, characterized in that Before obtaining the standard spectrum data of the Chinese medicine to be detected, the method further includes: Based on the data acquisition standard of the Chinese medicine to be detected, obtain the visible light-near infrared full-band spectrum data of the Chinese medicine to be detected, and pre-process the visible light-near infrared full-band spectrum data to obtain a standardized spectrum curve; Based on the priori component information of the Chinese medicine to be detected, the standardized spectral curve is screened for characteristic wavelengths to obtain characteristic bands, and the spectral data of the characteristic bands is used as the standard spectral data.

4. The method according to claim 1, characterized in that The step of fusing the visual feature data and the spectral feature data to obtain an abnormal feature vector includes: Performing feature alignment processing on the visual feature data and the spectral feature data to obtain aligned visual feature data and aligned spectral feature data, and performing fusion processing on the aligned visual feature data and the aligned spectral feature data using a multi-view representation learning method to obtain multi-view integrated features; The fused multi-view integrated features are subjected to dimensionality reduction processing to obtain a low-dimensional abnormal feature vector, and a feature selection method based on L1 regularization is adopted to automatically search for the low-dimensional abnormal feature vector by introducing a sparse penalty term to obtain a sparse abnormal feature vector representation; The supervised dimensionality reduction method based on the Fisher criterion is used to optimize the inter-class divergence and intra-class divergence represented by the abnormal feature vector to obtain the abnormal feature vector.

5. The method according to claim 4, characterized in that The method of using a multi-view representation learning method to fuse the aligned visual feature data and the aligned spectral feature data to obtain a multi-view integrated feature includes: constructing a visual feature view based on the aligned visual feature data, and constructing a spectral feature view based on the aligned spectral feature data; Constructing a visual feature view and a spectral feature view, and applying a manifold learning method in the visual feature view and the spectral feature view to maintain a local neighborhood structure of features, so as to obtain a first visual feature view and a first spectral feature view; Learning the collaborative representation of the first visual feature view and the first spectral feature view through a bilinear mapping function to obtain a second visual feature view and a second spectral feature view; A multi-view consistency constraint is introduced to optimize the consistency of the second visual feature view and the second spectral feature view, so as to obtain the multi-view integrated feature.

6. The method according to claim 5, characterized in that The method of learning the collaborative representation of the first visual feature view and the first spectral feature view by a bilinear mapping function to obtain a second visual feature view and a second spectral feature view includes: Initializing the parameters of the bilinear mapping function, and constructing a visual mapping matrix and a spectral mapping matrix based on the parameters of the bilinear mapping function; Mapping the visual mapping matrix and the spectral mapping matrix from the original feature space to a common subspace, and solving an optimization problem of collaborative representation learning in the common subspace to obtain an optimal inter-view mapping matrix; The first visual feature view and the first spectral feature view are mapped to the common subspace by using the optimal inter-view mapping matrix to obtain the second visual feature view and the second spectral feature view.

7. The method according to claim 1, characterized in that The abnormal state determination is performed based on the abnormal state data and a preset abnormal state determination threshold to obtain the abnormal state detection result of the Chinese medicine to be detected, including: Calculating the confidence of the abnormal state data to obtain confidence scores of various abnormal states; The confidence scores of the various abnormal states are compared with the preset abnormal state determination threshold. If the confidence score exceeds the preset abnormal state determination threshold, the abnormal state detection result of the Chinese medicine to be detected is determined to be an abnormal state; if the confidence scores of all abnormal states do not exceed the preset abnormal state determination threshold, the abnormal state detection result of the Chinese medicine to be detected is determined to be a normal state.

8. A system for identifying abnormal status of traditional Chinese medicine, characterized in that: The system comprises: A data acquisition module, used to acquire product information of the Chinese medicine to be detected, and determine the data acquisition standard of the Chinese medicine to be detected based on the product information of the Chinese medicine to be detected and the preset detection rules; A preprocessing module, based on the data acquisition standard of the Chinese medicine to be detected, acquires standard image data and standard spectrum data of the Chinese medicine to be detected, and preprocesses the standard image data and standard spectrum data to obtain image data and spectrum data; A first preliminary abnormal region detection module, used for inputting the image data and the spectral data into a preset image abnormality detection model to obtain an image abnormal region and a spectral abnormal region; A second preliminary abnormal region detection module is used to extract features from the abnormal image region to obtain visual feature data, and to extract features from the abnormal spectral region to obtain spectral feature data; A feature fusion module, used for performing feature fusion on the visual feature data and the spectral feature data to obtain an abnormal feature vector; The state determination module is used to input the abnormal feature vector into a preset abnormal state determination model, output abnormal state data, and perform abnormal state determination based on the abnormal state data and a preset abnormal state determination threshold to obtain the abnormal state detection result of the Chinese medicine to be detected.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes a processor, a memory and a transceiver, the memory is used to store instructions, the transceiver is used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 7.

Images