Chinese herbal medicine identification method based on time-related feature mining in smart medical scenarios
Through time-related feature mining technology, using Retinex image enhancement and pyramid network model and other means, the generalized features and time-invariant features of Chinese herbal medicine are extracted, which solves the problems of low recognition efficiency and difficulty in Chinese herbal medicine recognition, and improves the recognition accuracy and quality judgment.
Patent Information
- Application Number
- CN202210637330.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-07
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-06-07
AI Technical Summary
There are problems such as a wide variety of target types, similar shapes and colors, and adulteration in Chinese herbal medicine identification, which makes it difficult to ensure low recognition efficiency and accuracy, especially when there is a lack of experience and the quality of medicinal materials.
Using the method of time-related feature mining, the generalized features and time-invariant features of Chinese herbal medicine are extracted through the Retinex image enhancement algorithm and pyramid network model, activation area generation network, feature map structure block and other technologies, and the generalized features of Chinese herbal medicines are constructed to improve recognition. Accuracy and quality judgment.
It significantly improves the recognition accuracy of similar Chinese herbal medicines and the accuracy of Chinese herbal medicine quality judgment, and solves the problems of spatial information positioning deviation and poor model generalization ability in traditional methods.
Smart Images

Figure CN115100486B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of artificial intelligence target recognition, and specifically relates to a method for identifying Chinese herbal medicines by mining time-related features in a smart medical scenario. Background Art
[0002] Traditional Chinese medicine is a complete medical theory system that incorporates a large amount of clinical experience. It has been listed as the most influential medical outline by the World Health Organization. Chinese herbal medicine has important value in the field of traditional Chinese medicine. It not only has excellent performance in treating diseases, but also can regulate the body under the guidance of the theoretical system of traditional Chinese medicine. However, due to the wide variety of Chinese herbal medicines, it is difficult for inexperienced personnel to accurately identify Chinese herbal medicines and the uneven quality of herbal medicines. With the continuous development of artificial intelligence, it has already emerged in the field of image recognition, and has also promoted the development of traditional Chinese medicine image processing technology in the direction of quantitative image description, three-dimensional image, visualization and networking.
[0003] Existing identification of Chinese herbal medicines is often done through manual analysis and research, which requires researchers to have a wealth of knowledge and experience in traditional Chinese medicine. Traditional methods are time-consuming and labor-intensive, with low recognition efficiency and difficulty in ensuring recognition rates. In order to improve the scientific, modern and intelligent level of Chinese herbal medicine identification, new means are urgently needed to improve and upgrade traditional Chinese herbal medicines. In addition, some experts and scholars have extracted features from Chinese herbal medicine images through traditional machine vision technology to identify Chinese herbal medicines, but the effect of this method is also difficult to meet actual needs. Most of them are limited to laboratory-level research, and the model has poor generalization ability and robustness. Deep learning is one of the important advances in artificial intelligence. It subverts the design ideas of traditional algorithms, can automatically learn features from data, and avoids the complexity and limitations of manually designing and extracting features in traditional algorithms. Without human intervention, it has great feature learning potential and is expected to further improve and solve a series of problems existing in the current Chinese herbal medicine identification methods. Summary of the invention
[0004] Technical problems solved: The present invention aims to solve the problems in the Chinese medicine pharmacy scenario of smart medical care, such as the large variety of Chinese herbal medicine targets to be identified, many of which are very similar in shape and color, as well as the phenomenon of Chinese herbal medicine adulteration, doping, passing off fake products as genuine ones, and selling inferior products as good ones, which makes it difficult for inexperienced personnel to accurately identify Chinese herbal medicines and the uneven quality of Chinese herbal medicines. A Chinese herbal medicine identification method based on time-related feature mining in the scenario of smart medical care is proposed. By training on the Chinese herbal medicine dataset, the feature representations of different types of generalized Chinese herbal medicines are extracted, thereby improving the recognition accuracy of similar Chinese herbal medicines of the same type and the accuracy of judging the uneven quality of Chinese herbal medicines.
[0005] Technical solution:
[0006] A method for identifying Chinese herbal medicines by mining time-related features in a smart medical scenario, the method comprising the following steps:
[0007] S1, collect Chinese herbal medicine pictures, mark the picture category labels and growth cycle labels on the Chinese herbal medicine pictures, generate Chinese herbal medicine image samples, and construct a Chinese herbal medicine image sample dataset; use the Retinex image enhancement algorithm to preprocess the acquired image dataset, and divide each sample image into a training set for training the Chinese herbal medicine recognition model and a test set for testing the Chinese herbal medicine recognition model according to a preset ratio;
[0008] S2, constructing an M-layer pyramid network model, using the pyramid network model to process the imported Chinese herbal medicine pictures to output high-resolution feature maps with spatial sensitivity, reducing the spatial information positioning deviation when the feature maps on the spatial stream and the semantic stream are fused; the pyramid network model consists of two paths: a bottom-up spatial stream path and a top-down semantic stream path; the Chinese herbal medicine pictures are processed by the spatial stream path to obtain the corresponding feature maps, and the obtained feature maps are imported into the semantic stream path, and the target feature area blocks are generated through pixel shuffling and feature perception reorganization;
[0009] S3, construct an activation region generation network, which includes a competitive channel attention module, a spatial attention module and a feature processing structure block; the competitive channel attention module is used to make the semantic stream and the spatial stream compete adaptively, and obtain the activation weight matrix of the channel dependency relationship between the spatial stream and the semantic stream; the spatial attention module is used to obtain the activation weight matrix between local features and global features between different levels, and fuse the channel dependency activation weight matrix learned in the competitive channel attention module to obtain the activation feature map output by the activation region generation network; the feature processing structure block includes a self-attention module and a feature decomposition module; the self-attention module is used to capture the context information of the time series and obtain the correlation of the features in the sequence; the feature decomposition module is used to obtain the time-invariant features in the time series features of Chinese herbal medicine;
[0010] S4, constructing a feature mapping structure block, establishing a mapping relationship between the features of the Chinese herbal medicines in the current growth cycle and the features of the Chinese herbal medicines in the remaining growth cycles;
[0011] S5, constructing a Chinese herbal medicine recognition network model, using the training set to train the Chinese herbal medicine recognition network model to obtain a trained Chinese herbal medicine recognition network model; inputting the test set into the trained Chinese herbal medicine recognition network model to obtain the Chinese herbal medicine recognition result.
[0012] Furthermore, in step S1, the acquired image data set is preprocessed using the Retinex image enhancement algorithm according to the following formula:
[0013]
[0014]
[0015] Where r(x,y) represents the output image; R(x,y) represents the reflected image; S(x,y) represents the input image, L(x,y) represents the brightness image; F(x,y) represents the Gaussian center surround function; λ is the normalization coefficient; q is the Gaussian surround scale; x represents the horizontal coordinate of the image pixel; y represents the vertical coordinate of the image pixel.
[0016] Furthermore, in step S2, the process of using an upsampling module to process the imported Chinese herbal medicine image to output a high-resolution feature map with spatial sensitivity includes the following steps:
[0017] S21, set up an M-layer pyramid network model. The pyramid network model consists of two paths: a bottom-up spatial flow path and a top-down semantic flow path. On the bottom-up spatial flow path, the features of the i-th layer are output as feature maps through the upsampling module. i∈{1,2,3,…,M-1}, where H i , W i , C b Represent the height, width, and number of channels of the feature map of the i-th layer respectively; in the bottom-up spatial flow path, the output feature of the last convolutional layer of the i-th layer pyramid is
[0018] S22, the feature D output by the last convolutional layer of the Mth pyramid in the bottom-up spatial flow path M The top-level features after channel adjustment as the top-down semantic flow path Get feature map Any pixel position Form a shape centered at z and of size k up ×k up Predefined feature area blocks Through the convolution of size 1×1 To adjust the channel, the formula is as follows:
[0019]
[0020] In the formula, Represents the features after channel adjustment by 1×1 convolution kernel;
[0021] S23, through the pixel shuffling operator The channel dimension is expanded in the spatial dimension to obtain the feature map Any pixel position The regional upsampling kernel is as follows:
[0022]
[0023] In the formula, Representing coordinates The output feature pixels of α represents the scaling factor, αk up Indicates the size of the regional convolution block;
[0024] S24, by the feature map Perform pixel shuffling operations on all pixel positions of each channel to obtain the predicted upsampling kernel Get feature map Any pixel position Corresponding to the predicted upsampling kernel F M Feature pixel blocks on The feature pixel block F is reorganized by channel reorganization operation. z The channel dimension is converted into the spatial dimension to generate the regional upsampling kernel. The formula is as follows:
[0025] F′ z = reshape(F z )
[0026] Where reshape(·) represents the channel reorganization operation; F z ′ represents the size of αk up ×αk up The regional upsampling kernel of
[0027] S25, predefined feature area blocks With the region upsampling kernel F z 'The target feature region block is generated through the feature-aware reorganization module, and the formula is as follows:
[0028]
[0029] In the formula, U′ M-1 Represents any pixel position on the input feature map The mapped pixel position z′=(i′,j′) on the target feature region block on the output feature map, αk up Represents the region upsampling kernel size.
[0030] Furthermore, in step S3, the competitive channel attention module is used to make the semantic stream and the spatial stream compete adaptively, and the process of obtaining the activation weight matrix of the channel dependency relationship between the spatial stream and the semantic stream includes the following steps:
[0031] A31, the feature D output by the last convolutional layer of the i-th pyramid in the bottom-up spatial flow pathi After lateral connection through 1×1 convolution operation, the output is The formula is as follows:
[0032]
[0033] In the formula, represents the features after channel adjustment by 1×1 convolution kernel; i represents the i-th layer; · represents the convolution operation;
[0034] A32, let the input of the i-th layer feature fusion module of the pyramid network be The input from the spatial stream and the semantic stream is globally averaged and pooled to squeeze the two-dimensional features (H×W) of each channel of each set of feature maps to obtain the average feature. The formula is as follows:
[0035]
[0036]
[0037] In the formula, Descriptor representing the cth channel of the input features on the spatial stream; Represents the descriptor of the kth channel in the input feature on the semantic stream; F sq (·) represents the global average pooling operation; and Represent the features of the kth channel in the i-th layer of input features on the spatial stream and the semantic stream, respectively. Represents the pixel point on the feature map;
[0038] A33, will and After channel dimension connection, it is used as joint excitation input. The semantic stream and the spatial stream are made to compete adaptively, and the activation weight matrix of the channel dependency between the spatial stream and the semantic stream is obtained. The formula is as follows:
[0039]
[0040] In the formula, F ex (·) is the excitation operation, representing the forward propagation of two fully connected layers; W ex is the weight matrix of the two fully connected layers,
[0041] Furthermore, in step S3, the spatial attention module is used to obtain the activation weight matrix between local features and global features between different levels, and the channel dependency activation weight matrix learned in the competitive channel attention module is integrated to obtain the activation feature map output by the activation region generation network. The process includes the following steps:
[0042] B31, the input L of the feature fusion module of the i-th layer of the pyramid network i , U i After the global average pooling operation across channels, the spatial descriptor of the position (x, y) is obtained, and the formula is as follows:
[0043]
[0044]
[0045] In the formula, G sq (·) represents the global average pooling operation across channels; [l i ] x,y 、[u i ] x,y The positions of all channels in the input features of the bottom-up spatial stream path and the top-down semantic stream path in the two streams of the i-th layer are (x, y);
[0046]
[0047] Will and Connect the channel dimensions to obtain the spatial relationship information of the fusion features of the i-th layer. The formula is as follows:
[0048]
[0049] In the formula, is the weight matrix for adjusting the spatial information relationship between the spatial stream and the semantic stream; G ex (·) represents the forward propagation operation of two convolutional layers; w V is the weight matrix of the two convolutional layers; V i 1 、V i 2 is the spatial information relationship weight matrix of the bottom-up path and the top-down path;
[0050] B32, the weight matrix P that coordinates the channel information relationship between the spatial stream and the semantic stream i The weight matrix V of the spatial information relationship between the spatial stream and the semantic stream i Multiply them, and the formula is as follows:
[0051]
[0052]
[0053] In the formula, stands for element-wise multiplication; Represents the activation value of the activation region generation network;
[0054] B33, Generate network activation values by activating regions The activation feature map output by the activation region generation network is obtained, and the formula is as follows:
[0055]
[0056] In the formula, Represents the activation area to generate the network output feature map; Stands for element-wise multiplication.
[0057] Furthermore, in step S3, the process of using the self-attention module to capture the context information of the time series and obtaining the correlation degree of the features in the sequence includes the following steps:
[0058] C31, stacking the features output by the last convolutional layer of the i-th pyramid in the bottom-up spatial flow path into a temporal feature sequence Set three key learning parameters The formula is as follows:
[0059]
[0060]
[0061]
[0062] In the formula, Represents the time series X i Feature learning parameters of the p-th cycle in; represents the weight learning matrix;
[0063] C32, the feature learning parameters are stacked into three parameter sequences, respectively expressed as
[0064] Get the feature sequence X i Any growth cycle characteristics represented by The three key learning parameter sequences are Get the weight parameter of each head, the formula is as follows:
[0065]
[0066] Where σ(·) represents the softmax function; represents the parameter matrix; d q express The dimension, d k express The dimension, d v express The dimension of d k =d v =d q / h, h represents the number of heads;
[0067] C33, fuses each head output parameter matrix through the learnable parameter matrix W, the formula is as follows:
[0068]
[0069] In the formula, Represents the feature weight sequence of Chinese herbal medicine image;
[0070] Furthermore, in step S3, the process of using the feature decomposition module to obtain the time-invariant features in the time series features of Chinese herbal medicine includes the following steps:
[0071] D31, the time characteristic series of Chinese herbal medicine X i Output the time feature weight sequence of Chinese herbal medicine through the self-attention function F The time feature weight sequence of Chinese herbal medicine is modeled as a linear combination of time-invariant feature components, time-varying feature components and draw features. in represents time-invariant features, Represents time-varying characteristics, Represents the average value of Chinese herbal medicine features; obtains the time-invariant and time-varying feature components in the Chinese herbal medicine sequence through a lightweight network composed of two fully connected layers, and estimates the parameters of the lightweight network through the expected maximum algorithm The formula is as follows:
[0072]
[0073] In the formula, θ (i) represents the given lightweight network parameters; θ represents the estimated lightweight network parameters; P(·) represents the posterior distribution; V represents the characteristics of the Chinese herbal medicine training samples; represents the characteristics of the jth growth cycle of the i-th Chinese herbal medicine; x ia,l With x ib,j denote the corresponding time-invariant factors and time-varying factors respectively;
[0074] D32, based on a given posterior distribution, performs maximization in an iterative manner The function updates the lightweight network parameters θ, and the formula is as follows:
[0075]
[0076] In the formula, θ represents the lightweight network parameters; θ (i+1) Represents updated lightweight network parameters;
[0077] D33, calculate the updated lightweight network parameters θ (i+1) , the formula is as follows:
[0078]
[0079]
[0080]
[0081]
[0082]
[0083]
[0084] In the formula, μ1(·) represents P(x ia,l |θ (i) ,V) and P(x ib,j |θ (i) ,V); μ2(·) represents the first-order moment of P(x ia,l |θ (i) ,V) and P(x ib,j |θ (i) ,V)’s second-order moment;
[0085] D34, time-invariant feature components are obtained through a lightweight network consisting of two fully connected layers and time-varying characteristic components Time-invariant characteristic sequence of herbal medicine Time-varying characteristic sequences of Chinese herbal medicines Perform a linear normal mapping:
[0086]
[0087]
[0088] In the formula, represents trainable parameters;
[0089] D35, by time-invariant feature sequence With time-varying characteristic sequence The correlation between them is expressed as:
[0090]
[0091] In the formula, μ ia , μ ib Represents Y ia With Y ib The mean of Represents Y ia With Y ib The variance of
[0092] D36, through the decorrelation loss function to reduce the correlation between the two feature components, reduce the impact of the time-varying feature component on the time-invariant feature component:
[0093]
[0094] In the formula, F represents the self-attention function; K represents and The correlation between them.
[0095] Furthermore, in step S4, the process of constructing a feature mapping structure block and establishing a mapping relationship between the features of the Chinese herbal medicines in the current growth cycle and the features of the Chinese herbal medicines in the remaining growth cycles includes the following steps:
[0096] S41, the time characteristic series of Chinese herbal medicine X i Output the time feature weight sequence of Chinese herbal medicine through the self-attention function F The time feature weight sequence set of multiple types of Chinese herbal medicine is j∈{1,2,3,…,m+, where the time characteristic sequence of the jth type of Chinese herbal medicine The time feature weight sequence sets of multiple types of Chinese herbal medicines are used to generate corresponding feature mapping areas. The formula is as follows:
[0097]
[0098] In the formula, represents the feature mapping area formed by the characteristics of the jth Chinese herbal medicine in the pth growth period; N p,j represents the total number of images of the jth Chinese herbal medicine in the pth growth period; represents the characteristics of the jth Chinese herbal medicine in the pth growth period;
[0099] S42, Definition Generate corresponding feature mapping regions for multiple types of Chinese herbal medicine time feature weight sequence sets; is the time characteristic series of Chinese herbal medicine Mapping relationship in feature mapping area; generate activation feature map of network output from activation area By mapping the feature objective function to the feature mapping area, its feature representation is obtained. The formula is as follows:
[0100]
[0101]
[0102]
[0103] In the formula, E represents the smoothing operator; λ1 and λ2 represent weight factors;
[0104] S43, calculate the time characteristic sequence of Chinese herbal medicine The feature relationship matrix with the remaining m-1 Chinese herbal medicines in any growth period is obtained, and the feature mapping matrix of any activated feature map is obtained. The formula is as follows:
[0105]
[0106]
[0107] In the formula, Represents the time characteristic series of Chinese herbal medicine The characteristic relationship matrix with the other m-1 Chinese herbal medicines in any growth period; Represents the feature map matrix that maps activation feature maps to feature map regions.
[0108] Beneficial effects:
[0109] 1. Compared with the traditional Chinese herbal medicine recognition algorithm, the proposed Chinese herbal medicine recognition method based on time-related feature mining in the smart medical scenario establishes a feature mapping area, maps the feature map of the Chinese herbal medicine to be identified with the characteristics of similar Chinese herbal medicines in different growth cycles through feature mapping, and mines the time mapping relationship between the same Chinese herbal medicines in different growth cycles, which significantly improves the accuracy of identifying similar Chinese herbal medicines of the same type.
[0110] 2. Compared with the previous upsampling algorithm, the improved upsampling algorithm recombines features in a predefined area centered on an arbitrary pixel position, effectively solving the problem of low spatial sensitivity caused by upsampling of coarse features in the top-level spatial information, which leads to spatial information positioning deviation during feature fusion.
[0111] 3. The Chinese herbal medicine identification method based on time-related feature mining in the smart medical scenario extracts time-invariant features from the time series of the same Chinese herbal medicine, and fuses the time-invariant features, global features and local features through the feature fusion structure block, so that the fused features can effectively represent the characteristic information of the same Chinese herbal medicine, thereby improving the accuracy of judging the quality differences of Chinese herbal medicines. BRIEF DESCRIPTION OF THE DRAWINGS
[0112] Figure 1 is a schematic structural diagram of an improved upsampling module according to an embodiment of the present invention;
[0113] Figure 2 is an activation area generation network structure diagram of an embodiment of the present invention;
[0114] Figure 3 is a feature processing structure block diagram of an embodiment of the present invention;
[0115] Figure 4 This is a flow chart of a method for identifying Chinese herbal medicines by mining time-related features in a smart medical scenario according to an embodiment of the present invention. DETAILED DESCRIPTION
[0116] The following examples will enable those skilled in the art to more fully understand the present invention, but are not intended to limit the present invention in any way.
[0117] Figure 4 This is a flow chart of a method for identifying Chinese herbal medicines by mining time-related features in a smart medical scenario according to an embodiment of the present invention. Figure 4 The Chinese herbal medicine identification method comprises the following steps:
[0118] S1, collect Chinese herbal medicine pictures, mark the picture category labels and growth cycle labels on the Chinese herbal medicine pictures, generate Chinese herbal medicine image samples, and construct a Chinese herbal medicine image sample dataset; use the Retinex image enhancement algorithm to preprocess the acquired image dataset, and divide each sample image into a training set for training the Chinese herbal medicine recognition model and a test set for testing the Chinese herbal medicine recognition model according to a preset ratio.
[0119] S2, construct an M-layer pyramid network model, and use the pyramid network model to process the imported Chinese herbal medicine pictures to output high-resolution feature maps with spatial sensitivity, so as to reduce the spatial information positioning deviation when the feature maps on the spatial stream and the semantic stream are fused; the pyramid network model consists of two paths: a bottom-up spatial stream path and a top-down semantic stream path; the Chinese herbal medicine pictures are processed by the spatial stream path to obtain the corresponding feature maps, and the obtained feature maps are imported into the semantic stream path, and the target feature area blocks are generated through pixel shuffling and feature perception reorganization.
[0120] S3, construct an activation region generation network, which includes a competitive channel attention module, a spatial attention module and a feature processing structure block; the competitive channel attention module is used to make the semantic stream and the spatial stream compete adaptively, and obtain the activation weight matrix of the channel dependency relationship between the spatial stream and the semantic stream; the spatial attention module is used to obtain the activation weight matrix between local features and global features between different levels, and fuse the channel dependency activation weight matrix learned in the competitive channel attention module to obtain the activation feature map output by the activation region generation network; the feature processing structure block includes a self-attention module and a feature decomposition module; the self-attention module is used to capture the time series context information and obtain the correlation of features in the sequence; the feature decomposition module is used to obtain the time-invariant features in the time series features of Chinese herbal medicine.
[0121] S4, constructing a feature mapping structure block, and establishing a mapping relationship between the characteristics of the Chinese herbal medicines in the current growth cycle and the characteristics of the Chinese herbal medicines in the remaining growth cycles.
[0122] S5, constructing a Chinese herbal medicine recognition network model, using the training set to train the Chinese herbal medicine recognition network model to obtain a trained Chinese herbal medicine recognition network model; inputting the test set into the trained Chinese herbal medicine recognition network model to obtain the Chinese herbal medicine recognition result.
[0123] The specific steps include:
[0124] Step 1: Image preprocessing based on Retinex: Use the Gaussian surround function to convolve the pre-acquired image and output the reflection component to achieve image enhancement.
[0125] In actual Chinese medicine pharmacies, the image quality may be reduced due to the overlap of Chinese herbs or the reflection of the surface of Chinese herbs, making the texture features of the image blurred. In addition, Chinese herbal medicine recognition requires rich and clear features. These may extract wrong information during the feature extraction process. Image enhancement technology can reduce the degree of influence of adverse factors, extract specific information, and suppress interference information, thereby improving the quality of the image.
[0126] There are many types of image enhancement, mainly divided by the domain, which can be divided into frequency domain and spatial domain. Frequency domain enhancement is a method of processing after Fourier transform to frequency domain, and then transforming back to spatial domain to obtain enhanced image, such as commonly used low-pass filtering and high-pass filtering. Spatial domain directly processes the image in units of pixels.
[0127] This embodiment specifically uses the Retinex theory to perform image enhancement. The Retinex image enhancement algorithm is somewhat different from general image enhancement algorithms. It can achieve balance in color constancy, dynamic range compression, and edge enhancement, has obvious adaptive characteristics, and meets the processing requirements of various types of images.
[0128] The single-scale Retinex theory uses S(x,y) to represent the input image, which can be decomposed into two different images: the reflection image R(x,y) and the brightness image L(x,y). The incident light shines on the observed object, and the reflected light is reflected by the surface of the observed object to enter the human eye or the collection device. It can be expressed as:
[0129]
[0130] The core idea of the single-scale Retinex algorithm is to remove the influence of ambient light L(x,y) on the overall image, and try to restore the object's reflected light component R(x,y) containing important information to obtain the appearance of the object itself. However, the image we get is only the result image S(x,y), so whether L(x,y) can be estimated well is the key to the effect of the Retinex algorithm. Since L(x,y) needs to be separated, for the convenience of calculation, the best way is to transform the mathematical calculation formula into the logarithmic domain and change the product relationship into an addition and subtraction relationship:
[0131] log(S(x,y))=log(R(x,y))+log(L(x,y));
[0132] For the convenience of representation, let s(x,y)=log(S(x,y)), r(x,y)=log(R(x,y)), l(x,y)=log(L(x,y)), then:
[0133]
[0134] r(x,y)=log S(x,y)-log[F(x,y)*S(x,y)];
[0135] In the above formula, r(x,y) represents the output image, and F(x,y) represents the Gaussian center surround function, which is as follows:
[0136]
[0137] Among them, λ is the normalization coefficient, q is the Gaussian surround scale, x represents the horizontal coordinate of the image pixel, and y represents the vertical coordinate of the image pixel. The convolution operation in the single-scale Retinex algorithm is a processing step for the estimated illumination component in the spatial domain. Its physical meaning is to estimate the illumination change in the image by calculating the weighted average of the pixels of the input image and the surrounding area, and finally remove it through logarithmic domain processing to obtain the enhanced image.
[0138] Step 2: Construct an improved upsampling module; the input feature map passes through the pixel shuffling and feature reorganization modules to obtain an output feature map with high-resolution representation to make up for the lack of spatial resolution.
[0139] Since the high-resolution representation obtained by upsampling the low-resolution representation has good semantic expression ability, upsampling itself cannot completely make up for the loss of spatial resolution, and the traditional upsampling method only obtains the upsampling kernel through the spatial position of the pixel point, which does not fully utilize the semantic information of the feature map. The ideal upsampling operator first needs to have a large receptive field to better utilize the information around the pixel; secondly, the semantic information of the upsampling kernel and the feature map is associated; finally, the upsampling operator has the characteristic of being lightweight.
[0140] refer to Figure 1 , defines the feature D output by the last convolutional layer of the Mth pyramid in the bottom-up path M Top-level features as a top-down path Get feature map Any pixel position z=(i,j) forms a pixel with z as the center and size k up ×k up Predefined feature area blocks Upsampling requires prediction of the upsampling kernel, the feature area block at each pixel position The required size is k up ×k up The upsampling kernel of Required size The upsampling kernel is α, and α is the upsampling multiple. The feature map is transformed through a 1×1 convolution layer. To make channel adjustments:
[0141]
[0142] By adjusting the feature map of the channel Any pixel position Perform pixel shuffling operation to obtain The regional upsampling kernel is as follows:
[0143]
[0144] Feature Map Repeat the above operation, by Perform pixel shuffling operations on all pixel positions of each channel to obtain the predicted upsampling kernel Get feature map Any pixel position Corresponding to the predicted upsampling kernel F M Feature pixel blocks on The feature pixel block F is reorganized by channel reorganization operation. z The channel dimension is converted to the spatial dimension to generate the regional upsampling kernel. The formula is as follows:
[0145] F′ z = reshape(F z );
[0146] Predefined feature area blocks With the regional upsampling kernel F′ z The target feature region block is generated through the feature-aware reorganization module. The formula is as follows:
[0147]
[0148] By inputting feature maps Repeat the operation for all pixel positions to obtain the output feature map U M , the size is αH i ×αW i ×C b .
[0149] Step 3: Construct an activation region generation network; the included competitive channel attention module allows the spatial stream and the semantic stream to compete adaptively, selectively emphasizing effective features and eliminating redundant features. The included spatial attention module dynamically weighs global information and local information.
[0150] Since the traditional feature pyramid feature fusion method processes the features in the spatial stream and semantic stream indiscriminately at all scales, a large number of features are repeated when fusion is performed, resulting in redundant feature fusion and low efficiency of fusion. Therefore, in order to improve network efficiency, a competitive channel attention module and a spatial attention module are added based on the feature pyramid to achieve global feature enhancement so that the network can actively learn to activate feature maps. The weight of .
[0151] refer to Figure 2 , the global feature is defined as the feature map tensor D of the entire input image i , size (H i ,W i,C),The condition for feature map fusion is to ensure that the size and number of channels of the feature map are consistent to achieve feature fusion. The 1×1 convolution layer is used to change D i The number of channels is as follows:
[0152]
[0153] After passing through the 1×1 convolution layer, the number of channels changes to C b , the output feature map L of the convolutional layer i The size is (H i ,W i ,C b ). The pyramid network has M layers, and the feature map D of the Mth layer is M Output feature map U through 1×1 convolution layer M , through U M The upsampling generates a top-down path, where the i-th layer in the top-down path is denoted as U i .
[0154] The feature L of the i-th layer in the bottom-up path i and the feature U of the i-th layer in the top-down path i As input to the activation region generation network, the activation feature map is learned The weight of .
[0155] The details of building a new neural network to train the activation vector through the channel attention module are as follows:
[0156] In order to eliminate the characteristic map at high H g and width W g The influence on the average feature is obtained by first using a global pooling operation F sq ,
[0157] The formula is as follows:
[0158]
[0159]
[0160] Under this global operation, we i with U i The size of each channel is H i ×W i Feature map and Perform a global average pooling operation on the pooled output of each feature map and Connect to form 1×1×C b One-dimensional vector of and The three-dimensional features are reduced to a one-dimensional vector to eliminate the interference of spatial information. Next, the global average pooling operation outputs As the joint excitation input of the two-layer fully connected neural network, the activation weight matrix of the channel dependency between the spatial stream and the semantic stream is obtained, and the formula is as follows:
[0161]
[0162] Where w1 and w2 are the learning parameters of the two fully connected layers. The activation function of the first fully connected layer is RELU and the activation function of the second fully connected layer is Sigmoid, so the activation vector P i The value P in i Mapped to (0,1), and then input activation feature map And the output activation matrix P i Recalibration is done by weighting at the channel level, as follows:
[0163]
[0164] The final competitive channel attention module outputs the activation feature map Its size is H i ×W i ×C b .
[0165] Another module of the global feature activation network is the spatial channel attention module; the spatial attention module adaptively quantifies the importance of each image region, allowing the network to focus more on important locations in the image. The specific training process is as follows:
[0166] First, a cross-channel global pooling operation G is used sq To eliminate the influence of each channel information, the formula is as follows:
[0167]
[0168]
[0169] Obtain the spatial descriptors of the spatial stream and semantic stream at position (x, y). Considering all positions on the feature map, we can get Size (H i ,W i ,1); and To connect the channel dimension, a lightweight network consisting of two convolutional layers is used to and Modeling in space:
[0170]
[0171] Vi is the activation weight matrix obtained after spatial modeling of the two information flows, with a scale of H i ×W i ×2. The weight matrix P that coordinates the channel information relationship between the spatial stream and the semantic stream i The weight matrix V of the spatial information relationship between the spatial stream and the semantic stream i Multiply them, and the formula is as follows:
[0172]
[0173]
[0174] Finally, the activation value of the activation region generation network is obtained
[0175] The activation area generates the activation value of the network Multiply the features on the spatial stream and semantic stream respectively, the formula is as follows:
[0176]
[0177] Get the activation feature map output by the activation region generation network Size H i ×W i ×C b .
[0178] Step 4: Construct a feature processing structure block; the included self-attention module is used to capture relevant information in the target sequence and obtain the activation weight in the target sequence; the included feature decomposition module is used to obtain the time-invariant features of Chinese herbal medicine.
[0179] Due to the influence of the growth of Chinese herbal medicine plants, the shape and related features of the same Chinese herbal medicine will change at different growth stages. However, during the growth process of Chinese herbal medicine, some features basically do not change with the change of time, which is called time-invariant features. The problem of uneven quality of Chinese herbal medicine can be judged by judging whether the features of the Chinese herbal medicine to be identified contain time-invariant features. Therefore, in order to complete the above method, the self-attention module and the feature decomposition module are added to obtain the time-invariant features of the same Chinese herbal medicine.
[0180] refer to Figure 3 , the time series image of Chinese herbal medicine is input into the backbone network Alexnet to obtain the time feature sequence of Chinese herbal medicine Based on the temporal feature sequence of Chinese herbal medicine, it is necessary to learn the interaction between each Chinese herbal medicine image and capture the contextual information of the sequence. To obtain three key learning parameters The formula is as follows:
[0181]
[0182]
[0183]
[0184] By stacking the feature learning parameters into three parameter sequences, they are represented as So through the parameter matrix Get the weight parameter of each head, the formula is as follows:
[0185]
[0186] The output parameter matrix of each head is fused through the learnable parameter matrix W, the formula is as follows:
[0187]
[0188] Time characteristic series of Chinese herbal medicine X i Output the time feature weight sequence of Chinese herbal medicine through the self-attention function F The time feature weight sequence of Chinese herbal medicine is modeled as a linear combination of time-invariant feature components, time-varying feature components and average features, that is, The time-invariant and time-varying feature components in the Chinese herbal medicine sequence are obtained through a lightweight network composed of two fully connected layers, and the parameters of the lightweight network The parameters can be estimated by the expected maximum algorithm, the formula is as follows:
[0189]
[0190] According to the given posterior distribution, the maximization is performed in an iterative manner The function updates the lightweight network parameters θ, and the formula is as follows:
[0191]
[0192] In the formula, θ represents the lightweight network parameters; θ (i+1) Represents updated lightweight network parameters.
[0193] By solving the above optimization problem, we can get the updated lightweight network parameters θ (i+1) , the formula is as follows:
[0194]
[0195]
[0196]
[0197] A=∑ i,j μ2(x ia,l ,x ia,l ), B = ∑ i,j μ2(x ib,j ,x ib,j );
[0198]
[0199] E=∑ i,j μ2(x ia,l ,x ib,j ), F = ∑ i,j μ2(x ib,j, x ia,l ).
[0200] The time-invariant feature components can be obtained through a lightweight network consisting of two fully connected layers and time-varying characteristic components Time-invariant characteristic sequence of herbal medicine Time-varying characteristic sequences of Chinese herbal medicines Perform linear normal mapping as follows:
[0201]
[0202]
[0203] By converting the sequence into a herbal medicine time-invariant feature sequence Time-varying characteristic sequences of Chinese herbal medicines A linear regular mapping is performed to represent the similarity of the features between the two. The formula is as follows:
[0204]
[0205] The decorrelation loss function is used to reduce the correlation between the two feature components and reduce the influence of the time-varying feature component on the time-invariant feature component. The formula is as follows:
[0206]
[0207] Step 5: Construct a feature mapping structure block; by establishing a feature mapping region, establish a mapping relationship between the characteristics of the Chinese herbal medicine in the current growth cycle and the characteristics of the same Chinese herbal medicine in other growth cycles.
[0208] Due to the influence of the growth cycle of Chinese herbal medicine, the appearance of the same Chinese herbal medicine in different cycles may be very different, resulting in intra-class variation exceeding inter-class variation. The current Chinese herbal medicine recognition technology is to train and learn the same Chinese herbal medicine in a single growth stage in isolation, without considering the correlation and connection between the same Chinese herbal medicine in different growth stages, resulting in low recognition accuracy of the same Chinese herbal medicine in different growth stages. Therefore, in order to solve the above problems, based on the current Chinese herbal medicine recognition technology, a feature mapping structure block is constructed to obtain the correlation and connection between the same Chinese herbal medicine in different growth stages.
[0209] refer to Figure 4 , multiple groups of Chinese herbal medicine images are used to obtain the time feature sequence of Chinese herbal medicine through the backbone network Alexnet X i Output the time feature weight sequence of Chinese herbal medicine through the self-attention function F It can be seen that the time feature weight sequence set of multiple types of Chinese herbal medicine is j∈{1,2,3,…,m}, where the time characteristic sequence of the jth type of Chinese herbal medicine The time feature weight sequence set of multiple types of Chinese herbal medicine is used to generate the corresponding feature mapping area. The formula is as follows:
[0210]
[0211] definition Generate corresponding feature mapping regions for multiple types of Chinese herbal medicine time feature weight sequence sets; is the time characteristic series of Chinese herbal medicine The mapping relationship in the feature map area. The activation area generates the activation feature map output by the network By mapping the feature objective function to the feature mapping area, its feature representation is obtained. The formula is as follows:
[0212]
[0213]
[0214]
[0215] By solving the above objective function, we can get the time characteristic sequence of Chinese herbal medicine: The feature relationship matrix with the remaining m-1 Chinese herbal medicines in any growth period is obtained, and the feature mapping matrix of any activated feature map is obtained. The formula is as follows:
[0216]
[0217]
[0218] In the formula, Represents the time characteristic series of Chinese herbal medicine The characteristic relationship matrix with the other m-1 Chinese herbal medicines in any growth period; Represents the feature map matrix that maps activation feature maps to feature map regions.
Claims
1. A method for identifying Chinese herbal medicines by mining time-related features in a smart medical scenario, characterized in that: The Chinese herbal medicine identification method comprises the following steps: S1, collect Chinese herbal medicine pictures, mark the picture category labels and growth cycle labels on the Chinese herbal medicine pictures, generate Chinese herbal medicine image samples, and construct a Chinese herbal medicine image sample dataset; use the Retinex image enhancement algorithm to preprocess the acquired image dataset, and divide each sample image into a training set for training the Chinese herbal medicine recognition model and a test set for testing the Chinese herbal medicine recognition model according to a preset ratio; S2, constructing an M-layer pyramid network model, using the pyramid network model to process the imported Chinese herbal medicine pictures to output high-resolution feature maps with spatial sensitivity, reducing the spatial information positioning deviation when the feature maps on the spatial stream and the semantic stream are fused; the pyramid network model consists of two paths: a bottom-up spatial stream path and a top-down semantic stream path; the Chinese herbal medicine pictures are processed by the spatial stream path to obtain the corresponding feature maps, and the obtained feature maps are imported into the semantic stream path, and the target feature area blocks are generated through pixel shuffling and feature perception reorganization; S3, construct an activation region generation network, which includes a competitive channel attention module, a spatial attention module and a feature processing structure block; the competitive channel attention module is used to make the semantic stream and the spatial stream compete adaptively, and obtain the activation weight matrix of the channel dependency relationship between the spatial stream and the semantic stream; the spatial attention module is used to obtain the activation weight matrix between local features and global features between different levels, and fuse the channel dependency activation weight matrix learned in the competitive channel attention module to obtain the activation feature map output by the activation region generation network; the feature processing structure block includes a self-attention module and a feature decomposition module; the self-attention module is used to capture the context information of the time series and obtain the correlation of the features in the sequence; the feature decomposition module is used to obtain the time-invariant features in the time series features of Chinese herbal medicine; S4, constructing a feature mapping structure block, establishing a mapping relationship between the features of the Chinese herbal medicines in the current growth cycle and the features of the Chinese herbal medicines in the remaining growth cycles; S5, constructing a Chinese herbal medicine recognition network model, training the Chinese herbal medicine recognition network model using the training set to obtain a trained Chinese herbal medicine recognition network model; inputting the test set into the trained Chinese herbal medicine recognition network model to obtain a Chinese herbal medicine recognition result; In step S4, the process of constructing a feature mapping structure block and establishing a mapping relationship between the features of the Chinese herbal medicines in the current growth cycle and the features of the Chinese herbal medicines in the remaining growth cycles includes the following steps: S41, the time characteristic series of Chinese herbal medicine X i Output the time feature weight sequence of Chinese herbal medicine through the self-attention function F The time feature weight sequence set of multiple types of Chinese herbal medicine is j∈{1,2,3,…,m}, where the time characteristic sequence of the jth type of Chinese herbal medicine The time feature weight sequence sets of multiple types of Chinese herbal medicines are used to generate corresponding feature mapping areas. The formula is as follows: In the formula, represents the feature mapping area formed by the characteristics of the jth Chinese herbal medicine in the pth growth period; N p,j represents the total number of images of the jth Chinese herbal medicine in the pth growth period; represents the characteristics of the jth Chinese herbal medicine in the pth growth period; S42, Definition Generate corresponding feature mapping regions for multiple types of Chinese herbal medicine time feature weight sequence sets; is the time characteristic series of Chinese herbal medicine Mapping relationship in feature mapping area; generate activation feature map of network output from activation area By mapping the feature objective function to the feature mapping area, its feature representation is obtained. The formula is as follows: In the formula, E represents the smoothing operator; λ1 and λ2 represent weight factors; S43, calculate the time characteristic sequence of Chinese herbal medicine The feature relationship matrix with the remaining m-1 Chinese herbal medicines in any growth period is obtained, and the feature mapping matrix of any activated feature map is obtained. The formula is as follows: In the formula, Represents the time characteristic series of Chinese herbal medicine The characteristic relationship matrix with the other m-1 Chinese herbal medicines in any growth period; Represents the feature map matrix that maps activation feature maps to feature map regions.
2. The method for identifying Chinese herbal medicines by mining time-related features in a smart medical scenario according to claim 1, characterized in that: In step S1, the acquired image data set is preprocessed using the Retinex image enhancement algorithm according to the following formula: Where r(x, y) represents the output image; R(x, y) represents the reflected image; S(x, y) represents the input image, L(x, y) represents the brightness image; F(x, y) represents the Gaussian center surround function; λ is the normalization coefficient; q is the Gaussian surround scale; x represents the horizontal coordinate of the image pixel; y represents the vertical coordinate of the image pixel.
3. The method for identifying Chinese herbal medicines by mining time-related features in a smart medical scenario according to claim 1, characterized in that: In step S2, the process of processing the imported Chinese herbal medicine image using the upsampling module to output a high-resolution feature map with spatial sensitivity includes the following steps: S21, set up an M-layer pyramid network model. The pyramid network model consists of two paths: a bottom-up spatial flow path and a top-down semantic flow path. On the bottom-up spatial flow path, the features of the i-th layer are output as feature maps through the upsampling module. i∈{1, 2, 3, ..., M-1}, where H i , W i , C b Represent the height, width, and number of channels of the feature map of the i-th layer respectively; in the bottom-up spatial flow path, the output feature of the last convolutional layer of the i-th layer pyramid is S22, the feature D output by the last convolutional layer of the Mth pyramid in the bottom-up spatial flow path M The top-level features after channel adjustment as the top-down semantic flow path Take feature map Any pixel position Form a shape centered at z and of size k up ×k up Predefined feature area blocks Through the convolution of size 1×1 To adjust the channel, the formula is as follows: In the formula, Represents the features after channel adjustment by 1×1 convolution kernel; S23, through the pixel shuffling operator The channel dimension is expanded in the spatial dimension to obtain the feature map Any pixel position The regional upsampling kernel is as follows: In the formula, Representing coordinates The output feature pixels of α represents the scaling factor, αk up Indicates the size of the regional convolution block; S24, by the feature map Perform pixel shuffling operations on all pixel positions of each channel to obtain the predicted upsampling kernel Take feature map Any pixel position Corresponding to the predicted upsampling kernel F M Feature pixel blocks on The feature pixel block F is reorganized by channel reorganization operation. z The channel dimension is converted into the spatial dimension to generate the regional upsampling kernel. The formula is as follows: F′ z =reshape(F z ) Where reshape(·) represents the channel reorganization operation; F′ z The representative size is αk up ×αk up The regional upsampling kernel of S25, predefined feature area blocks With the regional upsampling kernel F′ z The target feature region block is generated through the feature-aware recombination module, and the formula is as follows: In the formula, U′ M-1 Represents any pixel position on the input feature map The mapped pixel position z′=(i′, j′) on the target feature region block on the output feature map, αk up Represents the region upsampling kernel size.
4. The method for identifying Chinese herbal medicines by mining time-related features in a smart medical scenario according to claim 1, characterized in that: In step S3, the competitive channel attention module is used to make the semantic stream and the spatial stream compete adaptively, and the process of obtaining the activation weight matrix of the channel dependency relationship between the spatial stream and the semantic stream includes the following steps: A31, the feature D output by the last convolutional layer of the i-th pyramid in the bottom-up spatial flow path i After lateral connection through 1×1 convolution operation, the output is The formula is as follows: In the formula, represents the features after channel adjustment by 1×1 convolution kernel; i represents the i-th layer; · represents the convolution operation; A32, let the input of the i-th layer feature fusion module of the pyramid network be The input from the spatial stream and the semantic stream is globally averaged and pooled to squeeze the two-dimensional features (H×W) of each channel of each set of feature maps to obtain the average feature. The formula is as follows: In the formula, Descriptor representing the cth channel of the input features on the spatial stream; Represents the descriptor of the kth channel in the input feature on the semantic stream; F sq (·) represents the global average pooling operation; and Represent the features of the kth channel in the i-th layer of input features on the spatial stream and the semantic stream, respectively. Represents the pixel point on the feature map; A33, will and After connecting in the channel dimension, it is used as the joint excitation input. The semantic stream and the spatial stream are made to compete adaptively, and the activation weight matrix of the channel dependency between the spatial stream and the semantic stream is obtained. The formula is as follows: In the formula, F ex (·) is the excitation operation, representing the forward propagation of two fully connected layers; W ex is the weight matrix of the two fully connected layers, 5. The method for identifying Chinese herbal medicines by mining time-related features in a smart medical scenario according to claim 4, characterized in that: In step S3, the spatial attention module is used to obtain the activation weight matrix between local features and global features between different levels, and the channel dependency activation weight matrix learned in the competitive channel attention module is integrated to obtain the activation feature map output by the activation region generation network. The process includes the following steps: B31, the input L of the feature fusion module of the i-th layer of the pyramid network i , U i After the global average pooling operation across channels, the spatial descriptor of the position (x, y) is obtained, and the formula is as follows: In the formula, G sq (·) represents the global average pooling operation across channels; [l i ] x,y 、[u i ] x,y The positions of all channels in the input features of the bottom-up spatial stream path and the top-down semantic stream path in the two streams of the i-th layer are (x, y); Will and Connect the channel dimensions to obtain the spatial relationship information of the fusion features of the i-th layer. The formula is as follows: In the formula, is the weight matrix for adjusting the spatial information relationship between the spatial stream and the semantic stream; G ex (·) represents the forward propagation operation of two convolutional layers; w V is the weight matrix of the two convolutional layers; is the spatial information relationship weight matrix of the bottom-up path and the top-down path; B32, the weight matrix P that coordinates the channel information relationship between the spatial stream and the semantic stream i The weight matrix V of the spatial information relationship between the spatial stream and the semantic stream i Multiply them, and the formula is as follows: In the formula, stands for element-wise multiplication; Represents the activation value of the activation region generation network; B33, Generate network activation values by activating regions The activation feature map output by the activation region generation network is obtained, and the formula is as follows: In the formula, Represents the activation area to generate the network output feature map; Stands for element-wise multiplication.
6. The method for identifying Chinese herbal medicines by mining time-related features in a smart medical scenario according to claim 5, characterized in that: In step S3, the self-attention module is used to capture the context information of the time series, and the process of obtaining the correlation degree of the features in the sequence includes the following steps: C31, stacking the features output by the last convolutional layer of the i-th pyramid in the bottom-up spatial flow path into a temporal feature sequence Set three key learning parameters The formula is as follows: In the formula, Represents the time series X i Feature learning parameters of the p-th cycle in; represents the weight learning matrix; C32, the feature learning parameters are stacked into three parameter sequences, respectively expressed as Get the feature sequence X i Any growth cycle characteristics represented by The three key learning parameter sequences are Get the weight parameter of each head. The formula is as follows: Where σ(·) represents the softmax function; represents the parameter matrix; d q express The dimension, d k express The dimension, d v express The dimension of d k =d v =d q / h, h represents the number of heads; C33, fuses each head output parameter matrix through the learnable parameter matrix W, the formula is as follows: In the formula, Represents the feature weight sequence of Chinese herbal medicine image; 7. The method for identifying Chinese herbal medicines by mining time-related features in a smart medical scenario according to claim 6, characterized in that: In step S3, the process of using the feature decomposition module to obtain the time-invariant features in the time series features of Chinese herbal medicine includes the following steps: D31, the time characteristic series of Chinese herbal medicine X i Output the time feature weight sequence of Chinese herbal medicine through the self-attention function F The time feature weight sequence of Chinese herbal medicine is modeled as a linear combination of time-invariant feature components, time-varying feature components and draw features. in represents time-invariant features, Represents time-varying characteristics, Represents the average value of Chinese herbal medicine features; obtains the time-invariant and time-varying feature components in the Chinese herbal medicine sequence through a lightweight network composed of two fully connected layers, and estimates the parameters of the lightweight network through the expected maximum algorithm The formula is as follows: In the formula, θ (i) represents the given lightweight network parameters; θ represents the estimated lightweight network parameters; P(·) represents the posterior distribution; V represents the characteristics of the Chinese herbal medicine training samples; represents the characteristics of the jth growth cycle of the i-th Chinese herbal medicine; x ia,l With x ib,j denote the corresponding time-invariant factors and time-varying factors respectively; D32, based on a given posterior distribution, performs maximization in an iterative manner The function updates the lightweight network parameters θ, and the formula is as follows: In the formula, θ represents the lightweight network parameters; θ (i+1) Represents updated lightweight network parameters; D33, calculate the updated lightweight network parameters θ (i+1) , the formula is as follows: In the formula, μ1(·) represents P(x ia,l |θ (i) , V) and P(x ib,j |θ (i) , V) first-order moment; μ2(·) represents P(x ia,l |θ (i) , V) and P(x ib,j |θ (i) , the second moment of V); D34, time-invariant feature components are obtained through a lightweight network consisting of two fully connected layers and time-varying characteristic components Time-invariant characteristic sequence of herbal medicine Time-varying characteristic sequences of Chinese herbal medicines Perform a linear normal mapping: In the formula, represents trainable parameters; D35, by time-invariant feature sequence With time-varying characteristic sequence The correlation between them is expressed as: In the formula, μ ia , μ ib Represents Y ia With Y ib The mean of Represents Y ia With Y ib The variance of D36, through the decorrelation loss function to reduce the correlation between the two feature components, reduce the impact of the time-varying feature component on the time-invariant feature component: In the formula, F represents the self-attention function; K represents and The correlation between them.
