Abnormality analysis method and device, storage medium and electronic equipment

Abstract

The invention discloses an anomaly analysis method and device, a storage medium and electronic equipment, and the method comprises the steps: firstly obtaining a plurality of lead electrocardiograms of a user, determining a first feature of the electrocardiograms through a feature extraction model, and extracting a second feature of the electrocardiograms according to a preset to-be-extracted feature parameter; and obtaining a fusion feature based on the first feature and the second feature. And finally, inputting the fusion features into an exception analysis model to obtain an exception analysis result of the user output by the exception analysis model. According to the method, the model with the small parameter quantity is used for carrying out feature extraction and outputting the anomaly analysis result, the second feature obtained through the preset to-be-extracted feature parameter is fused with the first feature extracted by the model to carry out feature supplementation, and then the accuracy of the anomaly analysis result is improved. By means of the feature supplementing mode, the accuracy of the abnormal analysis result is guaranteed, the energy consumption of the model is low, and therefore the use durability of the portable electrocardiograph is improved.

Description

Technical Field

[0001] This specification relates to the field of machine learning, and in particular to an abnormality analysis method, device, storage medium and electronic device. Background Art

[0002] With the development of medical technology, doctors can diagnose whether the user's heart is abnormal based on the ECG signals in the user's ECG, and then formulate a treatment plan. Existing methods use deep learning to train abnormal analysis models, so that the abnormal analysis model can automatically analyze the user's ECG and obtain abnormal analysis results to assist doctors in treatment. Among them, the electrocardiograph used to obtain ECG results can be fixed or portable, so that users can monitor their heart health at home or other non-medical environments, and can also be used for preliminary diagnosis or screening in emergency situations.

[0003] Generally speaking, the anomaly analysis model is usually deployed inside the electrocardiograph in order to obtain the anomaly analysis results in a timely manner. However, the current anomaly analysis model has a large number of parameters and consumes a lot of energy. When the electrocardiograph is portable, since the portable electrocardiograph is powered by a mobile battery, the energy consumption of the anomaly analysis model is high, resulting in low durability of the portable electrocardiograph. Although the anomaly analysis model with fewer parameters consumes less energy, the accuracy of the anomaly analysis results obtained is lower. Therefore, how to ensure the accuracy of the anomaly analysis results while reducing the energy consumption of the anomaly analysis model is an urgent problem to be solved.

[0004] Based on this, this specification provides an abnormality analysis method. Summary of the invention

[0005] This specification provides an abnormality analysis method, device, storage medium and electronic device to at least partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides an abnormality analysis method, including:

[0008] Obtaining electrocardiograms of several leads of the user;

[0009] Determine the first feature of the electrocardiogram through a feature extraction model; and obtain the second feature of the electrocardiogram according to preset feature parameters to be extracted;

[0010] Based on the first feature and the second feature, obtaining a fusion feature;

[0011] The fused features are input into an anomaly analysis model to obtain an anomaly analysis result of the user.

[0012] Optionally, determining the first feature of the electrocardiogram by a feature extraction model specifically includes:

[0013] Preprocessing the electrocardiogram signal in the electrocardiogram to obtain a preprocessed electrocardiogram signal;

[0014] Inputting the preprocessed ECG signal into a feature extraction model to obtain a convolution vector;

[0015] Determine an embedding sequence and a position code of the embedding sequence according to the convolution vector;

[0016] A first feature of the electrocardiogram is determined according to the embedded sequence and the position code of the embedded sequence.

[0017] Optionally, the feature extraction model includes a Transformer encoding module;

[0018] Determining a first feature of the electrocardiogram according to the embedded sequence and the position code of the embedded sequence specifically includes:

[0019] Determining an encoding result through the Transformer encoder module according to the embedded sequence and the position encoding of the embedded sequence;

[0020] The first feature is determined according to a fully connected layer of the feature extraction model and the encoding result.

[0021] Optionally, determining an encoding result through the Transformer encoder module according to the embedded sequence and the position encoding of the embedded sequence specifically includes:

[0022] Determining a sequence input result according to the embedded sequence and the position code of the embedded sequence;

[0023] The sequence input result is input into the Transformer encoder module to determine the encoding result, and the Transformer encoder module includes an attention mechanism.

[0024] Optionally, the Transformer encoder module includes a preset number of Transformer encoders, each of which includes a multi-head self-attention mechanism sublayer and a fully connected feedforward network.

[0025] Optionally, determining the feature parameters to be extracted specifically includes:

[0026] Determine the importance of the feature parameter set to be screened according to a preset method;

[0027] According to the importance, the feature parameters to be screened are gathered to determine the feature parameters to be extracted.

[0028] Optionally, according to a preset method, determining the importance of the feature parameter set to be screened specifically includes:

[0029] The feature parameters to be screened are input into the importance prediction model to obtain the importance of the feature parameters to be screened.

[0030] Optionally, obtaining a fusion feature based on the first feature and the second feature specifically includes:

[0031] Determining static features based on the user information;

[0032] A fusion feature is determined based on the first feature, the static feature, and the second feature.

[0033] This specification provides an abnormality analysis device, the device comprising:

[0034] An image acquisition module, used to acquire electrocardiograms of several leads of the user;

[0035] A feature extraction module, used to determine the first feature of the electrocardiogram through a feature extraction model; and obtain the second feature of the electrocardiogram according to preset feature parameters to be extracted;

[0036] A feature fusion module, used for obtaining a fusion feature based on the first feature and the second feature;

[0037] The anomaly analysis module is used to input the fusion feature into the anomaly analysis model to obtain the anomaly analysis result of the user.

[0038] Optionally, the feature extraction module is specifically used to preprocess the ECG signal in the ECG to obtain a preprocessed ECG signal; input the preprocessed ECG signal into a feature extraction model to obtain a convolution vector; determine an embedded sequence and a position coding of the embedded sequence according to the convolution vector; and determine a first feature of the ECG according to the embedded sequence and the position coding of the embedded sequence.

[0039] Optionally, the feature extraction model includes a Transformer encoding module;

[0040] The feature extraction module is specifically used to determine the encoding result through the Transformer encoder module according to the embedded sequence and the position encoding of the embedded sequence; and determine the first feature according to the fully connected layer of the feature extraction model and the encoding result.

[0041] Optionally, the feature extraction module is specifically used to determine a sequence input result according to the embedded sequence and the position encoding of the embedded sequence; input the sequence input result into the Transformer encoder module to determine the encoding result, and the Transformer encoder module includes an attention mechanism.

[0042] Optionally, the Transformer encoder module includes a preset number of Transformer encoders, each of which includes a multi-head self-attention mechanism sublayer and a fully connected feedforward network.

[0043] Optionally, the device further comprises:

[0044] The module for determining the characteristic parameters to be extracted is used to determine the importance of the characteristic parameter set to be screened according to a preset method; according to the importance, the characteristic parameters to be extracted are determined by collecting the characteristic parameters to be screened.

[0045] Optionally, the module for determining the characteristic parameters to be extracted is specifically used to input the characteristic parameters to be screened into an importance prediction model to obtain the importance of the characteristic parameters to be screened.

[0046] Optionally, the feature fusion module is specifically used to determine a static feature based on the user information; and determine a fusion feature based on the first feature, the static feature and the second feature.

[0047] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned abnormality analysis method is implemented.

[0048] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the above-mentioned abnormality analysis method is implemented when the processor executes the program.

[0049] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0050] It can be seen from the abnormal analysis method provided in this specification that the electrocardiograms of several leads of the user are first obtained, and the first feature of the electrocardiogram is determined through the feature extraction model, and the second feature of the electrocardiogram is obtained according to the preset feature parameters to be extracted. Then, based on the first feature and the second feature, a fusion feature is obtained. Finally, the fusion feature is input into the abnormal analysis model to obtain the abnormal analysis result of the user output by the abnormal analysis model. This method uses a model with a small number of parameters to extract features and output abnormal analysis results, and obtains the second feature through the preset feature parameters to be extracted and the first feature extracted by the model for feature supplementation, thereby improving the accuracy of the abnormal analysis results. The feature supplementation method not only ensures the accuracy of the abnormal analysis results, but also makes the model energy consumption lower, thereby improving the durability of the portable electrocardiograph. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0052] Figure 1 A flowchart of an abnormality analysis method provided in this specification;

[0053] Figure 2 A schematic diagram of a process for determining feature parameters to be extracted provided in this specification;

[0054] Figure 3 A schematic diagram of the internal structure of an importance prediction model is provided for this specification;

[0055] Figure 4 Window diagram provided for this manual;

[0056] Figure 5 A schematic diagram of the model extraction feature flow provided for this specification;

[0057] Figure 6 A schematic diagram of the internal structure of the Transformer encoder module provided for this specification;

[0058] Figure 7 A schematic diagram of the internal structure of a single Transformer encoder provided for this specification;

[0059] Figure 8 A schematic diagram of sub-layer hopping connections provided in this specification;

[0060] Fig. 9 A schematic diagram of the abnormal analysis process provided for this manual;

[0061] Fig.10A schematic diagram of an abnormality analysis device provided in this specification;

[0062] Fig.11 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0064] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0065] Figure 1 A flowchart of an abnormality analysis method provided in this specification specifically includes the following steps:

[0066] S100: Obtaining electrocardiograms of several leads of the user.

[0067] Electrocardiograms can assist doctors in diagnosing heart abnormalities. Doctors can determine whether the user has heart abnormalities and the category of the user's heart abnormalities, such as atrial fibrillation, bradycardia, etc., through the waveform features in the electrocardiogram, and then perform treatment. In order to improve the diagnostic efficiency of doctors, the abnormal analysis results determined by the abnormal analysis model trained by machine learning can assist doctors in diagnosis, but the current abnormal analysis model has a large number of parameters. For example, the abnormal analysis model based on convolutional neural networks (CNN), recurrent neural networks (RNN), long short-term memory networks (LSTM) and autoencoder-decoder has a large number of parameters. Then, due to the large number of parameters of the abnormal analysis model, the amount of calculation is also relatively large, resulting in high energy consumption, which reduces the working time of the portable electrocardiograph, that is, the durability is low. Therefore, this manual provides an abnormal analysis method.

[0068] The execution subject of this specification can be an image processing device that processes medical images, or it can be other electronic devices that can be deployed to process various models of medical images. For the sake of convenience, this specification is described with the image processing device as the execution subject. Various models can be deployed in the image processing device to obtain abnormal analysis results. When the type of electrocardiograph is portable, the image processing device can be placed in the portable electrocardiograph. When the type of electrocardiograph is fixed, the image processing device can be placed in the fixed electrocardiograph, or it can exist independently and communicate with the fixed electrocardiograph to complete the abnormal analysis. This specification does not limit the type of electrocardiograph. For portable electrocardiographs, the abnormal analysis method provided in this specification can reduce energy consumption and improve the durability of portable electrocardiographs while ensuring the accuracy of abnormal analysis results. For fixed electrocardiographs, it can also save energy and ensure the accuracy of abnormal analysis results.

[0069] In one or more embodiments of the present specification, since the abnormal analysis result is determined based on the medical image of the user, the image processing device may first obtain electrocardiograms of several leads of the user.

[0070] Specifically, the electrocardiograph obtains electrocardiograms of several leads based on the doctor's lead wire connection operation, and the electrocardiograph sends the electrocardiograms of several leads to the image processing device, and the image processing device receives the electrocardiograms of several leads sent by the electrocardiograph. The electrocardiogram can be an electrocardiogram of any one or more of the 12 leads. Of course, the user himself or other personnel can also connect the lead wires to obtain the electrocardiogram. The aforementioned scenario can be used to obtain abnormal analysis results in emergency situations or other situations. This specification does not limit the specific object of operating the electrocardiograph.

[0071] S102: Determine a first feature of the electrocardiogram through a feature extraction model; and extract a second feature of the electrocardiogram according to preset feature parameters to be extracted.

[0072] After acquiring the electrocardiogram, the image processing device can perform feature extraction on the electrocardiogram so as to obtain abnormal analysis results based on the features of the electrocardiogram. The image processing device can determine the first feature of the electrocardiogram through a feature extraction model. That is, the electrocardiogram is input into the feature extraction model to obtain the first feature of the electrocardiogram output by the feature extraction model.

[0073] The feature extraction model can be deployed in an electrocardiograph. Since the feature extraction model has a small number of parameters, the energy used by the feature extraction model is less than that used by an existing abnormal analysis model with a large number of parameters. In other words, the energy consumption of the feature extraction model is lower than that of an existing abnormal analysis model with a large number of parameters. Then, when the type of the electrocardiograph is portable, the durability of the portable electrocardiograph deployed with the feature extraction model is higher than that of the portable electrocardiograph deployed with the existing abnormal analysis model with a large number of parameters.

[0074] Further, the image processing device can also obtain the second feature of the electrocardiogram according to the preset feature parameters to be extracted, and the feature parameters to be extracted may include the minimum heart rate (HRmin), T wave multiscale permutation entropy (T wave multiscalepermutation entropy), maximum heart rate (HRmax), R peak width and a series of power spectrum density information, etc. The second feature includes the specific value of the feature parameter to be extracted in the electrocardiogram. For example, the feature parameter to be extracted is the minimum heart rate. The image processing device can calculate the time interval between two adjacent R waves, that is, the RR interval, according to the electrocardiogram signal in the electrocardiogram, and then use the heart rate conversion formula: Beat Per Minute, BPM = 60 / (RR interval / 1000) to determine several heart rates, and finally determine that the second feature is 60 times / minute. The feature parameter to be extracted can be determined from the set of feature parameters to be screened that is manually determined based on experience, or can be preset according to other methods, and this specification does not limit this.

[0075] Since the features extracted by the feature extraction model with a smaller parameter amount will affect the accuracy of the final abnormality analysis results, the image processing device obtains a second feature with a supplementary and / or prompting function through the preset features to be extracted, and uses the first feature and the second feature in combination to improve the accuracy of the abnormality analysis results.

[0076] S104: Obtain a fusion feature based on the first feature and the second feature.

[0077] In one or more embodiments provided in this specification, the image processing device may perform feature fusion on the first feature extracted by the feature extraction model and the second feature obtained according to the preset feature parameters to be extracted to obtain a fused feature. This specification does not limit the specific feature fusion method. For example, the image processing device may directly perform feature splicing on the first feature and the second feature, and use the spliced ​​feature as the fused feature. It is also possible to use a weighted summation method to multiply each feature by the corresponding weight coefficient and then add them together to obtain a weighted feature as the fused feature.

[0078] The fusion feature combines the features extracted by the model and the supplementary / prompt features obtained by the feature parameters to be extracted. The features extracted by different feature extraction methods may be complementary, so that the amount of information provided by the fusion feature is greater than that provided by a single feature. In addition, when multiple features are combined together, a richer representation can be formed, which helps the subsequent abnormal analysis model capture more complex feature information. Therefore, using fusion features to determine the abnormal analysis results is conducive to improving the accuracy of the determined abnormal analysis results.

[0079] S106: Input the fusion feature into an abnormality analysis model to obtain an abnormality analysis result of the user.

[0080] Specifically, the image processing device inputs the fused feature into the abnormal analysis model, so that the abnormal analysis model analyzes the fused feature, determines several cardiac abnormality categories matching the fused feature, and feature expressions corresponding to the cardiac abnormality categories, and outputs them as the user's abnormal analysis result, and the abnormal analysis result may also include normal values ​​corresponding to normal cardiac performance, etc. For example, the abnormal analysis result is sinus tachycardia: the heart rate is continuously higher than the normal range, the normal range is less than 100 beats / minute, and the cardiac abnormality category is arrhythmia.

[0081] It should be noted that the sum of the parameter amount of the feature extraction model in this specification and the parameter amount of the abnormality analysis model in step S106 of this specification is less than the existing abnormality analysis model. Therefore, the energy consumption of determining the abnormality analysis result through the abnormality analysis method provided in this specification is lower than the energy consumption of obtaining the abnormality analysis result through the existing abnormality analysis model.

[0082] based on Figure 1 The abnormal analysis method shown first obtains the electrocardiograms of several leads of the user, determines the first feature of the electrocardiogram through a feature extraction model, and obtains the second feature of the electrocardiogram according to the preset feature parameters to be extracted. Then, based on the first feature and the second feature, a fusion feature is obtained. Finally, the fusion feature is input into the abnormal analysis model to obtain the abnormal analysis result of the user output by the abnormal analysis model. This method uses a model with a small number of parameters to perform feature extraction and output abnormal analysis results, and obtains the second feature obtained by the feature parameters to be extracted and the first feature extracted by the model for feature supplementation, thereby improving the accuracy of the abnormal analysis results. The feature supplementation method not only ensures the accuracy of the abnormal analysis results, but also makes the model energy consumption low, thereby improving the durability of the portable electrocardiograph.

[0083] Before executing step S102 of extracting the second feature, the image processing device needs to predetermine the feature parameters to be extracted. Figure 2 A flow chart of determining the feature parameters to be extracted provided in this specification, such as Figure 2 shown.

[0084] The preset feature parameters to be extracted are determined from the feature parameter set to be screened, and the feature parameter set to be screened can be determined manually based on experience, or can be determined based on features extracted from an abnormal analysis model with a large number of parameters, or can be determined by other methods, which are not limited in this specification. For example, several features of the electrocardiogram signal of the electrocardiogram, including heart rate variability features and morphological features, are extracted from lead II by manual extraction, and stored in the feature parameter set to be screened.

[0085] When determining the feature parameters to be extracted, the image processing device can determine the importance of the feature parameter set to be screened according to a preset method. That is, the importance of each feature parameter to be screened in the feature parameter set to be screened is determined, and then, according to the importance, the feature parameters to be extracted are determined in the feature parameter set to be screened. For example, the feature parameters to be screened are sorted according to their importance, and in the sorting result, a preset number of feature parameters to be screened with higher importance are selected as feature parameters to be extracted. Of course, the feature parameter set to be screened can also be screened by other methods to determine the feature parameters to be extracted, such as cluster analysis, by clustering the feature parameters to be screened to identify similar feature groups, and then selecting representative features in each group as feature parameters to be extracted. Another example is principal component analysis (PCA), etc.

[0086] By adding additional features, features that are not extracted by the feature extraction model can be supplemented, so that the anomaly analysis model can obtain more features to improve the accuracy of the anomaly analysis results. In addition, the second feature is obtained based on the feature parameters to be extracted, and the feature parameters to be extracted are feature parameters with higher importance selected from the set of feature parameters to be selected. Therefore, for the anomaly analysis model, determining the results based on features with higher importance also helps to improve the accuracy of the anomaly analysis results.

[0087] The preset method for the image processing device to determine the importance of the feature parameters to be screened may include using an importance prediction model to determine it. Specifically, the feature parameters to be screened may be input into the importance prediction model to obtain the importance of the feature parameters to be screened. The importance prediction model may be a trained random forest model or other types of models as long as it can determine the importance of the feature parameters to be screened. Figure 3 A schematic diagram of the internal structure of an importance prediction model is provided for this specification, such as Figure 3The importance prediction model includes several decision trees. The feature parameters to be screened are input into the importance prediction model. Each decision tree processes the feature parameters to be screened, determines the result output by the decision tree, and then determines the importance of the feature parameters to be screened.

[0088] The method of determining the importance of feature parameters through a model in this specification is more convenient, efficient, and more accurate, which facilitates the subsequent determination of the second feature.

[0089] Before executing the extraction of the first feature in step S102, the image processing device may also preprocess the electrocardiogram, and the preprocessing may include frequency upsampling, frequency downsampling, denoising, normalization, and at least one of intercepting an electrocardiogram signal of a preset length. For example, for each electrocardiogram signal, the electrocardiogram signal is upsampled or downsampled to 500Hz so that the electrocardiogram signals have the same sampling rate for subsequent use. Then, the electrocardiogram signal is denoised by a finite impulse response (FIR) bandpass filter with a bandwidth between 3Hz and 45Hz, and the electrocardiogram signal with a signal frequency between 3Hz and 45Hz is retained. Afterwards, the electrocardiogram signal is normalized so that the electrocardiogram signal range of each channel is between -1 and 1.

[0090] Figure 4 The window diagram provided for this manual is as follows: Figure 4 shown.

[0091] For each electrocardiogram, the electrocardiogram signal in the window area can be extracted according to a window with a preset time width. Figure 4 The dotted rectangle is a window with a preset fixed time width, which includes ECG signals of two heart cycles. If the preset time width T=15 seconds, the 15-second window length can usually include 15 to 25 complete heart cycles. Then, due to the limited length of the ECG signal, there may be a situation where there is no ECG signal at a certain moment in the window. In this case, the image processing device can apply zero padding to the window area without ECG signal.

[0092] Preprocessing ECG signals can improve their quality, remove noise, enhance useful signal features, and help improve subsequent feature extraction to obtain more accurate abnormality analysis results.

[0093] Further, for step S102, the processed ECG signal is obtained, and the image processing device can input the pre-processed ECG signal into the feature extraction model, and the embedding network in the feature extraction model can perform multi-layer convolution operations on the ECG signal to obtain a convolution vector. Afterwards, based on the convolution vector, the embedding sequence and the position coding of the embedding sequence are determined. Specifically, the convolution vector can be downsampled by a preset downsampling factor to obtain the embedding sequence and the position coding of the embedding sequence, and the downsampling factor represents the change ratio of the dimension of the convolution vector before downsampling relative to the dimension of the convolution vector after downsampling. The convolution vector can also be downsampled by other methods to obtain the embedding sequence and the position coding of the embedding sequence, such as by sparse convolution, maximum pooling, average pooling, global pooling, etc. Among them, the embedding sequence can be used (x1,...,x n ) indicates that the position encoding of the embedded sequence can be expressed as p = (p1, ..., p n Finally, the image processing device can determine the first feature of the electrocardiogram according to the embedded sequence and the position code of the embedded sequence.

[0094] Convolution of the ECG signal in the ECG can capture the latent space representation of the ECG signal and further improve the accuracy of the abnormal analysis results. Downsampling can simplify the convolution vector, thereby reducing the computational complexity and amount of calculation to reduce energy consumption. The position code is obtained so that the subsequent feature extraction model can determine the position of the ECG waveform corresponding to the embedded sequence according to the position code of the embedded sequence. Moreover, even if two ECG signal fragments have similar features, if they appear at different time points, they can be understood as appearing at different positions, then the information they carry may be different. The position code ensures that the feature extraction model can identify these differences and extract relevant feature information. By adding a unique position code to each embedded sequence, the feature extraction model can better understand the change pattern of the ECG signal over time, and then extract better features to improve the accuracy of the abnormal analysis results.

[0095] Figure 5 The model extraction feature flow diagram provided for this manual is as follows: Figure 5 shown.

[0096] First, the image processing device processes the electrocardiogram through the embedding network of the feature extraction model to obtain the embedding sequence of the medical image and the position coding of the embedding sequence. Then, according to the embedding sequence and the position coding of the embedding sequence, the encoding result is determined through the Transformer encoder module included in the feature extraction model. Then, according to the fully connected layer of the feature extraction model and the encoding result, the first feature is determined.

[0097] Specifically, when determining the encoding result, the image processing device first determines the sequence input result according to the embedding sequence and the position encoding of the embedding sequence, and then inputs the sequence input result into the Transformer encoder module to determine the encoding result. The Transformer encoder module includes an attention mechanism. The sequence input result can be obtained by directly adding the embedding sequence and the corresponding position encoding. For example, the embedding sequence is (x1, ..., x n ), the position encoding of the embedded sequence is represented by p = (p1, ..., p n ) means, then the sequence input result is (x1+p1, ..., x n +p n ), x i 、p i is a real number. Of course, it can also be determined in other ways, such as concatenating the position encoding vector and the embedding vector, and after concatenation, the dimension needs to be reduced to the original dimension through a linear layer or other means. This specification does not limit the method of determining the sequence input result, but it is necessary to enable the Transformer encoder module to determine the time order of the input embedding sequence.

[0098] This is because the Transformer encoder module itself does not have the ability to process the temporal order information of sequence data. In other words, the Transformer encoder module is position agnostic, which means that it cannot distinguish the temporal order of the embedded sequence based on the input embedding sequence itself. Therefore, we need to explicitly encode the position information that can represent the temporal order of the sequence into the input embedding sequence so that the Transformer encoder module can understand the temporal order of the input embedding sequence and determine the first feature.

[0099] Since the Transformer encoder module includes an attention mechanism, after the sequence input result is input into the Transformer encoder module, the Transformer encoder module can process the sequence input result through an attention mechanism, such as a scaled dot product attention mechanism, based on the pre-learned network weight matrix to generate a query vector, a key vector, and a value vector. Then the query vector, the key vector, and the value vector are stacked into matrices Q, K, and V. Among them, the network weight matrix includes the query weight matrix W Q , key weight matrix W K , value weight matrix W V .

[0100] Furthermore, the self-attention mechanism is used to determine the encoding result. The specific formula is as follows:

[0101]

[0102] dk represents the dimension of the key vector, QK T is the dot product of matrix Q and matrix K. Attention(Q, K, V) is the self-attention weight, and the stacked matrix V is weighted and summed by the self-attention weight to obtain the encoding result.

[0103] Since ECG signals usually contain a large number of sampling points, traditional RNNs may be difficult to process effectively due to problems such as gradient vanishing and gradient exploding. However, the Transformer encoder module can handle these long sequence data well, and combined with the attention mechanism, the Transformer encoder module can capture local and global features at the same time, which is helpful for ECG signal feature analysis. In addition, the self-attention layer in the Transformer encoder module can be calculated in parallel, which means that the Transformer encoder module can process all elements in the embedded sequence at the same time, without having to process them element by element like RNN, which makes the Transformer encoder module faster in processing long sequences. When there are many ECG signals, the processing efficiency is significantly improved.

[0104] Figure 6 The internal structure diagram of the Transformer encoder module provided for this manual is as follows Figure 6 shown.

[0105] The Transformer encoder module includes a preset number of Transformer encoders, which is illustrated as 8. The Transformer encoder module includes 8 Transformer encoders, each of which is connected in series, that is, the output of the previous Transformer encoder can be used as the input of the next Transformer encoder.

[0106] Using multiple Transformer encoders in the feature extraction model enables the feature extraction model to learn deeper feature representations. Each encoder layer can focus on learning a specific level of abstract features, so that the model can better understand the complex structure of the input embedding sequence. In addition, by stacking multiple Transformer encoders, the feature extraction model can learn richer feature combinations, making it easier to process complex sequence data and improve the performance of the feature extraction model.

[0107] Figure 7 The internal structure diagram of a single Transformer encoder provided for this specification is as follows Figure 7 shown.

[0108] Each Transformer encoder may include a multi-head self-attention mechanism sublayer. This specification takes the example that each Transformer encoder includes 8 multi-head self-attention mechanism sublayers. Each multi-head self-attention mechanism sublayer is connected to a normalization layer, and the multi-head self-attention mechanism sublayers run in parallel. A fully connected feedforward network is connected to each Transformer encoder, so the outputs of the 8 multi-head self-attention mechanism sublayers pass through their own normalization layer and are input into the fully connected feedforward network. Since the Transformer encoder is combined with the attention mechanism, each multi-head self-attention mechanism sublayer has its own network weight matrix, so that the feature extraction model can extract the dependencies of the ECG signals at different positions in the embedded sequence.

[0109] The multi-head attention mechanism enables the Transformer encoder module to learn different features from multiple representation subspaces, which helps capture multiple patterns in ECG signals. For complex ECG signals, this helps improve the quality of extracted features to improve the accuracy of abnormal analysis results. The normalization layer helps speed up the training process of the feature extraction model and improve the stability of the feature extraction model.

[0110] Figure 8 The sub-layer hopping connection diagram provided for this specification is as follows: Figure 8 shown.

[0111] Each multi-head self-attention mechanism sublayer can also have a jump connection. If the input of the multi-head self-attention mechanism sublayer is x and the output is y, then x and y can be added and input into the normalization layer.

[0112] In deep neural networks, as the number of network layers increases, the gradient may gradually become smaller during the back propagation process, resulting in the gradient vanishing phenomenon. Skip connections allow the gradient to be directly passed to shallower network layers, helping to alleviate the problem of gradient vanishing. In addition, skip connections enable each layer to not only learn new features but also retain information from the previous layer. This enhances the overall expressiveness of the model because each layer can focus on learning specific changes or patterns without losing information from previous layers.

[0113] It should be noted that, generally speaking, the encoder and decoder are used together, but since in this specification, the Transformer encoder module is mainly used to extract features to facilitate the subsequent determination of abnormal analysis results rather than sequence prediction, the decoder may not be used in this specification, and only the encoder is used to further reduce the number of model parameters to reduce energy consumption.

[0114] After obtaining the encoding result, the image processing device can perform global average pooling on the encoding result, and then use the fully connected layer of the feature extraction model to determine the first feature of the electrocardiogram. That is, the encoding result after global average pooling is input into the fully connected layer of the feature extraction model to obtain the first feature.

[0115] Global average pooling of the encoding results can significantly reduce the feature dimension, thereby reducing the computational cost and memory consumption, which helps to improve the operating efficiency of the model. In addition, it can also enable the anomaly analysis model to automatically select important features, which helps to extract key information and thus improve the anomaly analysis results. Processing through the fully connected layer is to facilitate the subsequent classification tasks in the anomaly analysis and obtain the anomaly classification results.

[0116] With respect to step S104, in one or more embodiments of this specification, Fig. 9 The abnormal analysis flow diagram provided for this manual is as follows: Fig. 9 shown.

[0117] The image processing device may also determine a static feature based on the user's information, the static feature including at least one of the user's gender and the user's age. The fusion feature is determined using the first feature obtained by the feature extraction model, the second feature obtained according to the preset feature parameters to be extracted, and the static feature determined according to the user's information. In other words, the fusion feature is determined based on the first feature, the static feature, and the second feature. Step S106 is then executed to obtain an abnormality analysis result.

[0118] By using the user's information to determine the user's personal static features, when the fusion features based on the user's personal static features are subsequently used to determine the abnormal analysis results, it can help the abnormal analysis model better understand the individual's physiological differences and age differences, and achieve personalized analysis to obtain more accurate abnormal analysis results. This is because some heart diseases are more common in people of a certain gender or age group, and some electrocardiogram abnormalities may be normal variations in young people, but may indicate the presence of disease in the elderly. Adding features such as gender and age can also provide the abnormal analysis model with more information about the individual's background, which helps to improve the accuracy of the abnormal analysis results output by the abnormal analysis model.

[0119] This specification also provides a model training method, in which an initial feature extraction model and an initial anomaly analysis model are jointly trained. The execution subject of the model training method provided in this specification is a server or other computing device that can execute model training, such as a personal computer (PC), etc. This specification uses a server for illustration.

[0120] First, the server obtains the annotated abnormality analysis results as labels, and obtains a sample ECG including several leads. Then, the sample ECG is input into the initial feature extraction model to obtain the first prediction feature of the sample ECG, and the second prediction feature of the sample ECG is obtained using the predetermined feature parameters to be extracted. Based on the first prediction feature and the second prediction feature, a fused prediction feature is obtained, and the fused prediction feature is input into the initial abnormality analysis model to obtain the sample abnormality analysis result output by the initial abnormality analysis model. Finally, according to the label, the sample abnormality analysis result and a preset loss function, such as the average binary cross entropy, the loss is determined, and the initial abnormality analysis model and the initial feature extraction model are trained according to the loss to obtain the abnormality analysis model and the feature extraction model. We train separate models for 2, 3, 6 and 12 leads respectively.

[0121] The server can also train a feature extraction model and anomaly analysis model dedicated to a specific lead type. The training process is similar to the aforementioned training process, and the model differs only in the channel dimension of the convolution. The model structure can be adaptively adjusted according to the lead type of the ECG.

[0122] This is because the lead types of the ECG used for abnormal analysis are different each time, and there may be a lack of ECGs of a certain lead type, which in turn affects the abnormal analysis results. Therefore, for ECGs of different lead types, training specific abnormal analysis models and feature extraction models can make the model specialize in one type of ECG, thereby improving the accuracy of the abnormal analysis results. Of course, when using, the abnormal analysis models and feature extraction models of multiple lead types can be used together to obtain multiple abnormal analysis results. Finally, the final abnormal analysis results are determined by majority voting and averaging.

[0123] The above is an abnormality analysis method provided in one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding abnormality analysis device, such as Fig.10 The device comprises:

[0124] This specification provides an abnormality analysis device, the device comprising:

[0125] The image acquisition module 1000 is used to acquire the electrocardiogram of several leads of the user;

[0126] The feature extraction module 1002 is used to determine the first feature of the electrocardiogram through a feature extraction model; and obtain the second feature of the electrocardiogram according to the preset feature parameters to be extracted;

[0127] A feature fusion module 1004, configured to obtain a fusion feature based on the first feature and the second feature;

[0128] The anomaly analysis module 1006 is used to input the fusion feature into an anomaly analysis model to obtain an anomaly analysis result of the user.

[0129] Optionally, the feature extraction module 1002 is specifically used to preprocess the ECG signal in the ECG to obtain a preprocessed ECG signal; input the preprocessed ECG signal into a feature extraction model to obtain a convolution vector; determine an embedded sequence and a position coding of the embedded sequence according to the convolution vector; and determine a first feature of the ECG according to the embedded sequence and the position coding of the embedded sequence.

[0130] Optionally, the feature extraction model includes a Transformer encoding module;

[0131] The feature extraction module 1002 is specifically used to determine the encoding result through the Transformer encoder module according to the embedded sequence and the position encoding of the embedded sequence; and determine the first feature according to the fully connected layer of the feature extraction model and the encoding result.

[0132] Optionally, the feature extraction module 1002 is specifically used to determine a sequence input result according to the embedded sequence and the position encoding of the embedded sequence; input the sequence input result into the Transformer encoder module to determine the encoding result, and the Transformer encoder module includes an attention mechanism.

[0133] Optionally, the Transformer encoder module includes a preset number of Transformer encoders, each of which includes a multi-head self-attention mechanism sublayer and a fully connected feedforward network.

[0134] Optionally, the device further comprises:

[0135] The module for determining the characteristic parameters to be extracted is used to determine the importance of the characteristic parameter set to be screened according to a preset method; according to the importance, the characteristic parameters to be extracted are determined by collecting the characteristic parameters to be screened.

[0136] Optionally, the module for determining the characteristic parameters to be extracted is specifically used to input the characteristic parameters to be screened into an importance prediction model to obtain the importance of the characteristic parameters to be screened.

[0137] Optionally, the feature fusion module 1004 is specifically used to determine a static feature based on the user information; and determine a fusion feature based on the first feature, the static feature and the second feature.

[0138] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 Provides anomaly analysis methods.

[0139] This manual also provides Fig.11 The structural diagram of the electronic device shown in FIG. Fig.11 As shown, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0140] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0141] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0142] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0143] For the convenience of description, the above device is described by dividing it into various units according to its functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0144] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0148] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0149] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0150] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0151] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0152] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0153] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0154] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0155] The above description is only an embodiment of this specification and is not intended to limit this specification. For those skilled in the art, this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification should be included in the scope of the claims of this application.

Claims

1. An abnormality analysis method, It is characterized in that The method comprises: Obtaining electrocardiograms of several leads of the user; Determine the first feature of the electrocardiogram through a feature extraction model; and obtain the second feature of the electrocardiogram according to preset feature parameters to be extracted; Based on the first feature and the second feature, obtaining a fusion feature; The fused features are input into an anomaly analysis model to obtain an anomaly analysis result of the user.

2. The method according to claim 1, It is characterized in that Determining the first feature of the electrocardiogram by a feature extraction model specifically includes: Preprocessing the electrocardiogram signal in the electrocardiogram to obtain a preprocessed electrocardiogram signal; Inputting the preprocessed ECG signal into a feature extraction model to obtain a convolution vector; Determine an embedding sequence and a position code of the embedding sequence according to the convolution vector; A first feature of the electrocardiogram is determined according to the embedded sequence and the position code of the embedded sequence.

3. The method according to claim 2, It is characterized in that The feature extraction model includes a Transformer encoding module; Determining a first feature of the electrocardiogram according to the embedded sequence and the position code of the embedded sequence specifically includes: Determining an encoding result through the Transformer encoder module according to the embedded sequence and the position encoding of the embedded sequence; The first feature is determined according to a fully connected layer of the feature extraction model and the encoding result.

4. The method according to claim 3, It is characterized in that According to the embedding sequence and the position coding of the embedding sequence, determining the coding result through the Transformer encoder module specifically includes: Determining a sequence input result according to the embedded sequence and the position code of the embedded sequence; The sequence input result is input into the Transformer encoder module to determine the encoding result, and the Transformer encoder module includes an attention mechanism.

5. The method according to claim 3, It is characterized in that The Transformer encoder module includes a preset number of Transformer encoders, each of which includes a multi-head self-attention mechanism sublayer and a fully connected feedforward network.

6. The method according to claim 1, It is characterized in that Determine the feature parameters to be extracted, including: Determine the importance of the feature parameter set to be screened according to a preset method; According to the importance, the feature parameters to be screened are gathered to determine the feature parameters to be extracted.

7. The method according to claim 6, It is characterized in that According to the preset method, the importance of the feature parameter set to be screened is determined, including: The feature parameters to be screened are input into the importance prediction model to obtain the importance of the feature parameters to be screened.

8. The method according to claim 1, It is characterized in that Based on the first feature and the second feature, a fusion feature is obtained, which specifically includes: Determining static features based on the user information; A fusion feature is determined based on the first feature, the static feature, and the second feature.

9. A computer-readable storage medium, It is characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

10. An electronic device, It is characterized in that The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images