Electrocardiogram image enhancement processing method and system based on deep reinforcement learning

Abstract

The invention discloses an electrocardiogram image enhancement processing method and system based on deep reinforcement learning, and particularly relates to the technical field of image enhancement. The method comprises the following steps: acquiring electrocardiogram image data of a patient, and performing denoising and signal standardization processing to obtain standardized electrocardiogram image data; extracting local pathological features and global visual features of the electrocardiogram image by using the standardized electrocardiogram image data to form multi-scale feature space data; constructing a global visual feature reward function based on the multi-scale feature space data, generating global visual feature reward data, and generating local sensitive region enhanced reward data by setting a local sensitive region of a weak pathological waveform and a reward sensitivity dynamic adjustment method; and reward sparseness evaluation is executed according to the global visual feature reward data and the local sensitive area enhanced reward data, and enhanced electrocardiogram image data is output through a deep reinforcement learning model, so that the clinical diagnosis value of the electrocardiogram image is enhanced.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and more specifically, to a method and system for enhancing electrocardiogram images based on deep reinforcement learning. Background Technology

[0002] Electrocardiograms (ECGs) are an important tool for the clinical diagnosis of heart diseases, and are often presented visually in the form of images. However, during the actual acquisition process, signals are easily affected by factors such as equipment noise, external interference, and individual patient differences, leading to reduced image quality and blurred characteristic waveforms in ECG images, which affects doctors' accurate identification and judgment of pathological features.

[0003] Existing deep reinforcement learning image enhancement methods typically use the improvement of overall visual quality as the optimization goal, without considering the insufficient weight of weak pathological waveform features in electrocardiograms that have important diagnostic value but low frequency of occurrence in the overall optimization goal. This results in the enhanced image suppressing key pathological features and reducing the clinical diagnostic reliability of the enhanced image. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide an electrocardiogram image enhancement processing method and system based on deep reinforcement learning to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The ECG image enhancement processing method based on deep reinforcement learning includes the following steps: S1: Collect the patient's electrocardiogram (ECG) image data, perform noise reduction and signal standardization processing to obtain standardized ECG image data; S2: Based on standardized electrocardiogram (ECG) image data, construct a multi-scale convolutional neural network model to extract local pathological features and global visual features of ECG images, forming multi-scale feature space data; S3: Based on the multi-scale feature space data, a reinforcement learning strategy is used to optimize the construction of a global visual feature reward function, evaluate the degree of improvement in global visual quality, and output global visual feature reward data; S4: Based on multi-scale feature space data, local sensitive areas of weak pathological waveforms are defined, and a reward sensitivity dynamic adjustment method is adopted to output enhanced reward data for local sensitive areas; S5: Based on the global visual feature reward data and the local sensitive area enhancement reward data, perform reward sparsity evaluation, determine whether the reward enhancement mechanism is triggered, and output the adjusted comprehensive reward data; S6: Based on the adjusted comprehensive reward data, generate an electrocardiogram (ECG) image enhancement strategy through a deep reinforcement learning model to generate enhanced ECG image data.

[0006] In a preferred embodiment, S1 specifically refers to: Collect electrocardiogram (ECG) images of patients; Noise filtering was performed on the patient's electrocardiogram (ECG) image data to remove high-frequency noise and baseline drift interference, resulting in denoised ECG image data. Signal standardization is performed on the denoised electrocardiogram (ECG) image data to adjust the amplitude range and baseline level of the waveforms in the ECG image data, thereby obtaining standardized ECG image data.

[0007] In a preferred embodiment, S2 specifically refers to: Based on standardized electrocardiogram (ECG) image data, a multi-scale convolutional neural network model was constructed. Convolution operations were performed on the standardized ECG image data to extract ECG image features at different scales, thus obtaining multi-scale ECG image feature data. Based on multi-scale electrocardiogram (ECG) image feature data, local pathological features and global visual features are extracted from the ECG images to obtain local pathological feature data and global visual feature data. Feature mapping is performed between local pathological feature data and global visual feature data to form multi-scale feature space data.

[0008] In a preferred embodiment, S3 specifically refers to: Based on multi-scale feature space data, a global visual feature reward function is established for the reinforcement learning strategy optimization method; A deep Q-network model is used to define the action space of global visual feature data in multi-scale feature space data, and the state transition probability is set according to the Markov decision process. The initial reward weights of the global visual feature reward function are set based on the reinforcement learning strategy optimization method, and the reward values ​​of each augmentation action are calculated under different state transition probabilities. The reward value of each enhancement action is iteratively updated to determine the optimal enhancement action and its corresponding reward value, and global visual feature reward data is generated.

[0009] In a preferred embodiment, S4 specifically refers to: Based on multi-scale feature space data, the local sensitive regions where weak pathological waveforms are located in electrocardiogram images are identified. In the local sensitive area, feature waveforms associated with weak pathological waveform characteristics are selected, and initial reward weights are set for the feature waveforms. By combining the initial reward weight of the feature waveform, the reward weight corresponding to the feature waveform in the local sensitive area is dynamically increased to obtain enhanced reward data for the local sensitive area.

[0010] In a preferred embodiment, S5 specifically refers to: A reward sparsity evaluation mechanism is established based on global visual feature reward data and local sensitive area enhancement reward data; Based on the reward sparsity evaluation mechanism, the reward density of the feature waveform in the enhanced reward data of local sensitive areas is calculated, and the reward density of the feature waveform is compared with a pre-set reward density threshold. Based on the comparison results, determine whether the reward enhancement mechanism has been triggered; Based on the triggering result of the reward enhancement mechanism, the reward weight corresponding to the feature waveform in the local sensitive area is increased to obtain the adjusted comprehensive reward data.

[0011] In a preferred embodiment, S6 specifically refers to: A deep reinforcement learning model is constructed based on the adjusted comprehensive reward data. In the deep reinforcement learning model, the reward objective of the deep reinforcement learning model is defined by the adjusted comprehensive reward data, and an electrocardiogram image enhancement strategy is generated. Based on the electrocardiogram (ECG) image enhancement strategy, determine the action sequence for enhancing standardized ECG image data; Based on the action sequence, corresponding enhancement actions are sequentially performed on the standardized electrocardiogram (ECG) image data to enhance weak pathological waveform features and global visual features in the ECG images, generating enhanced ECG image data.

[0012] On the other hand, the present invention provides an electrocardiogram image enhancement processing system based on deep reinforcement learning, comprising: Data processing module: Collects electrocardiogram (ECG) image data from patients, performs denoising and signal standardization processing to obtain standardized ECG image data; Feature extraction module: Based on standardized electrocardiogram (ECG) image data, a multi-scale convolutional neural network model is constructed to extract local pathological features and global visual features of ECG images, forming multi-scale feature space data; Global reward module: Based on multi-scale feature space data, a reinforcement learning strategy is used to optimize and construct a global visual feature reward function, evaluate the degree of improvement in global visual quality, and output global visual feature reward data; Local reward module: Based on multi-scale feature space data, local sensitive areas of weak pathological waveforms are defined, and a reward sensitivity dynamic adjustment method is adopted to output enhanced reward data for local sensitive areas; Reward adjustment module: Based on global visual feature reward data and local sensitive area enhancement reward data, perform reward sparsity evaluation, determine whether to trigger the reward enhancement mechanism, and output the adjusted comprehensive reward data; Image generation module: Based on the adjusted comprehensive reward data, an ECG image enhancement strategy is generated through a deep reinforcement learning model to generate enhanced ECG image data.

[0013] The technical effects and advantages of the electrocardiogram image enhancement processing method and system based on deep reinforcement learning in this invention are as follows: By performing denoising and signal standardization on patients' electrocardiogram (ECG) image data, the stability and comparability of ECG image input are improved. A multi-scale convolutional neural network model simultaneously extracts local pathological features and global visual features from ECG images, enabling the enhancement process to balance overall visual quality with the expression of key pathological information. By constructing a global visual feature reward function and using reinforcement learning optimization methods, the enhancement strategy continuously converges towards image quality optimization during training. By defining local sensitive regions for weak pathological waveforms and employing a dynamic adjustment method for reward sensitivity, additional reward attention is given to key pathological features. By comprehensively evaluating reward sparsity using global visual feature reward data and enhanced reward data from local sensitive regions, and triggering the reward enhancement mechanism when the triggering conditions are met, the mechanism helps avoid weakening or suppressing rare and weak pathological waveforms with diagnostic value during reinforcement learning optimization. By generating an ECG image enhancement strategy based on the adjusted comprehensive reward data and outputting enhanced ECG image data, the overall visual quality is improved while better preserving and highlighting key clinical pathological features, significantly enhancing the diagnostic reliability and practical value of ECG images. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the electrocardiogram image enhancement processing method based on deep reinforcement learning according to the present invention; Figure 2 This is a schematic diagram of the electrocardiogram image enhancement processing system based on deep reinforcement learning according to the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0016] Example 1

[0017] Figure 1 The present invention provides a method for enhancing electrocardiogram images based on deep reinforcement learning, which includes the following steps: S1: Collect the patient's electrocardiogram (ECG) image data, perform noise reduction and signal standardization processing to obtain standardized ECG image data; S2: Based on standardized electrocardiogram (ECG) image data, construct a multi-scale convolutional neural network model to extract local pathological features and global visual features of ECG images, forming multi-scale feature space data; S3: Based on the multi-scale feature space data, a reinforcement learning strategy is used to optimize the construction of a global visual feature reward function, evaluate the degree of improvement in global visual quality, and output global visual feature reward data; S4: Based on multi-scale feature space data, local sensitive areas of weak pathological waveforms are defined, and a reward sensitivity dynamic adjustment method is adopted to output enhanced reward data for local sensitive areas; S5: Based on the global visual feature reward data and the local sensitive area enhancement reward data, perform reward sparsity evaluation, determine whether the reward enhancement mechanism is triggered, and output the adjusted comprehensive reward data; S6: Based on the adjusted comprehensive reward data, generate an electrocardiogram (ECG) image enhancement strategy through a deep reinforcement learning model to generate enhanced ECG image data.

[0018] S1: Acquire the patient's electrocardiogram (ECG) image data, perform denoising and signal standardization processing to obtain standardized ECG image data, including: Collect electrocardiogram (ECG) images of patients; A 12-lead electrocardiogram (ECG) acquisition device is used to collect raw ECG images of the patient at rest. The twelve leads are fixed to preset positions on the patient's limbs and chest, ensuring good conductivity between the electrodes and the patient's skin to improve the raw quality of the ECG image data. The acquisition process lasts from 30 seconds to 2 minutes; for example, a 1-minute acquisition time is set to ensure complete recording of the patient's cardiac electrical activity in the ECG image data.

[0019] Noise filtering was performed on the patient's electrocardiogram (ECG) image data to remove high-frequency noise and baseline drift interference, resulting in denoised ECG image data. High-frequency noise in electrocardiogram (ECG) image data was filtered using the Butterworth low-pass filter method. The filter order and cutoff frequency of the Butterworth low-pass filter were determined through comparative experiments; for example, the filter order could be set to 4th order and the cutoff frequency to 40Hz. By importing the original ECG image data into the Butterworth low-pass filter, only signal components with frequencies below 40Hz were retained, and high-frequency noise signals with frequencies above 40Hz were filtered out, thus achieving the removal of high-frequency noise.

[0020] For ECG image data processed by high-frequency filtering, wavelet transform was used to remove baseline drift interference. Using the Daubechies wavelet basis function as the basis function, wavelet decomposition was performed on the high-frequency filtered ECG image data. A decomposition scale of four levels was selected, based on comparative experiments showing that a scale of four effectively removed baseline drift without losing pathological information in the ECG image data. After wavelet transform, the wavelet coefficients at scales 1 to 4 were thresholded, and the thresholded wavelet coefficients were then reconstructed using wavelet transform to obtain the denoised ECG image data. A soft thresholding method was used, with the threshold set based on statistical analysis of the baseline drift noise characteristics of a large number of patient samples. The optimal threshold was determined by analyzing the probability distribution characteristics of the wavelet coefficient amplitude; for example, the threshold was set to 15% to 20% of the maximum amplitude of the wavelet coefficients. The ECG image data after high-frequency filtering and wavelet transform thresholding is the denoised ECG image data.

[0021] Signal standardization is performed on the denoised electrocardiogram (ECG) image data to adjust the amplitude range and baseline level of the waveforms in the ECG image data, thereby obtaining standardized ECG image data. The waveform amplitude range of the denoised electrocardiogram (ECG) image data is adjusted. The amplitude range adjustment employs a linear normalization method, statistically analyzing the maximum and minimum amplitude values ​​of all waveforms in the denoised ECG image data, and linearly mapping the waveform amplitude range to a unified standard range, for example, mapping the amplitude range to 0 to 1. The standardized waveform amplitude = (original waveform amplitude - minimum amplitude) / (maximum amplitude - minimum amplitude).

[0022] By calculating the maximum and minimum values ​​of all waveform amplitudes in the current patient's electrocardiogram image data, the standardized waveform amplitude of each waveform is calculated one by one, thereby achieving the standardization of the waveform amplitude range.

[0023] The baseline level of the waveform is adjusted by calculating the average voltage level of the waveform baseline in the electrocardiogram (ECG) image data. The average voltage level of the waveform baseline is defined as the average voltage of the stationary segments outside the QRS complex in each lead of the ECG data. This is achieved by marking multiple stationary segments in the denoised ECG image data, averaging the voltage values ​​of all stationary segments, and then subtracting the calculated average voltage level from the baseline values ​​of each waveform. The ECG image data after linear normalization of the waveform amplitude range and baseline adjustment is then standardized ECG image data.

[0024] S2: Based on standardized electrocardiogram (ECG) image data, construct a multi-scale convolutional neural network model to extract local pathological features and global visual features from the ECG images, forming multi-scale feature space data, including: Based on standardized electrocardiogram (ECG) image data, a multi-scale convolutional neural network model was constructed. Convolution operations were performed on the standardized ECG image data to extract ECG image features at different scales, thus obtaining multi-scale ECG image feature data. The multi-scale convolutional neural network model has a three-branch parallel structure. The kernel sizes of each branch are 3×3, 5×5, and 7×7, respectively. The kernel size selection was based on multiple sets of comparative experiments on standardized electrocardiogram (ECG) image data from a large number of patient samples. The results showed that 3×3, 5×5, and 7×7 kernel sizes can effectively capture local details and global image features at different scales in ECG images. Each convolutional branch includes three convolutional operations, followed by a batch normalization operation and a ReLU activation function. The number of output feature channels for each convolutional operation is set to 32, 64, and 128, respectively. For example, the first convolutional operation uses 32 kernels, the second uses 64 kernels, and the third uses 128 kernels. The above setting of the number of convolutional kernels is based on experimental verification that increasing the number of feature channels by 32, 64, and 128 can most effectively extract multi-scale ECG image features from standardized ECG image data, avoiding overfitting or loss of feature information.

[0025] Standardized electrocardiogram (ECG) image data were input into convolution branches with the three kernel sizes mentioned above, and multi-layer convolution operations were performed to obtain feature maps at different scales. Pooling was then performed on the feature maps output from each branch using max pooling with a 2×2 window size and a stride of 2. This max pooling operation reduced feature dimensionality and improved feature representation. After the three-branch convolution and pooling operations, three feature maps at different scales were obtained. These three feature maps were then concatenated and fused to obtain multi-scale ECG image feature data. The concatenation and fusion method involved directly concatenating the three feature maps along the feature channel dimension.

[0026] Based on multi-scale electrocardiogram (ECG) image feature data, local pathological features and global visual features are extracted from the ECG images to obtain local pathological feature data and global visual feature data. Multi-scale electrocardiogram (ECG) image feature data were input into two feature extraction convolutional sub-networks. The local pathological feature extraction convolutional sub-network consisted of two convolutional layers plus one fully connected layer. Each convolutional layer was followed by batch normalization and a ReLU activation function. The number of output nodes in the fully connected layer was set to 128, determined based on experimental results showing that 128 output nodes best represented and distinguished the local pathological features. The global visual feature extraction convolutional sub-network consisted of three convolutional layers plus one fully connected layer. The structure of the convolutional layers, batch normalization, and ReLU activation function was the same as that of the local pathological feature extraction convolutional sub-network. The number of output nodes in the fully connected layer was set to 256. Experiments showed that this effectively summarized the global visual features of the ECG images. By performing convolution operations and fully connected feature transformations on the multi-scale ECG image feature data using the two feature extraction convolutional networks, local pathological feature data and global visual feature data were obtained, respectively.

[0027] Feature mapping is performed between local pathological feature data and global visual feature data to form multi-scale feature space data; Feature vector standardization was performed on both local pathological feature data and global visual feature data. The standardization method was L2 norm standardization, where each feature vector was divided by its own L2 norm, thus unifying the two feature data sets to the same numerical range. The standardized local pathological feature data and the standardized global visual feature data were then fused using a feature vector concatenation approach, directly concatenating the two standardized feature vectors to form a unified fused feature vector. The fused feature vector has a dimension of 128 + 256 = 384. The dimension was determined based on the number of output nodes of the two feature extraction convolutional sub-networks. Experiments verified that a 384-dimensional fused feature vector yielded the best feature mapping effect, effectively describing ECG image features in a multi-scale feature space.

[0028] S3: Based on multi-scale feature space data, a reinforcement learning strategy is used to construct a global visual feature reward function, evaluate the degree of improvement in global visual quality, and output global visual feature reward data, including: Based on multi-scale feature space data, a global visual feature reward function is established for the reinforcement learning strategy optimization method; The global visual feature reward function is used to quantitatively evaluate the overall visual performance of electrocardiogram (ECG) images resulting from enhanced actions. Global visual performance includes, but is not limited to, background smoothness, waveform continuity, signal-texture consistency, and overall visual noise level. The global visual feature reward function is mathematically defined as a weighted summation operation using the global visual feature dimension of the fused feature vector. The reward function value equals the weighted result of the global visual features. The weight coefficients in the reward function are set by performing feature contribution analysis using standardized ECG image data from 40 patients. A feature-by-feature perturbation experiment is used to statistically analyze the impact of changes in each visual feature on the overall visual performance of the image. Weight coefficients are then assigned to each visual feature according to their contribution ratio. For example, the weight for visual smoothness can be set to 0.3, the weight for texture consistency to 0.25, the weight for background visual noise suppression to 0.2, and the weight for waveform continuity to 0.25. The sum of the weights equals 1 and can be dynamically adjusted based on the statistical characteristics of different patient samples.

[0029] A deep Q-network model is used to define the action space of global visual feature data in multi-scale feature space data, and the state transition probability is set according to the Markov decision process. The input to the Deep Q-Network model is defined as multi-scale feature space data with a dimension of 384. The Deep Q-Network model consists of three fully connected layers, each containing the ReLU activation function. The first fully connected layer has 256 nodes, the second has 128 nodes, and the third has 64 nodes. The node count was determined through experimental analysis of the model's ability to fit the reward function of augmentation actions under different node sizes. The 256-128-64 layer structure was found to effectively fit the reward function variation trend of ECG image augmentation actions. The output layer generates the action value function Q-value, and the number of output nodes equals the number of augmentation actions in the action space. For example, when the augmentation action space includes four types of actions: augmentation amplitude increase, augmentation amplitude decrease, local augmentation, and global augmentation, the number of output nodes is 4. The Deep Q-Network model determines the optimal augmentation action for the current multi-scale feature space data by calculating the action value function Q-value corresponding to each augmentation action.

[0030] A state is defined as the changed state of multi-scale feature space data, i.e., the new multi-scale feature space data generated after the enhancement action is performed. The method for setting the state transition probability is as follows: standardized electrocardiogram image data of 30 randomly sampled patients is used as the state transition experimental dataset. Different enhancement actions are performed, and the trend of feature value changes caused by each enhancement action is statistically analyzed. For example, if the probability that an action increasing the enhancement amplitude leads to an improvement in the smoothness feature in the global visual features is 0.65, then the probability value at the corresponding position in the state transition probability matrix is ​​set to 0.65. Each action is assigned a corresponding state change probability, thereby constructing a complete Markov state transition model.

[0031] The initial reward weights of the global visual feature reward function are set based on the reinforcement learning strategy optimization method, and the reward values ​​of each augmentation action are calculated under different state transition probabilities. Global visual feature data from the fused feature vector is input into the reward function, and the reward value is calculated using a weighted average. The reward value is mathematically expressed as the sum of the product of each visual feature and its respective weight. The reward function value is then combined with the state transition probability matrix of each augmented action to calculate the expected reward value for each augmented action in different states. Finally, the expected reward for each action is obtained by multiplying the reward function value by the state transition probability corresponding to that action.

[0032] The reward value of each enhancement action is iteratively updated to determine the optimal enhancement action and its corresponding reward value, and global visual feature reward data is generated. The difference between the action value function Q-value of the current state and the target action value function Q-value is used as the loss function input to the deep Q-network model. The weight parameters in each layer of the deep Q-network model are adjusted through backpropagation to make the action value function Q-value approximate the expected reward value. The target action value function Q-value is set by adding a discount factor to the reward function value and multiplying it by the maximum action value function Q-value of the next state. For example, the discount factor can be set to 0.9 to balance the relationship between the current reward and the future reward. The iterative update process continues until the loss function stabilizes within a preset threshold; for example, training stops when the loss function is less than 0.001. After multiple rounds of iterative training, the deep Q-network model can output the optimal augmented action and its corresponding reward value given the multi-scale feature space data. The optimal augmented action is defined as the action with the maximum action value function Q-value among all augmented actions, and the corresponding reward value is the global visual feature reward data.

[0033] S4: Based on multi-scale feature space data, define local sensitive regions of weak pathological waveforms, and use a dynamic adjustment method for reward sensitivity to output enhanced reward data for these local sensitive regions, including: Based on multi-scale feature space data, the local sensitive regions where weak pathological waveforms are located in electrocardiogram images are identified. Multi-scale feature space data is input into the local sensitive area localization program. The multi-scale feature space data is a fused feature vector with a length of 384 dimensions, containing local pathological feature data and global visual feature data. Feature dimension analysis is performed on the fused feature vector. Through experimental analysis of multi-scale feature space data from 40 patients, the range of feature dimensions corresponding to local pathological feature data is determined. Local pathological feature data occupies the first 128 dimensions of the fused feature vector. By analyzing the correlation between each dimension feature and weak pathological waveforms, feature dimensions with a high degree of correlation with pathological waveforms are determined. Correlation analysis calculates the correlation coefficient between each feature dimension in the fused feature vector and the weak waveform markers in the electrocardiogram that have been clinically diagnosed as pathological states. Feature dimensions with correlation coefficients greater than a set threshold are defined as relevant feature dimensions for local sensitive areas. For example, through experimental verification, when the Pearson correlation coefficient is greater than 0.7, it can be identified as a highly correlated feature dimension.

[0034] Based on the defined highly correlated feature dimensions, spatial location mapping is performed on waveforms in standardized electrocardiogram (ECG) image data to determine the spatial location of weak pathological waveforms, thereby identifying local sensitive regions. The spatial location mapping method involves inversely mapping the spatial location corresponding to each feature dimension in the feature map of the last layer of the multi-scale convolutional neural network model onto the original standardized ECG image data, obtaining the feature response region in two-dimensional image coordinate space, and defining it as a local sensitive region. For example, the spatial coordinates of the feature response region can be determined by the location of the maximum response in the feature map of the multi-scale convolutional neural network model. When the maximum response of a certain feature dimension in the feature map is located at coordinates (x=120, y=240), the local sensitive region is defined with that coordinate as the center. The region size can be set to 32×32 pixels, and the method for determining the region size is based on comparative experiments with different sizes.

[0035] In the local sensitive area, feature waveforms associated with weak pathological waveform characteristics are selected, and initial reward weights are set for the feature waveforms. Feature waveforms are selected from the identified local sensitive regions using a pathological waveform feature matching method. This method is a template matching algorithm. A pre-prepared pathological waveform template library contains typical weak pathological waveforms confirmed by clinical diagnosis, such as small Q waves, slight ST segment depression or elevation, and weak T wave inversion. A sliding window iterates through all waveforms within the local sensitive region, calculating the cosine similarity between each waveform and all templates in the library. Waveforms with a similarity greater than a set threshold are selected as feature waveforms. For example, experiments have shown that a cosine similarity greater than 0.85 is considered a feature waveform.

[0036] For each identified feature waveform, an initial reward weight is set. The initial reward weight is determined by assigning a weight based on the cosine similarity between the feature waveform and the pathological waveform template. The higher the cosine similarity, the greater the initial reward weight. For example, when the cosine similarity is 0.85, the initial reward weight is set to 0.5; when the cosine similarity is 0.95, the initial reward weight is set to 0.8. The initial reward weight is set using linear interpolation, i.e., the initial reward weight equals (cosine similarity − 0.8) × 5.

[0037] By combining the initial reward weight of the feature waveform, the reward weight corresponding to the feature waveform in the local sensitive area is dynamically increased to obtain enhanced reward data for the local sensitive area. The dynamic adjustment method is as follows: The sparsity of the feature waveform is dynamically adjusted based on its frequency of occurrence during training. Feature waveform sparsity is defined as the ratio of the number of times a feature waveform appears in the entire patient training sample set to the total number of samples. When the sparsity of the feature waveform is lower than a preset sparsity threshold, the corresponding reward weight is dynamically increased. The sparsity threshold is set by statistically analyzing the frequency of feature waveforms in the training data of 40 patients. When the frequency of a feature waveform is less than 10%, i.e., the sparsity is less than 0.1, the reward weight needs to be dynamically increased. For example, when the sparsity of the feature waveform is 0.05, the dynamic increase in reward weight is 50%. The formula for setting this dynamic increase in reward weight is: Dynamic reward weight = initial reward weight of feature waveform × (1 + (0.1 − sparsity of feature waveform) × 10). The new dynamic reward weight is calculated using the above formula. The lower the sparsity, the greater the increase in reward weight.

[0038] Dynamic reward weights are calculated for all selected feature waveforms to obtain a dynamically adjusted set of reward weights for all feature waveforms. This set of dynamic reward weights for all feature waveforms is defined as the local sensitive area enhancement reward data. The dimension of the local sensitive area enhancement reward data is equal to the number of feature waveforms, and the data format is an array, where each array element represents the dynamic reward weight of the corresponding feature waveform.

[0039] S5: Based on the global visual feature reward data and the local sensitive region enhancement reward data, perform reward sparsity evaluation, determine whether the reward enhancement mechanism is triggered, and output the adjusted comprehensive reward data, including: A reward sparsity evaluation mechanism is established based on global visual feature reward data and local sensitive area enhancement reward data; The reward density of feature waveforms in enhanced reward data for locally sensitive regions is defined. Feature waveform reward density is defined as the proportion of feature waveforms whose reward weight exceeds a preset reward density statistical threshold among all feature waveforms within a locally sensitive region. The formula for calculating feature waveform reward density is: Feature waveform reward density = (Number of feature waveforms with reward weight greater than the reward density statistical threshold) / (Total number of feature waveforms in the locally sensitive region). The range of feature waveform reward density is between 0 and 1; a smaller feature waveform reward density indicates that high-reward feature waveforms are sparser within the locally sensitive region. The method for setting the reward density statistical threshold is to analyze the statistical relationship between the clinical diagnosis results of weak pathological waveforms in standardized electrocardiogram image data of 50 patients and enhanced reward data to determine the reward density statistical threshold that can effectively distinguish between high-reward and low-reward feature waveforms. For example, if extensive clinical statistical analysis shows that a reward density statistical threshold of 0.6 can effectively distinguish clinically important pathological waveforms from general waveforms, then the reward density statistical threshold is set to 0.6.

[0040] Based on the reward sparsity evaluation mechanism, the reward density of the feature waveform in the enhanced reward data of local sensitive areas is calculated, and the reward density of the feature waveform is compared with a pre-set reward density threshold. Using enhanced reward data from localized sensitive regions as input, the algorithm iterates through each feature waveform within the localized sensitive region, counting the number of feature waveforms with reward weights greater than the reward density statistical threshold. For example, assuming there are 20 feature waveforms in the current patient's localized sensitive region, if the calculation finds that 4 feature waveforms have reward weights greater than the reward density statistical threshold of 0.6, then the calculated reward density of the current localized sensitive region's feature waveforms is 0.2, which is the reward density of the current localized sensitive region's feature waveforms.

[0041] The calculated reward density of the characteristic waveform is compared with a pre-set reward density threshold. The reward density threshold is determined by distinguishing whether the reward density of the characteristic waveform in a locally sensitive region is sufficiently sparse, thus deciding whether to increase the reward weight corresponding to the characteristic waveform. The reward density threshold is determined through analysis of standardized electrocardiogram image data from 100 patients. The method involves statistically analyzing the relationship curve between the diagnostic accuracy of weak pathological waveforms and the reward density of the characteristic waveform, and determining the optimal cut-off point for the reward density threshold through receiver operating characteristic (ROC) curve analysis. For example, ROC curve analysis shows that when the reward density threshold is set to 0.3, sensitivity and specificity achieve the best balance; therefore, the reward density threshold is set to 0.3. When the calculated reward density of the characteristic waveform is lower than 0.3, it indicates that the current reward sparsity is significant, and a reward enhancement mechanism should be triggered.

[0042] Based on the comparison results, determine whether the reward enhancement mechanism has been triggered; If the feature waveform reward density of a local sensitive region is lower than the reward density threshold, it is determined that the overall reward weight of the feature waveform in the current local sensitive region is too low, and the corresponding reward weight needs to be increased. If the feature waveform reward density of a local sensitive region is higher than or equal to the reward density threshold, it is determined that the current reward weight distribution is reasonable, and no reward enhancement is needed. For example, if the calculated feature waveform reward density is 0.2, since 0.2 is lower than the reward density threshold of 0.3, the reward enhancement mechanism is triggered.

[0043] Based on the triggering result of the reward enhancement mechanism, the reward weight corresponding to the feature waveform in the local sensitive area is increased to obtain the adjusted comprehensive reward data; The reward enhancement mechanism is implemented as follows: for feature waveforms where the current reward weight is greater than the reward density statistical threshold, the reward weight is increased; the increase in reward weight is calculated using an empirical formula. The formula for calculating the increase in reward weight is defined as follows: Increase in reward weight = Original reward weight × (1 + (Reward density threshold - Feature waveform reward density) × Reward adjustment coefficient).

[0044] The reward adjustment coefficient was determined by analyzing data from 50 patients to statistically determine the correlation between the increase in reward weight and the enhancement effect of clinical pathological waveforms, thus identifying the optimal reward adjustment coefficient. For example, experimental verification showed that when the reward adjustment coefficient was 2.0, increasing the reward weight of the feature waveform significantly improved the enhancement effect of the clinical pathological waveform; therefore, the reward adjustment coefficient was set to 2.0. Based on the above formula, taking a feature waveform with an original reward weight of 0.65 as an example, the increase in its reward weight was calculated as: Reward weight increase = 0.65 × (1 + (0.3 - 0.2) × 2.0) = 0.78. Thus, the reward weight corresponding to this feature waveform increased from the original value of 0.65 to the adjusted 0.78. The same method was used to increase the reward weight of all eligible feature waveforms within the local sensitive area, thereby obtaining the complete adjusted comprehensive reward data. The adjusted comprehensive reward data and the original local sensitive area enhancement reward data are a one-dimensional array of the same dimension, with each array element representing the adjusted reward weight of the corresponding feature waveform.

[0045] S6: Based on the adjusted comprehensive reward data, generate an electrocardiogram (ECG) image enhancement strategy using a deep reinforcement learning model, generating enhanced ECG image data, including: A deep reinforcement learning model is constructed based on the adjusted comprehensive reward data. The deep reinforcement learning model is a deep deterministic policy gradient reinforcement learning model, which includes two independent neural network structures: a policy function and a value function. The policy function is used to generate the sequence of reinforcement actions, and the value function is used to evaluate the value of the sequence of reinforcement actions. The policy function network structure includes an input layer, hidden layers, and an output layer. The number of nodes in the input layer is equal to the dimension of the adjusted comprehensive reward data, i.e., the same as the number of feature waveforms. The hidden layer consists of three fully connected layers, each using rectified linear units as the activation function, with 256, 128, and 64 nodes respectively. The number of nodes was determined by using standardized electrocardiogram images from 50 patients as training samples and conducting comparative experiments to determine the enhancement effect of the policy function under different node configurations. Experiments verified that setting the number of nodes in the three hidden layers to 256, 128, and 64 respectively achieves the optimal enhancement policy generation effect. The number of nodes in the policy function output layer is equal to the number of actions in the enhancement action space. The number of actions is determined by analyzing reasonable classification methods for enhancement action types. For example, if the action space includes four types of actions: enhancement amplitude increase, enhancement amplitude decrease, local enhancement, and global enhancement, then the number of output nodes is set to 4. The output layer uses the softmax function as the activation function and outputs the execution probability distribution of each enhancement action. The value function network structure includes an input layer, hidden layers, and an output layer. The number of nodes in the input layer is the same as the dimension of the adjusted comprehensive reward data, i.e., the number of feature waveforms. The hidden layer structure also includes a three-layer fully connected network, with each layer using rectified linear units as the activation function. The number of nodes is set in the same way as the policy function network, with 256, 128, and 64 nodes respectively. The output layer has one node, which is used to output the value function estimate of the current augmented action sequence.

[0046] In the deep reinforcement learning model, the reward objective of the deep reinforcement learning model is defined by the adjusted comprehensive reward data, and an electrocardiogram image enhancement strategy is generated. The adjusted comprehensive reward data is used as input to the policy function and value function. The training objective of the model is defined by the loss function of reinforcement learning. The loss function definition includes policy loss and value loss. The policy loss is the negative correlation between the action probability distribution output by the policy function and the value function estimate of the augmented action sequence output by the value function. That is, the policy function needs to output the augmented action sequence that maximizes the value function output value. The mathematical expression of the policy loss function is defined as follows: Strategy loss function = -E[value function(strategy function(adjusted overall reward data))].

[0047] Where E represents the expected operation; the value loss function is defined as the mean squared error between the estimated value function output by the deep reinforcement learning model and the target value function, and the target value function is defined as the weighted sum between the adjusted comprehensive reward data and the state reward after the action is executed, i.e.: Target value function = Current state reward value + γ × Next state reward value; Where γ is the discount coefficient, the optimal value of γ is determined by conducting an enhancement experiment on standardized electrocardiogram image data of 30 patients, statistically analyzing the balance between long-term and short-term rewards. For example, the experiment determined that the enhancement strategy is most effective when γ is 0.9. The mathematical expression of the value loss function is then defined as: Value loss function = E[(value function estimate - target value function)^2].

[0048] During model training, the loss function is used as the objective function of the deep reinforcement learning model. Gradient descent is used to iteratively update the network parameters in the policy function and value function. The learning rate setting method for gradient descent is as follows: a comparative experiment was conducted using data from 40 patients. Experiments verified that the loss function of the reinforcement learning model training had the best convergence effect when the learning rate was set to 0.001. Therefore, the learning rate was set to 0.001. The training process continued until the loss function value stably converged to a preset threshold. For example, training was stopped when both the policy loss function and the value loss function were stably less than 0.001. Finally, the trained deep reinforcement learning model was obtained, and an electrocardiogram image enhancement strategy was generated through the trained deep reinforcement learning model.

[0049] Based on the electrocardiogram (ECG) image enhancement strategy, determine the action sequence for enhancing standardized ECG image data; Standardized electrocardiogram (ECG) images of patients are input into a trained policy function. The policy function outputs a probability distribution of enhanced actions. This probability distribution is processed using a greedy strategy, defined as selecting the action sequence with the highest probability, i.e., choosing the enhanced action with the highest probability value as the current action sequence. The greedy strategy is implemented by sorting the action probability distribution output by the policy function and selecting the enhanced action sequence with the highest probability value. For example, if the current policy function outputs probabilities of increased enhancement amplitude, decreased enhancement amplitude, local enhancement, and global enhancement of 0.2, 0.1, 0.6, and 0.1 respectively, then local enhancement is selected as the current action sequence. This action selection process is repeated for each input standardized ECG image to obtain a complete enhanced action sequence.

[0050] Based on the action sequence, corresponding enhancement actions are sequentially performed on the standardized electrocardiogram image data to enhance weak pathological waveform features and global visual features in the electrocardiogram image, generating enhanced electrocardiogram image data. Based on a defined action sequence, corresponding enhancement actions are executed. For example, for local enhancement actions, the location of the local sensitive region corresponding to the action is first determined. According to the specified amplitude of local enhancement in the action sequence, an adaptive local histogram equalization algorithm is used to enhance the waveform amplitude of the local sensitive region. First, the distribution histogram of grayscale values ​​of the electrocardiogram image data in the local sensitive region is statistically analyzed. Based on the distribution histogram, adaptive local mapping is performed on the waveform amplitude data. The local mapping function is determined through experimental verification. For example, when the local mapping function is a Gaussian adaptive mapping function, the contrast and detail expression of the enhanced waveform are optimal. The Gaussian function parameters of the Gaussian adaptive mapping function are adjusted according to the enhancement amplitude defined in the action sequence, thereby realizing the control of local enhancement and completing the enhancement processing of the local sensitive region.

[0051] For global enhancement actions, a global contrast enhancement algorithm is used. Gray-level histogram statistics are performed on the standardized ECG image data, and a global histogram equalization method is used to map the overall gray-level values. The parameters of the mapping function are dynamically adjusted based on the enhancement amplitude defined by the action sequence. For example, when the enhancement amplitude defined by the action sequence is 0.8, the mapping function parameter is also 0.8. Experiments have shown that the enhancement effect is optimal when the mapping function parameter is between 0.6 and 0.9. Therefore, a value of 0.8 is used for the enhancement process, thereby achieving overall enhancement of the global visual features of the ECG image.

[0052] All enhancement actions defined in the action sequence are sequentially executed on the standardized ECG image data, ultimately generating enhanced ECG image data. The enhanced ECG image data is a two-dimensional image matrix, with the same dimensions as the original standardized ECG image data. For example, if the original standardized ECG image data has a dimension of 512×512 pixels, then the enhanced ECG image data will also have a dimension of 512×512 pixels.

[0053] Example 2

[0054] The difference between Embodiment 2 and Embodiment 1 is that this embodiment introduces an electrocardiogram image enhancement processing system based on deep reinforcement learning.

[0055] Figure 2 A schematic diagram of the electrocardiogram (ECG) image enhancement processing system based on deep reinforcement learning of the present invention is provided. The ECG image enhancement processing system based on deep reinforcement learning includes: Data processing module: Collects electrocardiogram (ECG) image data from patients, performs denoising and signal standardization processing to obtain standardized ECG image data; Feature extraction module: Based on standardized electrocardiogram (ECG) image data, a multi-scale convolutional neural network model is constructed to extract local pathological features and global visual features of ECG images, forming multi-scale feature space data; Global reward module: Based on multi-scale feature space data, a reinforcement learning strategy is used to optimize and construct a global visual feature reward function, evaluate the degree of improvement in global visual quality, and output global visual feature reward data; Local reward module: Based on multi-scale feature space data, local sensitive areas of weak pathological waveforms are defined, and a reward sensitivity dynamic adjustment method is adopted to output enhanced reward data for local sensitive areas; Reward adjustment module: Based on global visual feature reward data and local sensitive area enhancement reward data, perform reward sparsity evaluation, determine whether to trigger the reward enhancement mechanism, and output the adjusted comprehensive reward data; Image generation module: Based on the adjusted comprehensive reward data, an ECG image enhancement strategy is generated through a deep reinforcement learning model to generate enhanced ECG image data.

[0056] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0057] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0058] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0059] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0060] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0061] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0062] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0063] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0064] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A deep reinforcement learning-based electrocardiogram image enhancement processing method, characterized in that, Includes the following steps: S1: Collect the patient's electrocardiogram (ECG) image data, perform noise reduction and signal standardization processing to obtain standardized ECG image data; S2: Based on standardized electrocardiogram (ECG) image data, construct a multi-scale convolutional neural network model to extract local pathological features and global visual features of ECG images, forming multi-scale feature space data; S3: Based on the multi-scale feature space data, a reinforcement learning strategy is used to optimize and construct a global visual feature reward function, evaluate the degree of improvement in global visual quality, and output global visual feature reward data; S4: Based on multi-scale feature space data, local sensitive areas of weak pathological waveforms are defined, and a reward sensitivity dynamic adjustment method is adopted to output enhanced reward data for local sensitive areas; S5: Based on the global visual feature reward data and the local sensitive area enhancement reward data, perform reward sparsity evaluation, determine whether the reward enhancement mechanism is triggered, and output the adjusted comprehensive reward data; S6: Based on the adjusted comprehensive reward data, generate an electrocardiogram image enhancement strategy through a deep reinforcement learning model to generate enhanced electrocardiogram image data.

2. The deep reinforcement learning-based electrocardiogram image enhancement processing method according to claim 1, characterized in that, S1, specifically: Collect electrocardiogram (ECG) images of patients; Noise filtering was performed on the patient's electrocardiogram (ECG) image data to remove high-frequency noise and baseline drift interference, resulting in denoised ECG image data. Signal standardization is performed on the denoised electrocardiogram (ECG) image data to adjust the amplitude range and baseline level of the waveforms in the ECG image data, thereby obtaining standardized ECG image data.

3. The electrocardiogram image enhancement processing method based on deep reinforcement learning according to claim 2, characterized in that, S2, specifically: Based on standardized electrocardiogram (ECG) image data, a multi-scale convolutional neural network model was constructed. Convolution operations were performed on the standardized ECG image data to extract ECG image features at different scales, thus obtaining multi-scale ECG image feature data. Based on multi-scale electrocardiogram (ECG) image feature data, local pathological features and global visual features are extracted from ECG images to obtain local pathological feature data and global visual feature data. Feature mapping is performed between local pathological feature data and global visual feature data to form multi-scale feature space data.

4. The electrocardiogram image enhancement processing method based on deep reinforcement learning according to claim 3, characterized in that, S3, specifically: Based on multi-scale feature space data, a global visual feature reward function is established for the reinforcement learning strategy optimization method; A deep Q-network model is used to define the action space of global visual feature data in multi-scale feature space data, and the state transition probability is set according to the Markov decision process. The initial reward weights of the global visual feature reward function are set based on the reinforcement learning strategy optimization method, and the reward values ​​of each augmentation action are calculated under different state transition probabilities. The reward value of each enhancement action is iteratively updated to determine the optimal enhancement action and its corresponding reward value, and global visual feature reward data is generated.

5. The electrocardiogram image enhancement processing method based on deep reinforcement learning according to claim 4, characterized in that, S4, specifically: Based on multi-scale feature space data, the local sensitive regions where weak pathological waveforms are located in electrocardiogram images are identified. In the local sensitive area, feature waveforms associated with weak pathological waveform characteristics are selected, and initial reward weights are set for the feature waveforms. By combining the initial reward weight of the feature waveform, the reward weight corresponding to the feature waveform in the local sensitive area is dynamically increased to obtain enhanced reward data for the local sensitive area.

6. The electrocardiogram image enhancement processing method based on deep reinforcement learning according to claim 5, characterized in that, S5, specifically: A reward sparsity evaluation mechanism is established based on global visual feature reward data and local sensitive area enhancement reward data; Based on the reward sparsity evaluation mechanism, the reward density of the feature waveform in the enhanced reward data of local sensitive areas is calculated, and the reward density of the feature waveform is compared with a pre-set reward density threshold. Based on the comparison results, determine whether the reward enhancement mechanism has been triggered; Based on the triggering result of the reward enhancement mechanism, the reward weight corresponding to the feature waveform in the local sensitive area is increased to obtain the adjusted comprehensive reward data.

7. The electrocardiogram image enhancement processing method based on deep reinforcement learning according to claim 6, characterized in that, S6, specifically: A deep reinforcement learning model is constructed based on the adjusted comprehensive reward data. In the deep reinforcement learning model, the reward objective of the deep reinforcement learning model is defined by the adjusted comprehensive reward data, and an electrocardiogram image enhancement strategy is generated. Based on the electrocardiogram (ECG) image enhancement strategy, determine the action sequence for enhancing standardized ECG image data; Based on the action sequence, corresponding enhancement actions are sequentially performed on the standardized electrocardiogram (ECG) image data to enhance weak pathological waveform features and global visual features in the ECG images, generating enhanced ECG image data.

8. A deep reinforcement learning-based electrocardiogram (ECG) image enhancement processing system, used to implement the deep reinforcement learning-based ECG image enhancement processing method according to any one of claims 1-7, characterized in that, include: Data processing module: Collects electrocardiogram (ECG) image data from patients, performs denoising and signal standardization processing to obtain standardized ECG image data; Feature extraction module: Based on standardized electrocardiogram (ECG) image data, a multi-scale convolutional neural network model is constructed to extract local pathological features and global visual features of ECG images, forming multi-scale feature space data; Global reward module: Based on multi-scale feature space data, a reinforcement learning strategy is used to optimize and construct a global visual feature reward function, evaluate the degree of improvement in global visual quality, and output global visual feature reward data; Local reward module: Based on multi-scale feature space data, local sensitive areas of weak pathological waveforms are defined, and a reward sensitivity dynamic adjustment method is adopted to output enhanced reward data for local sensitive areas; Reward adjustment module: Based on global visual feature reward data and local sensitive area enhancement reward data, perform reward sparsity evaluation, determine whether to trigger the reward enhancement mechanism, and output the adjusted comprehensive reward data; Image generation module: Based on the adjusted comprehensive reward data, an ECG image enhancement strategy is generated through a deep reinforcement learning model to generate enhanced ECG image data.

Images