Respiratory rhythm recommendation method based on multi-modal data fusion and related device
The respiratory rhythm recommendation system, optimized through multimodal data fusion and reinforcement learning algorithms, solves the problem of unstable respiratory rhythm in CTA examinations, achieving more efficient respiratory control and improved image quality.
Patent Information
- Application Number
- CN202510936953.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-08
- Publication Date
- 2025-10-17
AI Technical Summary
In existing technologies, the stability of the recommended breathing rhythm for users during CTA examinations is low, mainly due to reliance on human prompts, which leads to inconsistent breathing rhythms.
By fusing multimodal data, the system monitors users' ECG and respiratory parameters in real time, uses reinforcement learning algorithms to optimize recommendation models, generates voice or visual guidance, realizes a closed-loop control system, and optimizes breathing behavior.
It improved the stability of respiratory rhythm recommendations and CTA imaging results, thereby enhancing image stability and diagnostic accuracy.
Smart Images
Figure CN120809158A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of data processing and machine learning, in particular to a respiratory rhythm recommendation method based on multi-modal data fusion and related devices. BACKGROUND
[0002] When a user is performing a CTA examination, a stable respiratory rhythm needs to be maintained to improve the stability and effect of the images collected during the examination. In existing solutions, the user is usually prompted to perform respiratory rhythm by a person, which is subject to the experience of the person during processing, resulting in low stability when recommending a respiratory rhythm. SUMMARY
[0003] The present application provides a respiratory rhythm recommendation method based on multi-modal data fusion and related devices, which can perform state evaluation according to the electrocardiogram signal of a user to be detected, and in a heart rate stable state, use the obtained respiratory rhythm evaluation parameter to predict respiratory rhythm information, obtain target respiratory rhythm information and push it to the user to be detected, thereby improving the stability of recommending target respiratory rhythm information.
[0004] Unlike the traditional method of relying only on artificial voice prompts, the present system continuously monitors the multi-modal physiological parameters of the user, such as electrocardiogram and respiration, dynamically evaluates whether the user is in a suitable guiding state, and at the right time when the heart rate is stable and the respiratory rhythm is controllable, generates guiding rhythm information (such as voice prompts, visual animations, beat signals, etc.) to guide the patient to hold their breath. Subsequently, the system synchronously collects the user's execution results (such as breath-holding response time, duration, cooperation accuracy, etc.), and uses them as training signals to input into the guiding strategy update module, and continuously optimizes the recommendation model using reinforcement learning algorithm, thereby constructing a closed-loop control system of "observation-analysis-guidance-feedback", and finally realizing intelligent intervention on respiratory behavior and optimization of coronary CTA imaging effect.
[0005] The first aspect of the present application provides a respiratory rhythm recommendation method based on multi-modal data fusion, which comprises: obtaining the electrocardiogram signal of a user to be detected during a CTA examination; performing heart rate state evaluation according to the electrocardiogram signal to obtain a heart rate state evaluation result; if the heart rate state evaluation result indicates that the user to be detected is in a heart rate stable state, obtaining a respiratory rhythm evaluation parameter of the user to be detected; inputting the respiratory rhythm evaluation parameter into a target respiratory rhythm recommendation model for operation to obtain target respiratory rhythm information; pushing the target respiratory rhythm information to the user to be detected.
[0006] In a possible implementation, the heart rate state evaluation according to the electrocardiosignal comprises: extracting an R-wave heart rate sequence from the electrocardiosignal to obtain a target R-wave heart rate time sequence; performing frequency domain transformation on the target R-wave heart rate time sequence to obtain a target R-wave heart rate frequency domain sequence; performing low-frequency power value calculation according to the target R-wave heart rate frequency domain sequence to obtain a first power value, and performing high-frequency power value calculation according to the target R-wave heart rate frequency domain sequence to obtain a second power value; determining a state evaluation parameter according to the first power value and the second power value; performing heart rate state evaluation according to the state evaluation parameter to obtain a heart rate state evaluation result.
[0007] In a possible implementation, the inputting of the respiration rhythm evaluation parameter into the target respiration rhythm recommendation model for operation to obtain target respiration rhythm information comprises: performing normalization processing on the respiration rhythm evaluation parameter to obtain a normalized respiration rhythm evaluation parameter; performing vector construction according to the normalized respiration rhythm evaluation parameter to obtain a model input vector; inputting the model input vector into the target respiration rhythm recommendation model for operation to obtain the target respiration rhythm information.
[0008] In a possible implementation, the objective function of the target respiration rhythm recommendation model is represented by the following formula: wherein, is a loss function, is a model complexity penalty.
[0009] In a possible implementation, the method further comprises obtaining a target respiration rhythm recommendation model; the obtaining of the target respiration rhythm recommendation model comprises: adjusting an initial respiration rhythm recommendation model by using a policy gradient reinforcement learning mechanism to obtain the target respiration rhythm recommendation model; the expected equation of the policy gradient reinforcement learning mechanism is represented by the following formula: ; wherein, is a current patient physiological state, is a current guide policy, is a feedback score of this training, is a future return discount factor.
[0010] A second aspect of the embodiment of the present application provides a respiratory rhythm recommendation device based on multi-modal data fusion, the device comprising: a first acquisition unit configured to acquire an electrocardiosignal of a user to be detected when a CTA examination is performed; an evaluation unit configured to perform heart rate state evaluation according to the electrocardiosignal to obtain a heart rate state evaluation result; a second acquisition unit configured to acquire a respiratory rhythm evaluation parameter of the user to be detected if the heart rate state evaluation result indicates that the user to be detected is in a heart rate stable state; an operation unit configured to input the respiratory rhythm evaluation parameter into a target respiratory rhythm recommendation model to perform operation to obtain target respiratory rhythm information; a pushing unit configured to push the target respiratory rhythm information to the user to be detected.
[0011] a guiding unit configured to generate a voice or image prompt signal according to the target respiratory rhythm information to guide the user to be detected to perform respiratory breath-hold cooperation according to the guided rhythm; a feedback acquisition unit configured to acquire a response feature and a cooperation effect of the user during execution of the guided process to form a feedback parameter; and a strategy updating unit configured to input the feedback parameter into a strategy optimization module to realize continuous optimization of the guided scheme.
[0012] In one possible implementation, the evaluation unit is specifically configured to: extract an R-wave heart rate sequence from the electrocardiosignal to obtain a target R-wave heart rate time sequence; perform frequency domain transformation on the target R-wave heart rate time sequence to obtain a target R-wave heart rate frequency domain sequence; perform low-frequency power value calculation according to the target R-wave heart rate frequency domain sequence to obtain a first power value, and perform high-frequency power value calculation according to the target R-wave heart rate frequency domain sequence to obtain a second power value; determine a state evaluation parameter according to the first power value and the second power value; perform heart rate state evaluation according to the state evaluation parameter to obtain a heart rate state evaluation result.
[0013] In one possible implementation, the operation unit is specifically configured to: perform normalization processing on the respiratory rhythm evaluation parameter to obtain a normalized respiratory rhythm evaluation parameter; perform vector construction according to the normalized respiratory rhythm evaluation parameter to obtain a model input vector; input the model input vector into the target respiratory rhythm recommendation model to perform operation to obtain the target respiratory rhythm information.
[0014] In a possible implementation, the objective function of the target respiratory rhythm recommendation model is characterized by the following formula: wherein, is a loss function, is a model complexity penalty.
[0015] In a possible implementation, the apparatus is further configured to obtain a target respiratory rhythm recommendation model. In the aspect of obtaining the target respiratory rhythm recommendation model, the apparatus is further configured to: adjust the initial respiratory rhythm recommendation model by using a policy gradient reinforcement learning mechanism to obtain the target respiratory rhythm recommendation model. The expected equation of the policy gradient reinforcement learning mechanism is characterized by the following formula: wherein, is a current physiological state of a patient, is a current guidance policy, is a feedback score of this training, is a future reward discount factor.
[0016] A third aspect of the embodiment of the present application provides a terminal, comprising a processor, an input device, an output device and a memory, which are connected to each other, wherein the memory is configured to store a computer program, the computer program comprises program instructions, and the processor is configured to invoke the program instructions to execute the step instructions as in the first aspect of the embodiment of the present application.
[0017] A fourth aspect of the embodiment of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program for electronic data exchange, and the computer program causes a computer to execute some or all of the steps described in the first aspect of the embodiment of the present application.
[0018] A fifth aspect of the embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in the first aspect of the embodiment of the present application. The computer program product can be a software installation package.
[0019] The embodiment of the present application has the following beneficial effects: By acquiring the electrocardiosignal of the to-be-detected user when the CTA examination is performed, heart rate state evaluation is performed according to the electrocardiosignal, and a heart rate state evaluation result is obtained; if the heart rate state evaluation result indicates that the to-be-detected user is in a heart rate stable state, a respiration rhythm evaluation parameter of the to-be-detected user is acquired, the respiration rhythm evaluation parameter is input into a target respiration rhythm recommendation model for operation, target respiration rhythm information is obtained, and the target respiration rhythm information is pushed to the to-be-detected user. Therefore, state evaluation can be performed according to the electrocardiosignal of the to-be-detected user, when the heart rate is stable, the acquired respiration rhythm evaluation parameter is used to predict the respiration rhythm information, the target respiration rhythm information is obtained and pushed to the to-be-detected user, and the stability when target respiration rhythm information is recommended is improved. BRIEF DESCRIPTION OF DRAWINGS
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0021] Figure 1 A flowchart of a respiration rhythm recommendation method based on multi-modal data fusion is provided for the embodiments of the present application. Figure 2 A structural diagram of a terminal is provided for the embodiments of the present application. Figure 3 A structural diagram of a respiration rhythm recommendation device based on multi-modal data fusion is provided for the embodiments of the present application. DETAILED DESCRIPTION
[0022] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.
[0023] The terms "first", "second", and the like in the description and in the claims of the present application and above-described drawings are used to distinguish between similar objects, not to describe a particular sequential order. Moreover, the terms "include", and "have", and any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or device that comprises a list of steps or units is not limited to the listed steps or units, but can optionally further include additional steps or units not listed, or can optionally further include other steps or units inherent to such process, method, product, or device.
[0024] Reference to "an embodiment" in this application means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that the embodiments described in this application can be combined with each other in their various permutations and combinations.
[0025] See Figure 1 , Figure 1 A flowchart of a respiratory rhythm recommendation method based on multi-modal data fusion is provided for an embodiment of the application. As shown in Figure 1 , the method comprises: 101. Obtain an electrocardiosignal of a user to be detected when performing a CTA examination.
[0026] The electrocardiosignal of the user to be detected when performing a CTA (CT angiography) examination can be obtained by using an electrocardiosignal acquisition device.
[0027] 102. Perform heart rate state evaluation according to the electrocardiosignal to obtain a heart rate state evaluation result.
[0028] A method of performing heart rate state evaluation according to the electrocardiosignal to obtain a heart rate state evaluation result comprises: A1. Extract an R-wave heart rate sequence from the electrocardiosignal to obtain a target R-wave heart rate time sequence; A2. Perform frequency domain transformation on the target R-wave heart rate time sequence to obtain a target R-wave heart rate frequency sequence; A3. Calculate a low-frequency power value according to the target R-wave heart rate frequency sequence to obtain a first power value, and calculate a high-frequency power value according to the target R-wave heart rate frequency sequence to obtain a second power value; A4. Determine a state evaluation parameter according to the first power value and the second power value; A5. Perform heart rate state evaluation according to the state evaluation parameter to obtain a heart rate state evaluation result.
[0029] where the time difference between two consecutive R-waves in the ECG signal is assumed to be The target R-wave heart rate time series can be expressed as: The target R-wave heart rate time series can be transformed into the frequency domain using a fast Fourier transform to obtain a target R-wave heart rate frequency series.
[0030] The first power value (LF) and the second power value (HF) are calculated based on a frequency domain analysis of the heart rate variability (HRV) sequence formed by the R-R intervals, The HRV frequency domain power calculation formula is as follows, which facilitates strict quantification of the heart rate variability frequency domain features in the system: x(t) represents the heart rate variability signal (usually the interpolated R-R interval time series); represents the frequency spectrum obtained by Fourier transforming the signal; is the power spectral density (PSD) at frequency ; By integrating in a specific frequency band, the LF and HF power values can be obtained, respectively. Fast Fourier transform (FFT) is used to convert the time series into a frequency spectrum signal. By calculating the power integral of the frequency spectrum in a specific frequency band, the LF and HF power values are obtained. Specifically, let the power spectral density function of the HRV signal be P(f), then the power calculation formula is as follows: where: represents the power spectral density at frequency ; The LF frequency band (0.04-0.15 Hz) mainly reflects the common regulation of sympathetic and parasympathetic nerves; The HF frequency band (0.15-0.4 Hz) mainly reflects the activity of the parasympathetic nerve (vagus nerve).
[0031] where, is the power spectral density.
[0032] The ratio between the first power value and the second power value can be determined as a state assessment parameter. If the state assessment parameter is stable and within the physiological normal range (approximately 0.5 to 2.0), the heart rate state assessment result is considered to be a stable heart rhythm state; otherwise, the heart rate state is unstable.
[0033] 103. If the heart rate state assessment result indicates that the user to be detected is in a stable heart rate state, obtain a respiratory rhythm assessment parameter of the user to be detected.
[0034] Among them, respiratory rhythm assessment parameters include respiratory airflow rate (FLOW); blood oxygen saturation (SpO2); heart rate (HR); respiratory rhythm curve (respiratory phase); breath-hold response time; and breath-hold duration.
[0035] 104. Input the respiratory rhythm assessment parameters into a target respiratory rhythm recommendation model for calculation to obtain target respiratory rhythm information.
[0036] 104-1. Converting the target respiratory rhythm information into voice instructions or visual prompts to guide the user to hold their breath in real time; 104-2. The system synchronously collects the user's actual breathing execution status and evaluates feedback parameters such as response delay, coordination error or premature interruption; 104-3. The above feedback parameters can be used to judge the effectiveness of guidance and serve as training signals for subsequent strategy optimization, and are input into the reinforcement learning module or model tuning process.
[0037] Specifically, a method for inputting the respiratory rhythm assessment parameters into a target respiratory rhythm recommendation model for calculation to obtain target respiratory rhythm information includes: B1. normalizing the respiratory rhythm assessment parameter to obtain a normalized respiratory rhythm assessment parameter; B2. constructing a vector based on the normalized respiratory rhythm assessment parameters to obtain a model input vector; B3. The model input vector is input into the target respiratory rhythm recommendation model for calculation to obtain the target respiratory rhythm information.
[0038] A general normalization processing method can be used to obtain normalized respiratory rhythm evaluation parameters.
[0039] After the vector is constructed, the model input vector can be expressed as: Specifically, the objective function of the target respiratory rhythm recommendation model is represented by the following formula: in, is the loss function, penalize for model complexity.
[0040] The target respiratory rhythm recommendation model can also be obtained; the target respiratory rhythm recommendation model comprises: The initial respiratory rhythm recommendation model is adjusted by using a policy gradient reinforcement learning mechanism to obtain the target respiratory rhythm recommendation model. The expected equation of the policy gradient reinforcement learning mechanism is represented by the following formula: ; wherein, is the current physiological state of the patient, is the current guidance strategy, is the feedback score of this training, is the future reward discount factor.
[0041] Specifically, the target respiratory rhythm recommendation model is a long short-term memory neural network (LSTM) that models the respiratory rhythm in time sequence, extracts deep time sequence features, focuses on key respiratory phases by combining the Attention mechanism, and is trained to obtain.
[0042] The LSTM structure equation in the target respiratory rhythm recommendation model is as follows: Let the input at time step t be , the previous state be , then: wherein, forget gate, which controls whether to retain the information of the previous state.
[0043] input gate, which controls the influence of the current input on the cell state.
[0044] output gate, which controls the influence of the cell state on the output.
[0045] current cell state, which carries long-term memory information.
[0046] candidate cell state, which is calculated based on the current input and the previous hidden state.
[0047] hidden state, which is the output of the LSTM.
[0048] 、 、 : weight matrix, corresponding to the forget gate, input gate, output gate and candidate state respectively.
[0049] 、 、 : bias term, corresponding to the above gating function respectively.
[0050] The attention weighting mechanism is as follows: Improve the model's attention to key phases (such as maximum inspiration, breath-hold starting point) through weighted averaging: ; ; wherein, : attention weight of time step t, used to focus on key breathing phases.
[0051] : intermediate variable of attention scoring function, representing the correlation score of the current time and global information.
[0052] : context vector after attention weighting, focusing on key timing features.
[0053] After obtaining the target breathing rhythm recommendation model, SHAP explainability analysis can also be performed on the target breathing rhythm recommendation model. Specifically, SHAP value calculation is performed on the target breathing rhythm recommendation model to explain the contribution of each feature to the prediction result and improve the clinical credibility. The calculation of SHAP value can be performed in a general manner, which will not be described here.
[0054] The strategy gradient reinforcement learning mechanism can be introduced to dynamically adjust the rhythm guidance strategy according to the feedback of each training: Bellman expectation equation: wherein: : system state at time step t (such as patient's physiological state vector); : current guidance strategy; : feedback score of this training; : future return discount factor. Collection and fusion of multi-modal data - The parameters that can be synchronously collected by the system are: respiratory airflow rate (FLOW); blood oxygen saturation (SpO2); heart rate (HR); respiratory rhythm curve (breathing phase); breath-hold response time; breath-hold duration.
[0055] All parameters are normalized to form a unified input vector: .
[0056] 105. Push the target respiratory rhythm information to the user to be detected.
[0057] Specifically, the target respiratory rhythm information can be pushed to the user to be detected by voice broadcast.
[0058] In this example, the electrocardiogram (ECG) signal of the user to be detected during the CTA examination is obtained, and a heart rate status assessment is performed based on the ECG signal to obtain a heart rate status assessment result. If the heart rate status assessment result indicates that the user to be detected is in a stable heart rate state, the respiratory rhythm assessment parameters of the user to be detected are obtained, and the respiratory rhythm assessment parameters are input into a target respiratory rhythm recommendation model for calculation to obtain target respiratory rhythm information, and the target respiratory rhythm information is pushed to the user to be detected. Therefore, a status assessment can be performed based on the ECG signal of the user to be detected. When the heart rate is in a stable state, the acquired respiratory rhythm assessment parameters are used to predict the respiratory rhythm information, and the target respiratory rhythm information is obtained and pushed to the user to be detected, thereby improving the stability when recommending the target respiratory rhythm information.
[0059] For the same example as above, please refer to Figure 2 , Figure 2 A schematic diagram of the structure of a terminal provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the system comprises a processor, an input device, an output device and a memory, which are connected to each other, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions, and the program includes instructions for executing the following steps; Acquire the electrocardiogram (ECG) signal of the user to be examined during a CTA examination; Performing a heart rate state assessment based on the electrocardiogram signal to obtain a heart rate state assessment result; If the heart rate state evaluation result indicates that the user to be detected is in a stable heart rate state, obtaining a respiratory rhythm evaluation parameter of the user to be detected; Inputting the respiratory rhythm assessment parameters into a target respiratory rhythm recommendation model for calculation to obtain target respiratory rhythm information; The target respiratory rhythm information is pushed to the user to be detected.
[0060] The above describes the scheme of the embodiments of the present application mainly from the perspective of the process of executing the method. It can be understood that, in order to implement the above functions, the terminal comprises a hardware structure and / or a software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the unit examples and algorithm steps of the examples described in the present embodiment, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered as beyond the scope of the present application.
[0061] The embodiments of the present application can divide the functional units of the terminal according to the above method examples. For example, each functional unit can be divided corresponding to each function, or two or more functions can be integrated in one processing unit. The integrated unit can be implemented in the form of hardware or software functional unit. It should be noted that the division of units in the embodiments of the present application is illustrative, and is only a logical function division. Actual implementation can have another division manner.
[0062] Consistent with the above, please refer to Figure 3 , Figure 3 The present embodiment provides a structure schematic diagram of a respiratory rhythm recommendation device based on multi-modal data fusion. As shown in Figure 3 , the device comprises: A first acquisition unit 301 is configured to acquire an electrocardiogram signal of a user to be detected when performing a CTA examination. An evaluation unit 302 is configured to perform heart rate state evaluation according to the electrocardiogram signal to obtain a heart rate state evaluation result. A second acquisition unit 303 is configured to acquire a respiratory rhythm evaluation parameter of the user to be detected if the heart rate state evaluation result indicates that the user to be detected is in a heart rate stable state. An operation unit 304 is configured to input the respiratory rhythm evaluation parameter into a target respiratory rhythm recommendation model to perform operation and obtain target respiratory rhythm information. A pushing unit 305 is configured to push the target respiratory rhythm information to the user to be detected.
[0063] A guiding unit 306 is configured to generate a voice or image prompt signal according to the target respiratory rhythm information to guide the user to be detected to perform respiratory breath-hold cooperation according to the guided rhythm. A feedback acquisition unit 307 is configured to acquire response characteristics and cooperation effects of the user during the execution of the guided process to form feedback parameters. A strategy updating unit 308 is configured to input the feedback parameters into a strategy optimization module to realize continuous optimization of the guiding scheme.
[0064] In a possible implementation, the evaluation unit 302 is specifically configured to: extract an R-wave heart rate sequence from the electrocardiosignal to obtain a target R-wave heart rate time sequence; perform frequency domain transformation on the target R-wave heart rate time sequence to obtain a target R-wave heart rate frequency domain sequence; perform low-frequency power value calculation according to the target R-wave heart rate frequency domain sequence to obtain a first power value, and perform high-frequency power value calculation according to the target R-wave heart rate frequency domain sequence to obtain a second power value; determine a state evaluation parameter according to the first power value and the second power value; perform heart rate state evaluation according to the state evaluation parameter to obtain a heart rate state evaluation result.
[0065] In a possible implementation, the operation unit 304 is specifically configured to: perform normalization processing on the respiratory rhythm evaluation parameter to obtain a normalized respiratory rhythm evaluation parameter; perform vector construction according to the normalized respiratory rhythm evaluation parameter to obtain a model input vector; input the model input vector into a target respiratory rhythm recommendation model to perform operation, and obtain target respiratory rhythm information.
[0066] In a possible implementation, a target function of the target respiratory rhythm recommendation model is represented by the following formula: wherein, is a loss function, is a model complexity penalty.
[0067] In a possible implementation, the apparatus is further configured to: obtain a target respiratory rhythm recommendation model; In the aspect of obtaining the target respiratory rhythm recommendation model, the apparatus is further configured to: adjust an initial respiratory rhythm recommendation model by using a policy gradient reinforcement learning mechanism to obtain the target respiratory rhythm recommendation model; an expectation equation of the policy gradient reinforcement learning mechanism is represented by the following formula: ; wherein, is a current patient physiological state, is a current guide policy, is a feedback score of this training, is a future return discount factor.
[0068] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any one of the respiratory rhythm recommendation methods based on multimodal data fusion as described in the above method embodiments.
[0069] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program enables a computer to execute part or all of the steps of any one of the respiratory rhythm recommendation methods based on multimodal data fusion as described in the above method embodiments.
[0070] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.
[0071] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0072] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.
[0073] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0074] In addition, each functional unit in the embodiments of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware, or in the form of a software program module.
[0075] When the integrated unit is realized in the form of a software program module and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the application essentially or the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various program codes that can be stored in the medium.
[0076] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer readable storage medium, which can include a flash disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.
[0077] The embodiments of the application are described in detail above, and the principles and implementation manners of the application are described by applying specific examples. By constructing an observation-analysis-guidance-feedback closed-loop control mechanism based on multi-modal physiological data, personalized and dynamic optimization of respiratory control guidance is realized, and the cooperation quality, diagnostic accuracy and examination efficiency of coronary CTA images are greatly improved, which has wide clinical application value. The above description of the embodiments is only used to help understand the method of the application and its core idea; meanwhile, for those of ordinary skill in the art, according to the idea of the application, the specific implementation manner and application range will be changed, and the above description of the specification should not be understood as a limitation of the application.
Claims
1. A respiratory rhythm recommendation method based on multimodal data fusion, characterized in that: The method comprises: Acquire the electrocardiogram (ECG) signal of the user to be examined during a CTA examination; Performing a heart rate state assessment based on the electrocardiogram signal to obtain a heart rate state assessment result; If the heart rate state evaluation result indicates that the user to be detected is in a stable heart rate state, obtaining a respiratory rhythm evaluation parameter of the user to be detected; Inputting the respiratory rhythm assessment parameters into a target respiratory rhythm recommendation model for calculation to obtain target respiratory rhythm information; The target respiratory rhythm information is pushed to the user to be detected.
2. The respiratory rhythm recommendation method based on multimodal data fusion according to claim 1, characterized in that: The step of performing a heart rate state assessment based on the electrocardiogram signal to obtain a heart rate state assessment result includes: Extracting an R-wave heart rate sequence from the electrocardiogram signal to obtain a target R-wave heart rate time series sequence; Performing frequency domain transformation on the target R wave heart rate time series to obtain a target R wave heart rate frequency domain series; Calculating a low-frequency power value based on the target R-wave heart rate frequency domain sequence to obtain a first power value, and calculating a high-frequency power value based on the target R-wave heart rate frequency domain sequence to obtain a second power value; determining a state assessment parameter according to the first power value and the second power value; A heart rate state assessment is performed according to the state assessment parameters to obtain a heart rate state assessment result.
3. The respiratory rhythm recommendation method based on multimodal data fusion according to claim 2, characterized in that: The step of inputting the respiratory rhythm assessment parameters into a target respiratory rhythm recommendation model for calculation to obtain target respiratory rhythm information includes: Normalizing the respiratory rhythm assessment parameter to obtain a normalized respiratory rhythm assessment parameter; A vector is constructed according to the normalized respiratory rhythm evaluation parameters to obtain a model input vector; The model input vector is input into the target respiratory rhythm recommendation model for calculation to obtain the target respiratory rhythm information.
4. The respiratory rhythm recommendation method based on multimodal data fusion according to any one of claims 1 to 3, characterized in that: The objective function of the target respiratory rhythm recommendation model is represented by the following formula: in, is the loss function, is the model complexity penalty.
5. The respiratory rhythm recommendation method based on multimodal data fusion according to claim 4, characterized in that: The method further includes obtaining a target respiratory rhythm recommendation model; The step of obtaining a target respiratory rhythm recommendation model includes: The policy gradient reinforcement learning mechanism is used to adjust the initial respiratory rhythm recommendation model to obtain the target respiratory rhythm recommendation model; The expected equation of the policy gradient reinforcement learning mechanism is represented by the following formula: ； in, is the current physiological state of the patient, is the current boot strategy, Feedback score for this training session: is the future earnings discount factor.
6. A respiratory rhythm recommendation device based on multimodal data fusion, characterized in that: The device comprises: A first acquiring unit is used to acquire an electrocardiogram signal of a user to be detected during a CTA examination; an evaluation unit, configured to perform a heart rate state evaluation based on the electrocardiogram signal to obtain a heart rate state evaluation result; a second acquiring unit, configured to acquire a respiratory rhythm evaluation parameter of the user to be detected if the heart rate state evaluation result indicates that the user to be detected is in a stable heart rate state; a calculation unit, configured to input the respiratory rhythm assessment parameters into a target respiratory rhythm recommendation model for calculation to obtain target respiratory rhythm information; A push unit, configured to push the target respiratory rhythm information to the user to be detected; The guidance unit is used to generate voice or image prompt signals based on the target respiratory rhythm information to guide the user to be tested to hold their breath according to the guided rhythm; the feedback collection unit is used to collect the user's response characteristics and coordination effects during the guidance process to form feedback parameters; the strategy update unit is used to input the feedback parameters into the strategy optimization module to achieve continuous optimization of the guidance plan.
7. The respiratory rhythm recommendation device based on multimodal data fusion according to claim 6, characterized in that: The evaluation unit is specifically used for: Extracting an R-wave heart rate sequence from the electrocardiogram signal to obtain a target R-wave heart rate time series sequence; Performing frequency domain transformation on the target R wave heart rate time series to obtain a target R wave heart rate frequency domain series; Calculating a low-frequency power value based on the target R-wave heart rate frequency domain sequence to obtain a first power value, and calculating a high-frequency power value based on the target R-wave heart rate frequency domain sequence to obtain a second power value; determining a state assessment parameter according to the first power value and the second power value; A heart rate state assessment is performed according to the state assessment parameters to obtain a heart rate state assessment result.
8. The respiratory rhythm recommendation device based on multimodal data fusion according to claim 7, characterized in that: The computing unit is specifically used for: Normalizing the respiratory rhythm assessment parameter to obtain a normalized respiratory rhythm assessment parameter; A vector is constructed according to the normalized respiratory rhythm evaluation parameters to obtain a model input vector; The model input vector is input into the target respiratory rhythm recommendation model for calculation to obtain the target respiratory rhythm information.
9. A terminal, characterized in that: The method comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the respiratory rhythm recommendation method based on multimodal data fusion as described in any one of claims 1 to 5.
10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the processor executes the respiratory rhythm recommendation method based on multimodal data fusion according to any one of claims 1 to 5.