Remote collaborative auscultation data processing method and system based on dual-link timing synchronization

Abstract

The application discloses a kind of remote collaborative auscultation data processing method and system based on double-link timing synchronization, it is related to medical data processing technical field.The method includes: establishing collaborative session generates collaborative session identification, generates auscultation signal acquisition parameter based on collaborative session identification and is issued to end side;Intelligent electronic stethoscope acquires physiological signal according to parameter and carries out pre-processing output physiological sound frequency data;Edge intelligent agent gateway carries out feature analysis to physiological sound frequency data, obtains edge side inference result, encapsulates it with audio data and collaborative session identification as media data stream, transmits through media stream passage, simultaneously transmits the control instruction bound by signaling channel;Cloud center is based on collaborative session identification, and media data stream and control instruction are double-link timing alignment, and audio data is carried out deep reasoning in combination with edge side inference result, and the auxiliary diagnosis and treatment decision result is obtained.

Description

Technical Field

[0001] This invention relates to the field of medical data processing technology, and in particular to a remote collaborative auscultation data processing method and system based on dual-link time synchronization. Background Technology

[0002] With the rapid development of telemedicine technology, remote auscultation has become an important means of screening and following up on cardiopulmonary diseases across regions. Medical institutions often need to obtain patients' physiological sounds remotely via the network to overcome the problem of uneven distribution of medical resources. In order to ensure the objectivity and timeliness of diagnosis, extremely high requirements are placed on the collection, transmission, and processing of auscultation data.

[0003] In practical clinical applications, traditional remote auscultation mainly relies on digital stethoscopes to collect audio data, which is then directly uploaded to the cloud or the doctor's end via conventional audio and video channels. However, this approach has significant limitations. First, physiological audio streams and device control signals are often transmitted through a single channel or misaligned independent channels, which can easily lead to packet delays or out-of-order delivery in complex network environments. This results in a timing mismatch between the sounds heard by the doctor and the actual acquisition status at the terminal, affecting the accurate judgment of pathological rhythms. Second, existing auscultation systems lack intelligent preprocessing capabilities on the end side. Directly transmitting massive amounts of raw audio data to the cloud not only consumes a large amount of network bandwidth and exacerbates transmission latency, but also increases the computational load on the cloud server due to data that has not undergone preliminary feature extraction, resulting in low overall feedback efficiency.

[0004] Furthermore, current remote auscultation devices often fail to effectively correlate and encapsulate audio data, acquisition parameters, and preliminary analysis results during data transfer, resulting in fragmented diagnostic data. This fragmentation makes it difficult for cloud-based systems to perform subsequent in-depth analysis without multi-dimensional references, hindering the precise identification of complex pathological features.

[0005] In summary, existing remote auscultation data processing methods suffer from a lack of timing synchronization mechanisms between audio / video streams and control commands, insufficient edge-cloud collaborative computing capabilities, and poor data encapsulation and correlation. This results in an inability to efficiently and objectively reconstruct the auscultation process and provide timely intelligent assistance. This not only increases the diagnostic difficulty for doctors but also limits the overall effectiveness of remote diagnostic systems in large-scale applications. Therefore, there is an urgent need for a remote auscultation data processing technology solution that can achieve multi-link data timing alignment and possess edge-cloud collaborative processing capabilities. Summary of the Invention

[0006] This invention provides a remote collaborative auscultation data processing method and system based on dual-link timing synchronization, which solves the problems of the lack of multi-link timing synchronization mechanism and end-to-cloud collaborative processing capability in the prior art.

[0007] The first aspect of this invention proposes a remote collaborative auscultation data processing method based on dual-link timing synchronization, comprising: Establish a collaborative session and generate a collaborative session identifier. The cloud center generates auscultation signal acquisition parameters based on the collaborative session identifier and sends the auscultation signal acquisition parameters to the edge intelligent agent gateway and the intelligent electronic stethoscope. The intelligent electronic stethoscope collects physiological signals based on auscultation signal acquisition parameters and performs preprocessing to output physiological audio data. The edge intelligent agent gateway performs feature analysis on physiological audio data to obtain edge-side inference results. It encapsulates the physiological audio data, edge-side inference results, and collaborative session identifiers to form a media data stream. The media data stream is transmitted to the cloud center through the media stream channel, and control commands bound to the collaborative session identifiers are transmitted to the cloud center through the signaling channel. Based on the collaborative session identifier, the cloud center performs dual-link timing alignment of the received media data stream and the received control commands, and combines the edge-side inference results to perform deep inference on the timing-aligned physiological audio and video data to obtain auxiliary diagnosis and treatment decision results.

[0008] In one alternative implementation, the auscultation signal acquisition parameters are generated based on clinical acquisition modes, including pure heart sound mode, pure lung sound mode, and simultaneous heart and lung sound acquisition mode. When the clinical acquisition mode is pure heart sound mode, the effective frequency band parameter for auscultation signal acquisition is 20 Hz to 200 Hz. When the clinical acquisition mode is pure lung sound mode, the effective frequency band parameter for auscultation signal acquisition is 100 Hz to 500 Hz. When the clinical acquisition mode is the simultaneous cardiopulmonary acquisition mode, the effective frequency band parameter for auscultation signal acquisition is 20 Hz to 2000 Hz.

[0009] In one alternative implementation, the physiological signals include cardiopulmonary audio signals and low-frequency vibration signals; the preprocessing includes sequentially performed anti-aliasing pre-normalization, baseline drift correction, noise suppression, adaptive resampling, and pathological feature-oriented enhancement. Anti-aliasing pre-normalization employs a two-stage combined analog and digital filter to remove aliasing clutter. Baseline drift correction is used to remove low-frequency noise and preserve the effective signal base waveform; Noise suppression is used to compress noise gain without signal amplitude distortion. Adaptive resampling is used to perform downsampling based on the auscultation signal acquisition parameters; Pathological feature-oriented enhancement is used to amplify the gain of a preset pathological frequency band using an envelope extraction algorithm, while locking the temporal phase of physiological audio-visual data during the gain amplification process.

[0010] In one alternative implementation, physiological audio data, edge-side inference results, and collaborative session identifiers are encapsulated to form a media data stream, including: Implicit watermarking technology is used to write the collaborative session identifier into the least significant bit of the physiological audio data in order to preserve the high-bit acoustic features; The edge-side inference result, collaborative session identifier, sampling parameters, device number, and auscultation site timestamp are written into the extended header of the real-time transmission protocol to form a media data stream; The media streaming channel is a transmission channel that uses a real-time audio and video streaming media protocol, while the signaling channel is a transmission channel that uses a publish-subscribe mechanism.

[0011] In one alternative implementation, the edge agent gateway has a built-in lightweight multilayer perceptron network model. Feature analysis of physiological acoustic data yields edge-side inference results, including: Physiological audio data is input into a lightweight multilayer perceptron network model for shallow feature perception to identify basic physiological indicators, including auscultation location on the body surface, baseline audio noise, baseline heart rate, and respiratory rate. The identified basic physiological indicators are then used as the results of edge-side inference.

[0012] In one optional implementation, deep reasoning is performed on the time-aligned physiological audio-visual data in conjunction with edge-side inference results to derive auxiliary diagnostic and treatment decision-making results, including: The time-aligned physiological audio data and the edge-side inference results are input into the dual-channel parallel network model; Short-term local pathological features of physiological audio data are extracted by a one-dimensional convolutional neural network in a dual-channel parallel network model, and long-term temporal rhythm-dependent features of physiological audio data are extracted by a multi-head attention mechanism in a dual-channel parallel network model. The system integrates short-term local pathological features, long-term temporal rhythm-dependent features, and marginal inference results to mine higher-order features of complex pathological murmurs. Based on these higher-order features, it generates auxiliary diagnostic and treatment decision-making results that include disease classification and disease risk grading.

[0013] In one alternative implementation, the dual-channel parallel network model is configured with adaptive reinforcement learning reward logic, which is used to drive the dual-channel parallel network model to perform online autonomous iterative optimization. The adaptive reinforcement learning reward logic specifically includes: obtaining the expert calibration matching factor, the model output confidence coefficient, the previous comprehensive training loss value, and the difficult sample bias coefficient; assigning preset weighted hyperparameters to the expert calibration matching factor, the model output confidence coefficient, the previous comprehensive training loss value, and the difficult sample bias coefficient respectively, and then performing weighted calculations to obtain the comprehensive adaptive reward value. Based on the comprehensive adaptive reward value, the samples corresponding to the physiological audio data are assigned to either the positive reward sample set or the negative reward sample set, and incremental rolling training of the dual-channel parallel network model is completed based on the positive reward sample set and the negative reward sample set.

[0014] In one alternative implementation, after obtaining the auxiliary diagnosis and treatment decision results, the method further includes a step of hierarchical data storage based on collaborative session identifiers: The physiological audio data in the media data stream is archived and stored in a persistent cloud database; The edge-side inference results and auxiliary diagnosis and treatment decision results are cached in the high-speed database of the edge agent gateway; Obtain expert review opinions and final diagnostic conclusions, and archive the expert review opinions and final diagnostic conclusions into the main medical business database; The collaborative session identifier is used to link and anchor heterogeneous data in the cloud persistent database, high-speed database and medical business master database.

[0015] In one alternative implementation, during the process of sending the auscultation signal acquisition parameters to the edge agent gateway and the intelligent electronic stethoscope, a hierarchical cyclic redundancy check mechanism based on the collaborative session identifier is executed: Generate a global cyclic redundancy check code bound to the collaborative session identifier, and assign independent sub-cyclic redundancy check codes to different levels of parameters in the auscultation signal acquisition parameters; When the edge agent gateway detects a verification anomaly based on global cyclic redundancy check (CRC) or sub-cyclic redundancy check (CRC), it automatically triggers a retransmission mechanism for the auscultation signal acquisition parameters.

[0016] A second aspect of this invention proposes a remote collaborative auscultation data processing system based on dual-link timing synchronization, comprising multiple modules for executing the remote collaborative auscultation data processing method based on dual-link timing synchronization as proposed in the foregoing embodiments, specifically including: The collaborative session and parameter distribution module is used to establish collaborative sessions and generate collaborative session identifiers. The cloud center generates auscultation signal acquisition parameters based on the collaborative session identifiers and distributes the auscultation signal acquisition parameters to the edge intelligent agent gateway and the intelligent electronic stethoscope. The preprocessing and output module is used to collect physiological signals based on the auscultation signal acquisition parameters through the intelligent electronic stethoscope, perform preprocessing, and output physiological audio data. The initial screening and dual-link transmission module is used to perform feature analysis on physiological audio data through the edge intelligent agent gateway to obtain edge-side inference results. It encapsulates the physiological audio data, edge-side inference results, and collaborative session identifier to form a media data stream. The media data stream is transmitted to the cloud center through the media stream channel, and control commands bound to the collaborative session identifier are transmitted to the cloud center through the signaling channel. The timing alignment and inference module is used to perform dual-link timing alignment of the received media data stream and the received control commands based on the collaborative session identifier through the cloud center, and to perform deep inference on the timing-aligned physiological audio data in combination with the edge-side inference results to obtain auxiliary diagnosis and treatment decision results.

[0017] Beneficial effects: This invention provides a remote collaborative auscultation data processing method and system based on dual-link timing synchronization. By introducing a globally unified collaborative session identifier, it organically combines cloud parameter distribution, edge preprocessing, and end-side data acquisition. This scheme utilizes an edge intelligent agent gateway to perform preliminary feature analysis on audio data, constructing an end-cloud collaborative processing architecture based on the Internet of Things (IoT). This effectively overcomes the limitations of existing technologies, such as asynchronous data transmission, high computational pressure on a single cloud platform, and high diagnostic latency. By uniformly encapsulating physiological audio data, edge-side inference results, and session identifiers into a media data stream, deep binding of multi-dimensional data is achieved. More importantly, a dual-link timing alignment mechanism for media streams and signaling streams is adopted, enabling the cloud to accurately match received media data with control commands based on the collaborative session identifier. This avoids timing misalignment problems in complex network environments, improves the objectivity and accuracy of assisted diagnostic decisions, and allows the system to maintain stable and efficient operation in multi-terminal collaborative diagnostic scenarios. Attached Figure Description

[0018] Figure 1 A flowchart illustrating the steps of a remote collaborative auscultation data processing method based on dual-link timing synchronization, provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the functional modules of a remote collaborative auscultation data processing system based on dual-link timing synchronization, provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0020] The present invention will be further described below with reference to the accompanying drawings.

[0021] This embodiment provides a remote collaborative auscultation data processing method based on dual-link timing synchronization. (See attached...) Figure 2 As shown, the remote collaborative auscultation data processing method based on dual-link timing synchronization includes the following steps: S101, Establish a collaborative session and generate a collaborative session identifier. The cloud center generates auscultation signal acquisition parameters based on the collaborative session identifier and sends the auscultation signal acquisition parameters to the edge intelligent agent gateway and the intelligent electronic stethoscope.

[0022] In remote collaborative auscultation clinical scenarios, participants typically include remote diagnostic experts, on-site patients, and medical staff operating the intelligent electronic stethoscope. To ensure logical isolation and temporal consistency in data flow between these multiple nodes, the cloud center first needs to verify the legitimate permissions of each participant through a unified identity authentication mechanism. When a remote auscultation task is initiated, the core scheduling module of the cloud center allocates and generates a globally unique collaborative session identifier (e.g., a Universally Unique Identifier, UUID). The generation of the collaborative session identifier relies on a timestamp mechanism and a hash operation with the Media Access Control Address (MAC) of the on-site device. This ensures that the collaborative session identifier not only serves as the unique primary key for subsequent audio data, control signaling, and diagnostic reports, but also enables the establishment of many-to-many real-time session channels in complex wide area network environments, thereby guaranteeing the security of session operation and the isolation of medical data.

[0023] After generating the collaborative session identifier, the cloud center dynamically customizes the auscultation signal acquisition parameters based on the current patient's medical record information or the expert's specified instructions. These auscultation signal acquisition parameters include low-level register configuration information for configuring the front-end hardware pickup circuit, analog filter cutoff frequency, programmable gain amplifier (PGA), and digital signal processor (DSP). Subsequently, the cloud center sends the auscultation signal acquisition parameters, carrying the collaborative session identifier, to the edge intelligent agent gateway deployed on-site via a wireless network or dedicated network. The edge intelligent agent gateway then transmits the data to the intelligent electronic stethoscope via short-range wireless communication, completing the data initialization and acquisition environment setup at the device end.

[0024] In one optional implementation, the auscultation signal acquisition parameters are generated based on a clinical acquisition mode, which includes a pure heart sound mode, a pure lung sound mode, and a simultaneous heart and lung sound acquisition mode. When the clinical acquisition mode is the pure heart sound mode, the effective frequency band parameter of the auscultation signal acquisition parameters is configured to be 20 Hz to 200 Hz. When the clinical acquisition mode is the pure lung sound mode, the effective frequency band parameter of the auscultation signal acquisition parameters is configured to be 100 Hz to 500 Hz. When the clinical acquisition mode is the simultaneous heart and lung sound acquisition mode, the effective frequency band parameter of the auscultation signal acquisition parameters is configured to be 20 Hz to 2000 Hz.

[0025] Regarding the above-mentioned optional implementation methods, the step of generating auscultation signal acquisition parameters by the cloud center based on the collaborative session identifier and distributing the auscultation signal acquisition parameters to the edge intelligent agent gateway and the intelligent electronic stethoscope further includes the following sub-steps: S1011. The cloud center obtains the clinical acquisition mode selected by the remote expert on the visual interactive interface, and queries the medical spectrum feature database preset in the cloud center according to the clinical acquisition mode. Because the mechanical vibration frequency generated by the closure of human heart valves is low, mainly concentrated in the low-frequency range, while the friction sound and turbulence sound generated by respiratory airflow through the airway have relatively high frequencies, it is necessary to accurately divide the frequency bands of sound pickup from different locations according to different diagnostic purposes in order to suppress the noise masking effect of non-target frequency bands.

[0026] S1012. When the clinical acquisition mode is the pure heart sound mode, the cloud center generates a first parameter template. The first parameter template limits the effective frequency band parameter in the auscultation signal acquisition parameters to between 20 Hz and 200 Hz. The frequency band range is set to cover the first heart sound (S1), the second heart sound (S2), and pathological low-frequency heart murmurs, while instructing the front-end hardware to use bandpass filtering logic to filter out high-frequency environmental background noise greater than 200 Hz and ultra-low-frequency baseline drift less than 15 Hz caused by human breathing and body tremors.

[0027] S1013. When the clinical acquisition mode is the pure lung sound mode, the cloud center generates a second parameter template. The second parameter template configures the effective frequency band parameter in the auscultation signal acquisition parameters to 100 Hz to 500 Hz. This configuration aims to capture normal vesicular breath sounds, bronchial breath sounds, and any wheezing or moist rales that may occur. By setting a lower cutoff frequency of 100 Hz, low-frequency heart sound interference and body vibration noise below 100 Hz are effectively filtered out; by setting an upper cutoff frequency of 500 Hz, high-frequency electromagnetic interference and human voices in the ward above 500 Hz are filtered out.

[0028] S1014. When the clinical acquisition mode is the simultaneous cardiopulmonary auscultation mode, in order to meet the comprehensive and rapid multi-site screening needs of critically ill patients, the cloud center generates a third parameter template. The third parameter template broadens the effective frequency band parameter in the auscultation signal acquisition parameters to 20 Hz to 2000 Hz. This configuration covers the acoustic characteristics of both heart sounds and lung sounds in a full-width frequency band, and, in conjunction with the built-in dual-channel independent notch filtering algorithm, performs polynomial sorting on the cross-aliasing clutter of heart and lung sounds in overlapping frequency bands (such as between 100 Hz and 200 Hz) to avoid mutual aliasing interference of heterogeneous physiological sounds.

[0029] S1015, the cloud center encapsulates the determined effective frequency band parameters, anti-aliasing filter order, noise reduction gain compression ratio, and other structured data with the collaborative session identifier in a structured manner. During this process, the data conversion engine generates lightweight data packets in standardized Extensible Markup Language (XML) or JavaScript Object Notation (JSON) format and pushes them to the distribution queue of the edge intelligent agent gateway.

[0030] In one optional implementation, during the process of sending the auscultation signal acquisition parameters to the edge agent gateway and the intelligent electronic stethoscope, a hierarchical cyclic redundancy check (CR) mechanism based on the collaborative session identifier is executed: a global CR code bound to the collaborative session identifier is generated, and independent sub-CR codes are assigned to different level parameters in the auscultation signal acquisition parameters; when the edge agent gateway detects a check anomaly based on the global CR code or the sub-CR code, the retransmission mechanism of the auscultation signal acquisition parameters is automatically triggered.

[0031] Regarding the above-mentioned optional implementation methods, the step of sending the auscultation signal acquisition parameters to the edge intelligent agent gateway and the intelligent electronic stethoscope further includes the following sub-steps: S1016. During the process of assembling and distributing data packets at the cloud center, in order to ensure the high reliability of medical-grade device control commands during long-distance wireless transmission and to avoid packet loss due to network congestion or physical layer bit flipping leading to incorrect filtering parameter settings of the medical device, a polynomial division algorithm is used to perform hash calculation on the payload of the entire data packet. The mathematical expression of this calculation process is as follows: [The text abruptly ends here, likely due to an incomplete sentence or a missing section.] Modulo-2 division is performed on the message polynomial to generate the global cyclic redundancy check (CRC) code associated with and bound to the cooperative session identifier. The mathematical model formula for the cyclic redundancy check is as follows:

[0032] In the formula, This represents the generated cyclic redundancy check remainder polynomial, which is the final check code bit sequence appended to the end of the message. This represents the original data message polynomial composed of the cooperative session identifier and all parameter sets; This represents the highest order of the predefined generator polynomial (e.g., under the CRC-16 standard). ); Represents a predefined generator polynomial (e.g.) ).

[0033] S1017. Further, the cloud center divides the auscultation signal acquisition parameters into functional layers, for example, dividing the analog circuit configuration, digital filter configuration, sampling rate configuration, and encryption key into four different independent blocks. A polynomial division operation is performed on each block individually to assign independent sub-cyclic redundancy check codes to different levels of parameters in the auscultation signal acquisition parameters. This layered verification mechanism can accurately locate the specific parameter level where the data packet is damaged when some bits are corrupted, reducing the granularity of retransmitted data.

[0034] S1018. The cloud center sends a data packet containing the global cyclic redundancy check (CRC) code and all the sub-CRC codes via a Transmission Control Protocol (TCP) network. Upon receiving the data packet, the edge agent gateway first extracts the collaborative session identifier at the application layer for legitimacy matching, and then uses the same generator polynomial locally... Recalculate the checksum of the received data.

[0035] S1019. When the edge intelligent agent gateway compares the locally calculated checksum with the received global cyclic redundancy check (CRC) code or the sub-CRC code, if a checksum inconsistency is found, i.e., a checksum anomaly is detected, the edge intelligent agent gateway immediately discards the currently corrupted data packet and sends a Negative Acknowledgment (NACK) message to the cloud center through the control signaling channel. This action automatically triggers the automatic retransmission mechanism of the auscultation signal acquisition parameters. The retransmission mechanism is executed cyclically until all levels of checksums match successfully. Only then does the edge intelligent agent gateway transparently transmit the parameters to the intelligent electronic stethoscope, performing the corresponding hardware state reset and register overwrite. This verification process achieves secure interlocking of medical parameters on the WAN and LAN transmission links, ensuring the accuracy of the terminal device configuration.

[0036] S102, the intelligent electronic stethoscope collects physiological signals based on auscultation signal acquisition parameters and performs preprocessing, outputting physiological audio-visual data.

[0037] After parameter distribution and security verification are completed, the microcontroller unit (MCU) inside the intelligent electronic stethoscope parses the received auscultation signal acquisition parameters. Based on the parsing results, the microcontroller unit uses the Serial Peripheral Interface (SPI) or the Inter-Integrated Circuit (I2C) bus to adjust the analog bias voltage of its built-in Micro-Electro-Mechanical Systems (MEMS) microphone array, the oversampling frequency of the high-precision analog-to-digital converter (ADC), and the computational execution flow of the digital signal processor. The operator attaches the intelligent electronic stethoscope to the corresponding auscultation area on the patient's body surface, and the intelligent electronic stethoscope begins to acquire mechanical vibration information and acoustic fluctuations within the body. Subsequently, the pre-processing signal normalization and data optimization process is initiated in the local hardware computing chip. The original acoustic signals have low amplitude characteristics in the microvolt to millivolt range, often accompanied by sensor background white noise, skin friction noise caused by the doctor's hand tremors, and power grid frequency interference at 50 Hz or 60 Hz. By performing serialized multi-level digital filtering and feature enhancement operations, the preprocessing process removes these interfering factors at the physical frequency band or mathematical model level, and finally outputs standardized, digitized, amplitude-normalized physiological audio data (e.g., lossless digital audio stream with a sampling bit depth of 16 or 24 bits, using Pulse Code Modulation, PCM format).

[0038] In one optional implementation, the physiological signal includes cardiopulmonary audio signals and low-frequency vibration signals; the preprocessing includes sequentially performing anti-aliasing pre-normalization, baseline drift correction, noise suppression, adaptive resampling, and pathological feature-oriented enhancement; the anti-aliasing pre-normalization employs a combined analog and digital two-stage filtering to filter out aliasing clutter; the baseline drift correction is used to strip away low-frequency noise and retain the effective signal base waveform; the noise suppression is used to compress noise gain without signal amplitude distortion; the adaptive resampling is used to perform downsampling processing based on the auscultation signal acquisition parameters; the pathological feature-oriented enhancement is used to amplify the gain of a preset pathological frequency band using an envelope extraction algorithm, and during the gain amplification process, the temporal phase of the physiological audio-visual data is locked.

[0039] Regarding the above-mentioned optional implementation methods, the step of the intelligent electronic stethoscope acquiring physiological signals based on auscultation signal acquisition parameters, preprocessing them, and outputting physiological audio-visual data further includes the following sub-steps: S1021. The intelligent electronic stethoscope acquires cardiopulmonary audio signals containing high-frequency respiratory fluid dynamics information through a wideband condenser microphone within its front acoustic cavity. Simultaneously, the intelligent electronic stethoscope utilizes a triaxial MEMS accelerometer rigidly connected to a diaphragm to acquire low-frequency vibration signals representing the mechanical kinetic energy of the apical impulse and the low-frequency displacement of atrial contractions. Both sensors share the same high-precision clock source to trigger sampling, thereby achieving high-precision synchronous acquisition of the physiological signals in both acoustic and dynamic modes.

[0040] S1022, the digital signal processor of the intelligent electronic stethoscope sequentially initiates a five-level personalized digital and analog joint processing link. First, the anti-aliasing pre-regulation is executed. In the hardware analog circuit processing stage, the original weak electrical signal is amplified by a low-noise amplifier (LNA) and then input to an active hardware low-pass filter (e.g., a second-order Butterworth low-pass filter constructed using operational amplifiers and RC networks). A physical cutoff frequency is set according to the stethoscope signal acquisition parameters. Its function is to prevent high-frequency environmental noise higher than half the Nyquist sampling frequency from folding into the effective baseband during the subsequent ADC analog-to-digital conversion stage, causing irreversible frequency aliasing distortion. Subsequently, after high-sampling-rate conversion (e.g., oversampled Sigma-Delta ADC), a high-order (e.g., an eighth- or tenth-order 128-tap digital finite impulse response (FIR) filter is applied again in the digital domain. The FIR filter implements a two-stage joint analog and digital filtering strategy. This combined strategy not only effectively filters out out-of-band aliasing waves in the spectrum, but also relies on the symmetry coefficient characteristics of the FIR filter to ensure the phase linearity of each frequency component in the passband, reducing the risk of phase shift delay in the generation of physiological acoustic signals.

[0041] S1023. After completing the anti-aliasing processing, the digital signal is input to the next-level register to perform the baseline drift correction. The periodic rise and fall of the chest caused by the patient's breathing and the non-steady sliding between the stethoscope diaphragm and the skin introduce low-frequency and large-amplitude slow drift into the voltage signal, causing the overall audio waveform to deviate from the absolute zero level axis. The intelligent electronic stethoscope uses an adaptive wavelet transform algorithm or an infinite impulse response (IIR) high-pass filter configured with zero-phase forward and backward filtering, and sets a strict lower cutoff frequency (e.g., dynamically set from 0.5 Hz to 15 Hz based on the auscultation signal acquisition parameters). The low-frequency drift noise is removed by mathematical algorithms, and the center baseline of the signal is corrected. At the same time, through the fine control of wavelet decomposition and reconstruction, the low-frequency mechanical vibration energy contained in the first and second heart sounds, as well as the morphological integrity of the effective signal base waveform, can be preserved.

[0042] S1024. After the baseline stabilizes, the noise suppression algorithm is executed. For common power grid interference (50 Hz or 60 Hz), mechanical fan noise from medical instruments, and background human voice reverberation in complex environments such as hospital intensive care units, a digital dual-band stop filter (i.e., a high-Q notch filter) is used to remove power frequency interference at specific frequencies. Simultaneously, an adaptive noise reduction algorithm, such as spectral subtraction or multi-band Wiener filtering, is executed using a computing chip. The algorithm estimates the power spectral density of non-stationary environmental noise in real time for each frame of signal and dynamically calculates the signal-to-noise ratio and noise gain compression ratio in the frequency domain. This process follows the core constraint principle of noise gain compression under the premise of no signal amplitude distortion. That is, when a sudden, high-frequency pathological noise characteristic frequency band energy (such as systolic jet noise) is detected, the microcontroller unit automatically reduces the noise reduction suppression amplitude corresponding to that frequency band, avoiding the misinterpretation of weak clinical pathological noise as background noise and filtering it out, thereby preserving the original amplitude mapping information of the pathological features.

[0043] S1025. To balance the high-resolution accuracy of signal acquisition at the device end with the bandwidth load limitation of the subsequent wireless wide area network transmission channel, adaptive resampling is performed. The ADC inside the intelligent electronic stethoscope initially performs high-speed oversampling at a high original frequency (e.g., 6000 samples per second, 6000 Sps) to obtain a distortion-free wideband signal. Subsequently, the operation logic performs downsampling processing based on the resampling ratio index issued in the stethoscope signal acquisition parameters, using an anti-aliasing decimation filter combined with cubic spline interpolation or linear polynomial interpolation algorithms. For example, in the pure heart tone mode where only low-frequency signals need to be focused, the sampling rate is smoothly decimated and reduced to 1200 Sps. While significantly reducing the data transmission load, according to Shannon's Information Theorem, all waveform information required for reconstruction within the target effective frequency band is preserved.

[0044] S1026. At the end of the preprocessing link, the core pathological feature-oriented enhancement step is executed. For faint diastolic rumbling murmurs or early fine crackles that are easily missed in clinical practice, conventional global amplitude amplification methods would cause normal background heart and lung sounds to be amplified proportionally, resulting in a masking effect on human hearing. To solve this problem, the intelligent electronic stethoscope, within the computing power allowed by the terminal DSP, uses the Hilbert Transform algorithm to extract the instantaneous energy envelope of the signal in a specific pathological frequency band. The mathematical kernel calculation formula for its envelope extraction is as follows:

[0045] In the formula, Represents a specific point in time extracted. The instantaneous envelope amplitude of the pathological frequency band signal; The original physiological sound signal components represent a specific preset pathological frequency band (such as the wheezing audio band from 1100 Hz to 1900 Hz) extracted through bandpass filtering; Represents the signal The orthogonal analytic signal component obtained after performing the Hilbert integral transform has a phase shift of 90 degrees.

[0046] Calculate the instantaneous envelope amplitude Subsequently, the algorithm calculates a dynamic adaptive gain coefficient matrix based on a preset activation threshold. This coefficient matrix is ​​then used to selectively amplify the extracted envelope region using nonlinear gain, meaning energy is boosted only at the instant the pathological murmur appears. During the mathematical operations of this gain amplification, the algorithm relies on strict internal register clock cycle instructions from the hardware DSP to lock the temporal phase of the physiological audio data. This means changing the vertical amplitude of the pathological murmur segment at a specific time point, but not altering its horizontal occurrence time node or frequency cycle interval. This phase-locking characteristic ensures that after the signal is amplified, the objective rhythmic characteristics of the heartbeat (such as the ratio of systolic to diastolic time) and the long respiratory cycle do not experience any temporal distortion, ultimately outputting physiological audio data (such as a standard PCM audio stream file) with prominent pathological features and accurate rhythm.

[0047] S103, the edge intelligent agent gateway performs feature analysis on the physiological audio data to obtain the edge-side inference result, encapsulates the physiological audio data, the edge-side inference result and the collaborative session identifier to form a media data stream, transmits the media data stream to the cloud center through the media stream channel, and at the same time transmits the control command bound to the collaborative session identifier to the cloud center through the signaling channel.

[0048] After acquiring the physiological audio data, the intelligent electronic stethoscope pushes the physiological audio data as a serial data stream to the paired edge intelligent agent gateway via the Bluetooth Low Energy (BLE) protocol stack. The edge intelligent agent gateway (whose physical form can be a smartphone, tablet, or professional industrial-grade IoT embedded gateway device with a dedicated medical application deployed) not only undertakes the protocol routing function for underlying data forwarding in the entire architecture, but also possesses first-level edge computing and shallow feature analysis capabilities. Internally, the edge intelligent agent gateway utilizes a lightweight artificial intelligence inference engine mounted on the operating system's background service thread. This engine first buffers and segments the received continuous audio data stream into millisecond-level overlapping time-domain slices, and extracts the time-domain short-time energy characteristics, zero-crossing rate physical characteristics, and frequency-domain Mel-Frequency Cepstral Coefficients (MFCC) acoustic characteristics of each audio slice. By performing matrix classification analysis on these low-dimensional, shallow acoustic features, the edge agent gateway can quickly calculate whether the currently collected audio belongs to valid human physiological sounds, whether it is accompanied by severe detachment noise caused by device detachment, and estimate basic heart rate indicators. This structured data obtained through terminal computing power constitutes the edge-side inference result. Subsequently, to solve the network timing disconnect problem caused by the independent transmission of audio and video media streams and device control signaling streams in traditional telemedicine solutions, the edge agent gateway performs a deep data packaging operation. The program code combines the physiological sound audio data representing the original mechanical waveform, the edge-side inference result representing the preliminary artificial intelligence diagnostic conclusion, and the collaborative session identifier representing the current global task context attribute by modifying the underlying structure of the multimedia streaming media encapsulation protocol to form an integrated media data stream. After packaging the data in memory, the edge intelligent agent gateway establishes two logically independent wide area network links using a dual-protocol layered communication architecture: the first link uses the high-bandwidth media stream channel, which supports dynamic packet loss retransmission and congestion control, to push the large volume of media data to the streaming media server cluster in the cloud center for real-time one-way streaming; the second link uses the low-latency, high-reliability signaling channel to encapsulate status events, including the current battery level of the smart stethoscope, the real-time connection status of BLE Bluetooth, and the recording control action triggered by medical staff pressing physical buttons, into the control command message, and forcibly associates and binds the collaborative session identifier with a key-value pair in the header of the command message, and then publishes it to the message queue server in the cloud center.This design, which separates the high-throughput data stream and the high-reliability signaling stream at the physical link layer and decouples and recouples them at the logical service layer through a unified identifier, effectively improves the transmission robustness of the device in complex 4G / 5G / Wi-Fi wide area network environments.

[0049] In one optional implementation, encapsulating the physiological audio data, the edge-side inference result, and the collaborative session identifier to form a media data stream includes: using implicit watermarking technology to write the collaborative session identifier into the least significant bit of the physiological audio data to preserve high-order acoustic features; writing the edge-side inference result, the collaborative session identifier, sampling parameters, device number, and auscultation site timestamp into the extended header of the real-time transmission protocol to form the media data stream; the media stream channel is a transmission channel using a real-time audio and video streaming media protocol, and the signaling channel is a transmission channel using a publish-subscribe mechanism.

[0050] Regarding the above-mentioned optional implementation methods, the step of encapsulating the physiological audio data, the edge-side inference result, and the collaborative session identifier to form a media data stream, transmitting the media data stream to the cloud center through the media stream channel, and simultaneously transmitting the control commands bound to the collaborative session identifier to the cloud center through the signaling channel, further includes the following sub-steps: S1031. To achieve deep integration between the underlying unformatted waveform data and the upper-level medical business logic, enabling reverse tracing of audio files even when they are removed from the cloud network environment, exported for offline viewing, or used for long-term medical record archiving, the edge intelligent agent gateway employs implicit digital watermarking technology to perform bit-level modification processing on the 16-bit PCM format physiological sound audio data before network packetization. Specifically, the watermark embedding algorithm scans the 16 binary bits of each audio sampling point in memory and extracts its least significant bit (LSB) through bitmasking operations. After the gateway program converts the feature hash code representing the unique identity of the collaborative session identifier into a binary bit stream, it replaces the original LSB bit value of the PCM signal with a preset pseudo-random low-density interval (e.g., selecting a target bit every 10 or 20 sampling points). Due to the masking effect of the human auditory system and the insensitivity of routine clinical auscultation to the minute quantization error caused by the least significant bit under 16-bit quantization precision, this write operation that directly modifies the underlying binary machine data effectively embeds the traceability identifier into the audio waveform structure and fully preserves all the minute pathological acoustic features carried by the first 15 high bits, thus achieving secure steganographic binding between the audio waveform and the session identifier.

[0051] S1032. After completing the underlying watermark embedding, the edge agent gateway initiates the data packet protocol layer assembly process based on the Real-time Transport Protocol (RTP). In the international standard RTP packet header structure, in addition to the fixed payload area for storing the watermarked PCM audio binary data, the protocol specification also reserves a variable-length extension header area. The edge agent gateway uses software code, through a custom Type-Length-Value (TLV) binary encoding structure, to sequentially write the edge-side inference result text processed by JSON serialization, the plaintext string copy of the collaborative session identifier, the sampling rate and bit depth parameters of the current ADC conversion, the factory-fixed physical device number of the smart stethoscope (such as the Media Access Control address, MAC Address), and the timestamp of the auscultation site, synchronized from the system's underlying clock and accurate to the microsecond level, into the extension header memory space of the RTP protocol. By modifying the extension header, the standard single-dimensional RTP stream is upgraded to the media data stream containing multi-dimensional medical metadata context.

[0052] S1033. After completing the double-layer encapsulation (watermark layer and RTP extension header layer), the edge intelligent agent gateway calls the system network socket interface (Socket) to establish two parallel remote transmission sessions at the network layer. For the media data stream carrying a large amount of continuous multi-dimensional information, the program uses an audio and video streaming media protocol based on the Web Real-Time Communication (WebRTC) open-source architecture as the media stream channel. This channel runs on top of the connectionless User Datagram Protocol (UDP), abandoning the handshake and retransmission delay of traditional TCP. At the same time, in conjunction with WebRTC's built-in Adaptive Bitrate (ABR) and network congestion control buffer algorithm at the application layer, even when facing severe network jitter in patients' homes or primary clinics, the core audio stream can be pushed unidirectionally to the cloud center with a low end-to-end latency (e.g., strictly controlling the latency within the audible threshold of 300 milliseconds).

[0053] S1034. Simultaneously, for control commands with low communication volume but extremely high reliability requirements, the edge agent gateway establishes another parallel signaling channel via the TCP protocol. This signaling channel employs a lightweight publish / subscribe mechanism such as Message Queuing Telemetry Transport (MQTT). The edge agent gateway uses the collaborative session identifier as a subscription topic at the MQTT protocol routing layer, serializing and publishing service information such as device startup, pause, mode switching status, and keep-alive signals to this specific topic. The signaling service server in the cloud center subscribes to this corresponding topic through the message broker, utilizing QoS 1 or QoS 2 (Quality of Service Level) mechanisms to receive all command events and actions bound to this collaborative session in real-time, stably, and without data loss, achieving complementary advantages and separate transmission of heterogeneous data streams on the physical channel.

[0054] In one optional implementation, the edge agent gateway has a built-in lightweight multilayer perceptron network model; the step of performing feature analysis on the physiological audio data to obtain the edge-side inference result includes: inputting the physiological audio data into the lightweight multilayer perceptron network model for shallow feature perception, identifying basic physiological indicators, including auscultation location on the body surface, baseline audio noise, baseline heart rate and respiratory rate, and using the identified basic physiological indicators as the edge-side inference result.

[0055] Regarding the above-mentioned optional implementation methods, the step of the edge intelligent agent gateway performing feature analysis on physiological audio data to obtain edge-side inference results further includes the following sub-steps: S1035. Considering that edge devices performing forwarding tasks (such as smartphones or portable medical gateways) are typically powered by lithium batteries and have relatively limited central processing unit computing power, algorithm developers pre-load a lightweight multilayer perceptron (MLP) feedforward neural network model that has undergone deep quantization compression (e.g., quantization from 32-bit floating-point to 8-bit integer) and structural pruning optimization into the application storage space of the edge agent gateway. This model has a streamlined structure, containing only a small number of fully connected hidden layers, resulting in a small total number of parameters. The number of floating-point operations (FLOPs) required for one forward inference operation is strictly controlled within the low load range allowed by the mobile device processor's power consumption and heat generation.

[0056] S1036. During the auscultation task, the data cache queue allocated in the internal memory of the edge agent gateway continuously collects the physiological audio data entering via Bluetooth, and uses a sliding window mechanism to divide it into short data frames of fixed length (e.g., 3 seconds each) with 50% temporal overlap. Then, using Fast Fourier Transform and Mel filter bank, the log-Mel spectrogram of each audio data frame is extracted and flattened into a one-dimensional feature vector.

[0057] S1037. The extracted one-dimensional feature vector is used as the input tensor matrix and directly fed into the input layer of the lightweight multilayer perceptron network model. Through matrix multiplication and nonlinear activation functions (such as rectified linear units, ReLU function) in each hidden layer, rapid shallow acoustic feature perception extraction is achieved. Based on the feature classification boundaries pre-trained on a large-scale clean and noisy physiological sound dataset in the cloud, the output layer of this MLP model can output parallel multi-dimensional probability distribution nodes through the Softmax function.

[0058] S1038. Based on the probabilities of each classification node in the output layer of the MLP model, combined with a preset hard threshold, the edge device software logic identifies basic physiological indicators. The specific identification process includes: using a pre-trained anatomical location feature classifier to determine whether the current pickup diaphragm is attached to the left sternal border or the apex of the heart (obtaining the auscultation location on the body surface); assessing the probability of non-periodic abnormal energy peaks appearing in the current 3-second audio segment (obtaining the baseline noise level of the audio); extracting the audio envelope signal, using a simple time-domain autocorrelation function to find periodic peaks, and combining the periodic feature probabilities output by the MLP network to estimate the number of heartbeats per minute (obtaining the baseline heart rate) and the rhythmic cycle of lung respiration per minute (obtaining the respiratory rate).

[0059] S1039. After completing the above low-computing-power analysis, the code logic of the edge intelligent agent gateway combines the identified four-dimensional basic physiological indicator values ​​in a structured key-value pair format to generate a standard JSON text string, and uses the identified basic physiological indicators, including location, noise ratio, heart rate, and respiratory rate, as the final edge-side inference result. Subsequently, this JSON string is sent to the RTP extension header encapsulation process described in step S1032. This method of distributing high-frequency computing pressure to terminal devices (i.e., computing power offloading mode) effectively reduces the computing pressure and network bandwidth bottlenecks faced by cloud server clusters when simultaneously handling tens of thousands of concurrent long connections.

[0060] S104, the cloud center, based on the collaborative session identifier, performs dual-link timing alignment of the received media data stream and the received control commands, and combines the edge-side inference results to perform deep inference on the timing-aligned physiological audio-visual data to obtain auxiliary diagnosis and treatment decision results.

[0061] Under the aforementioned dual-protocol layered communication architecture, the media data stream carrying the underlying raw audio waveform and edge screening features (via a connectionless WebRTC channel) and the control commands carrying the stethoscope device status and medical staff click interactions (via an MQTT signaling channel with a guaranteed arrival mechanism) will eventually reach the server firewall in the cloud center at different physical time points and through complex network nodes with drastically different routing paths. Due to the inherent connectionless and low-latency characteristics of the UDP protocol and the different handshake times of the MQTT protocol's handshake mechanism for quality assurance retransmission confirmation based on the Transmission Control Protocol, there are objectively millisecond-level or even tens of seconds of network physical time difference and out-of-order phenomena when these two heterogeneous data streams arrive at the cloud receiving queue.

[0062] To avoid logical misalignment between the waveform display timeline seen by remote consultation experts on the screen and the actual received business instructions such as "start auscultation," the collaborative consultation scheduling engine deployed in the cloud center must perform an alignment operation. This engine first extracts a common retrieval key, namely the collaborative session identifier, from the data packets of the two physical receiving channels, and then dynamically constructs a circular jitter buffer queue based on this identifier in the system's high-frequency memory for that session.

[0063] The engine compares the microsecond-level timestamps parsed from the RTP extension header with the command occurrence timestamps carried in the MQTT signaling payload. The sorting algorithm in the column reorders and caches out-of-order packets and signaling caused by network jitter, and uses the difference for corresponding delay compensation while waiting. Ultimately, it accurately restores the separated dual-link data to the strict timing relationship at the moment of its generation at the patient's location, completing dual-link timing alignment. After timing alignment, the original audio waveform data, edge MLP initial screening indicators, and device control status are converged in memory, forming a spatiotemporally consistent digital slice.

[0064] The business logic of the cloud center invokes its deep inference agent container deployed on a high-speed computing cluster of Graphics Processing Units (GPUs). This agent is a fusion-type artificial intelligence model with a huge number of parameters and a complex network topology. It not only deeply processes the micro-features of diastolic or systolic murmurs that are extremely difficult to detect in the physiological audio data, but also uses the edge-side inference results extracted from the preprocessing (such as estimating heart rate to assist in judging the duration of the cardiac cycle, and auscultation location to assist in focusing on specific valvular lesions) as prior attention prompts or encoded bias vectors to input into specific layers of the deep network for multi-dimensional cross-modal feature cross-validation and constraint compensation.

[0065] Through complex forward propagation calculations, the probability of specific pathological diagnoses such as mitral stenosis, ventricular septal defect, and pneumonia is derived, generating the auxiliary diagnostic decision-making results that cover clear disease subtyping and clinical severity grading. These results are then serialized into report data (graphics and images) and pushed in real-time to a remote expert's visualization terminal application, allowing the expert to review the medical findings in conjunction with the waveforms.

[0066] In one optional implementation, the step of combining the edge-side inference results to perform deep inference on the temporally aligned physiological audio-visual data to obtain auxiliary diagnostic and treatment decision results includes: inputting the temporally aligned physiological audio-visual data and the edge-side inference results into a dual-channel parallel network model; extracting short-term local pathological features of the physiological audio-visual data from a one-dimensional convolutional neural network in the dual-channel parallel network model, and extracting long-term temporal rhythm dependence features of the physiological audio-visual data from a multi-head attention mechanism in the dual-channel parallel network model; fusing the short-term local pathological features, the long-term temporal rhythm dependence features, and the edge-side inference results to mine higher-order features of complex pathological murmurs, and generating the auxiliary diagnostic and treatment decision results containing disease classification and disease risk grading based on the higher-order features of complex pathological murmurs.

[0067] Regarding the above-mentioned optional implementation methods, the step of performing deep inference on the time-aligned physiological audio-visual data based on the edge-side inference results to obtain auxiliary diagnostic and treatment decision results further includes the following sub-steps: S1041. To overcome the limitations of single deep learning networks (such as traditional CNNs, which easily lose long temporal relationships, and traditional RNNs, which have low parallel computation efficiency) when processing complex auscultation signals that often last tens of seconds, contain long respiratory cycles, and are interspersed with high-frequency transient short-term pathological murmurs (such as the click of mitral valve prolapse), the cloud center instantiates a specific composite deep learning architecture container, namely the dual-channel parallel network model. This model sets up two parallel independent branch extraction structures in the tensor layer at the input end. The data preprocessing module of the cloud center performs dimension adaptation and tensor alignment operations on the temporally aligned long-sequence high-dimensional one-dimensional audio waveform tensor (i.e., the physiological audio data) and the low-dimensional numerical structure vector (i.e., the edge-side inference result), and then synchronously inputs them into the bottom input layer of the dual-channel parallel network model.

[0068] S1042. In the first channel branch on the left side of the model, a one-dimensional convolutional neural network (1D-CNN) based on a one-dimensional convolution operator is deployed. This network layer uses a one-dimensional convolution kernel matrix with different receptive field sizes (e.g., sizes of 3, 5, and 7) to perform multi-step sliding scan inner product calculations along the time axis of the audio waveform tensor. Due to the local receptive field characteristics of the convolution operation itself, it has extremely high activation sensitivity to abrupt spikes and high-frequency sudden friction noises in one-dimensional time signals. After feature extraction by stacked multi-layer convolutional layers, batch normalization processing to accelerate convergence, and max pooling layer feature compression operations for dimensionality reduction, the 1D-CNN converts the waveform into a feature map, successfully extracting features such as the contour of a fine diamond-shaped noise waveform in the middle of contraction and intermittent fragments of high-frequency wheezing, that is, extracting the short-term local pathological features of the physiological audio data.

[0069] S1043. In the second channel branch on the right side of the model, a core component based on the Transformer architecture—the Multi-Head Attention Mechanism—is deployed in parallel. The mathematical core of this mechanism is to map the audio sequence into a query (Q), key (K), and value (V) matrix, and to calculate the dot product similarity matrix of the waveform feature vectors at any two time points in the sequence. The core calculation formula of the Multi-Head Attention Mechanism is:

[0070] In the formula, This represents the query feature tensor matrix generated from the input sequence, used to represent the target features of interest. The key feature tensor matrix is ​​used to represent the features at all positions in the sequence for matching. The representative feature tensor matrix contains the actual transmitted feature information; The dimension scalar representing the key vector is used to scale the dot product result and prevent gradient vanishing. This represents performing a mathematical transpose operation on the matrix; This represents the normalized exponential function, used to transform the dot product score into an attention weight probability distribution.

[0071] By calculating this global probability weight matrix, this mechanism breaks through distance limitations and can directly capture the interdependencies of signals over extremely long time series. For example, it can calculate and track the minute changes in the interval between the first and second heart sounds over tens of seconds through high-dimensional mapping of features at different times, or discover the alternating patterns of audio energy ratios between the inspiratory and expiratory phases in a long respiratory cycle, thereby extracting macroscopic, holistic cardiopulmonary circulation characteristics that transcend a single cardiac cycle, i.e., extracting the long-term temporal rhythm dependence features of the physiological audio data.

[0072] S1044. At the end of the parallel extraction of dual channels, the network architecture sets up a high-dimensional feature fusion layer. The algorithm engine performs vector concatenation operation on the channel dimension between the local short-term feature tensor matrix output by 1D-CNN and the global long-term circadian rhythm feature tensor matrix output by Transformer. At this time, as a key constraint, the model incorporates the edge-side inference results. The system encodes the discrete indicators such as the "apex" auscultation location and estimated heart rate obtained from the edge screening into continuous real-number vectors through a multi-layer fully connected network, and uses them as attention gating signals. The gating signal is used to perform adaptive element-wise multiplication on the concatenated multidimensional fusion feature matrix. This constrained fusion operation allows the deep network to combine the patient's prior anatomical location information on the body surface (e.g., logically, if the prior location is the apex, the gating network will automatically amplify the attention weight value of the systolic murmur channel feature vector). Through such complex nonlinear combinations, the network eventually mines the high-order mapping expression of hidden lesions caused by the interweaving of multiple pathological factors in the feature manifold of the latent space, that is, it extracts the high-order features of the complex pathological noise.

[0073] S1045. After extracting the high-order feature tensor representing the deep pathological mapping, it is input into the fully connected classifier head containing the Softmax activation function. The network architecture ultimately outputs two independent tensor data structures: one is a one-dimensional probability vector representing the probability of occurrence of different specific disease categories (e.g., the data structure shows: congenital heart disease with a confidence level of 90%, myocarditis with a confidence level of 5%); the other is a classification scalar index representing the degree of life-threatening danger posed by this type of disease (e.g., mapped to three discrete levels: high risk, intermediate risk, and low risk). The business layer logic of the cloud center performs textual and structured parsing of these two tensor outputs to generate a standard format electronic medical document report, which includes the auxiliary diagnosis and treatment decision-making results containing disease classification qualitative analysis and disease risk grading warning, thus completing the deep reasoning task.

[0074] In one optional implementation, the dual-channel parallel network model is configured with adaptive reinforcement learning reward logic, which drives the dual-channel parallel network model to perform online autonomous iterative optimization. Specifically, the adaptive reinforcement learning reward logic includes: obtaining expert calibration matching factors, model output confidence coefficients, previous comprehensive training loss values, and difficult sample bias coefficients; assigning preset weighted hyperparameters to the expert calibration matching factors, model output confidence coefficients, previous comprehensive training loss values, and difficult sample bias coefficients respectively, and then performing weighted calculations to obtain a comprehensive adaptive reward value; based on the comprehensive adaptive reward value, classifying the samples corresponding to the physiological audio data into a positive reward sample set or a negative reward sample set, and completing incremental rolling training of the dual-channel parallel network model based on the positive reward sample set and the negative reward sample set.

[0075] Regarding the aforementioned optional implementation methods, after obtaining the assisted diagnosis and treatment decision results, to enable the system to continuously learn and adapt to new clinical diseases, a further sub-step of online autonomous iterative optimization is included. Its core utilizes adaptive reinforcement learning (RL) reward evaluation logic. The mathematical core of this feedback evaluation logic can be rigorously expressed as a multivariate linear combination reward function calculation formula:

[0076] The corresponding optimization process is as follows: S1046, The training evaluation scheduler in the cloud center collects four independent key evaluation parameters required to drive the adaptive reinforcement learning reward logic from different business databases and forward computation processes.

[0077] The first step involves obtaining the medical diagnosis conclusion text, finalized and signed by the attending physician after reviewing the AI-assisted report, following remote expert verification. The algorithm uses natural language processing to extract intent and compares it against the classification results provided by the model. If the disease categories match, an assignment is made. (Expert-calibrated matching factor) Boolean value is 1, otherwise it is assigned 0 if there is an error; The second step is to extract the maximum probability value of the highest-ranking classification from the output tensor of the Softmax layer at the end of the dual-channel model, and use it as... (The model outputs a confidence coefficient, whose value is strictly limited to a real number between 0 and 1.) This coefficient reflects the model's confidence in its judgment. The third step is to extract the scalar values ​​from the backpropagation logs of the model training framework during the previous mini-batch training update process. (Previous comprehensive training loss value), this loss value is a weighted fusion of the CNN's feature fitting mean squared error (MSE) loss, the Transformer's temporal rhythm bias loss, and the final classification cross-entropy loss, reflecting the error level of the current network parameters; the fourth step uses anomaly detection algorithms such as Isolation Forest to evaluate the complexity spectral features and environmental signal-to-noise ratio of the background noise in the current original audio sample. If the system evaluation determines that the audio belongs to an early hidden small lesion or a complex multiple noise under heavy interference, it is assigned a higher value. (Difficult sample bias coefficient) is used to quantify the difficulty of recognizing the sound sample in the machine learning dimension.

[0078] S1047. After successfully obtaining the above four independent dimension parameters, the reinforcement learning algorithm assigns these four variables pre-defined weighted hyperparameters, determined through cross-validation on a large-scale medical physiological acoustic test dataset containing millions of samples, using the grid search algorithm. Specifically, for the weight control of different dimensions, (Identify and match weighting factors to control the dominance of expert opinions) (Confidence weighting factor, to encourage the model to output confidently) (Loss function penalizes weighting factor, controls parameter divergence) (The weighted factor for punishment of difficult cases, which is lenient for misjudging difficult cases) are hardcoded into the code to be specific empirical values ​​(e.g., parameter sets determined through convergence verification in the benchmark test set). , , , The cloud-based GPU tensor computation engine calls the above parameters to perform linear weighted arithmetic algebra operations, ultimately calculating a continuous real value. This refers to the comprehensive adaptive reward value representing the performance score of a single inference. Mathematically, this reward value positively rewards the judgment recognized by medical experts through an additive term, encouraging the model to produce high-confidence probability outputs under clear features; simultaneously, it penalizes excessively large previous training errors caused by parameter non-convergence through a subtractive term, and for objectively difficult-to-identify rare diseases, it subtracts a larger amount... This effectively reduces the overall severity of the system's punishment for any potential misjudgment.

[0079] S1048. The reinforcement learning environment engine sets a decision threshold for sample splitting (e.g., an empirical threshold of 0.60). For each segment of specific physiological audio data generated during a consultation, the engine determines and calculates the comprehensive adaptive reward value. Does the condition meet? When the value exceeds the set classification threshold, it indicates that the model has high positive learning value in this judgment logic and feature extraction. The classification algorithm automatically packages the desensitized sound tensor sample and its corresponding expert diagnosis label into a training pair and adds it to the positive reward sample set in the memory; conversely, if the value is less than the set threshold, the model will not perform well. If the result is below this threshold or a critical expert veto occurs, that is... When the value is 0, it indicates that the current network structure has a judgment bias or severe overfitting to this type of feature. The classification algorithm will strip the sample and mark it with a bias category, and include it in the negative reward sample set used to correct the suppression.

[0080] S1049. During idle periods when the cloud server is experiencing low concurrency requests (e.g., set in a nighttime scheduled task queue), the reinforcement learning environment configuration manager is automatically woken up. The training script uses the batch data in the positive reward sample set as the positive gradient guidance direction for optimizing the model parameters (i.e., maximizing the correct classification probability), while using the correction data in the negative reward sample set as the suppression target to limit model divergence (i.e., minimizing the probability of incorrect paths).

[0081] By constructing a reinforcement learning loss feedback strategy based on policy gradient balancing, backpropagation is performed using the Adam optimizer or stochastic gradient descent algorithm. The optimizer adaptively updates the weight matrix values ​​of each convolutional kernel in the 1D-CNN and the attention mask weight matrix in the Transformer multi-head structure with a small set learning rate based on the calculated gradient vector. This mechanism effectively overcomes the drawbacks of traditional static medical models, which rely on manual, periodic data collection and time-consuming, fully offline retraining. Architecturally, it achieves automated incremental training throughout the model's lifecycle, enabling the dual-channel parallel network model to autonomously and iteratively optimize its classification accuracy and generalization ability for complex physiological sounds as cloud-based medical record data continuously increases.

[0082] In one optional implementation, after obtaining the assisted diagnosis and treatment decision result, the method further includes a step of hierarchical data storage based on the collaborative session identifier: archiving the physiological audio data in the media data stream to a persistent cloud database for storage; caching the edge-side inference result and the assisted diagnosis and treatment decision result to a high-speed database of the edge intelligent agent gateway; obtaining expert review opinions and final diagnostic conclusions, and archiving the expert review opinions and final diagnostic conclusions to a main medical business database; and linking and anchoring the heterogeneous data in the persistent cloud database, the high-speed database, and the main medical business database through the collaborative session identifier.

[0083] Regarding the above-mentioned optional implementation methods, after obtaining the auxiliary diagnosis and treatment decision results, in order to meet the subsequent needs of medical record review and legal evidence presentation, a further sub-step of data hierarchical archiving and storage is included, the specific process of which is as follows: S1050. To meet the compliance requirements of the National Health and Health Administration for archiving and retaining electronic medical record attachments for 15 years or even longer, and to match different interface call times with different data volumes, the system executes a differentiated hierarchical storage routing strategy. First, for the physiological audio data in the original lossless PCM format that occupies the largest storage capacity and contains the patient's core and rich acoustic vital signs information (it should be noted that this part of the underlying data has already been physically embedded with an implicit watermark containing unique tracking information using bit operations in step S1031 above, and the RTP protocol shell data used for network transmission has been stripped at the parsing layer), the scheduling system writes it in batches as binary large objects (Blob) into the large-capacity, low-speed, and low-cost cloud persistent cold backup database for long-term cold data security storage through the Object Storage Service (OSS) module application interface or a distributed file system architecture configured with erasure coding technology (such as Hadoop Distributed File System, HDFS).

[0084] S1051. Simultaneously, the edge-side inference results, such as heart rate estimates and baseline murmur probability, are generated on-site by the edge intelligent agent gateway, along with the structured auxiliary diagnosis and treatment decision results (in JSON format, containing specific disease risk grading and disease codes) generated by a complex GPU model in the cloud. Considering that this pre-diagnosis inference data will be accessed frequently during subsequent early clinical rounds or as a data source for incremental model training in step S1049, the system utilizes in-memory key-value pair caching database technologies such as Redis or Memcached to temporarily cache these data in a high-throughput, high-speed database deployed on the network edge physical gateway device or a high-speed proxy server in the cloud, to meet the extremely low-latency, high-concurrency call requirements of front-end doctor tablets or model training scripts.

[0085] S1052. After reviewing the original waveforms transmitted from the patient's location and the auxiliary reports generated in the cloud via the operating interface, the attending physician at the remote consultation center finalizes the prescription or inputs corrections to the AI ​​results on the hospital's information system terminal. The application system's front-end interface module intercepts this high-value gold standard information with the highest medical legal effect, namely the expert review opinion and the signed final approved diagnosis conclusion. The back-end business code archives and inserts this text-based data, which has extremely strong logical relational attributes, strictly in accordance with the transactional database specifications that support atomicity, consistency, isolation, and durability, into the main medical business database (usually a relational database with a strict table structure such as MySQL or Oracle) connected to the hospital's core information system (HIS) server.

[0086] S1053. The three categories of medical data generated above are stored in three different physical storage media or logical database carriers with different physical locations of origin (patient end, edge end / cloud end AI, doctor end) and different data structures (binary audio stream, JSON text, relational records) due to their different data structures (binary audio stream, JSON text, relational records). Since the architecture of this invention has pre-established a globally unified identification mechanism in the initial S101 step, when the underlying code executes all three data writing and database commit operations, the storage engine program of each level of the database will force the UUID to be written into the record as a foreign key constraint or composite primary key index field in the database table structure. In this way, at any time in the future, the system server only needs to receive and execute a cross-table, cross-database joint query request command carrying the single collaborative session identifier (e.g., using SQL JOIN statement or cross-database middleware API query), and the central scheduling platform can accurately extract and seamlessly combine the original acoustic underlying waveforms, AI algorithm inference and judgment graphs, and the expert's final medical order text across the cloud object storage pool, distributed memory cache array, and intranet relational database. This architecture effectively anchors all spatially and format-dispersed heterogeneous diagnostic and treatment data by leveraging the invisible digital link of collaborative session identifiers. This design breaks down data silos in traditional medical systems, effectively solves the problem of difficult panoramic playback and reconstruction caused by severe fragmentation of historical data in telemedicine services, and ensures the integrity of medical traceability.

[0087] This invention also provides a remote collaborative auscultation data processing system based on dual-link timing synchronization, referring to... Figure 2 The diagram illustrates a functional block diagram of a remote collaborative auscultation data processing system 200 based on dual-link timing synchronization according to the present invention. The system may include the following modules: The collaborative session and parameter distribution module 201 is used to establish a collaborative session and generate a collaborative session identifier. The cloud center generates auscultation signal acquisition parameters based on the collaborative session identifier and distributes the auscultation signal acquisition parameters to the edge intelligent agent gateway and the intelligent electronic stethoscope. The preprocessing and output module 202 is used to collect physiological signals based on the auscultation signal acquisition parameters through the intelligent electronic stethoscope, perform preprocessing, and output physiological audio data. The initial screening and dual-link transmission module 203 is used to perform feature analysis on physiological audio data through the edge intelligent agent gateway to obtain edge-side inference results, encapsulate physiological audio data, edge-side inference results and collaborative session identifier to form a media data stream, transmit the media data stream to the cloud center through the media stream channel, and transmit control commands bound to the collaborative session identifier to the cloud center through the signaling channel. The timing alignment and inference module 204 is used to perform dual-link timing alignment of the received media data stream and the received control commands based on the collaborative session identifier through the cloud center, and to perform deep inference on the timing-aligned physiological audio data in combination with the edge-side inference results to obtain auxiliary diagnosis and treatment decision results.

[0088] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0089] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0090] Those skilled in the art will understand that all or part of the steps in the above method can be implemented by a program instructing related hardware. The program or the program described therein can be stored in a computer-readable storage medium. When the program is executed, it includes the following steps: at this time, the corresponding method steps are introduced. The storage medium can be ROM / RAM, magnetic disk, optical disk, etc.

[0091] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A remote collaborative auscultation data processing method based on dual-link time synchronization, characterized in that, include: A collaborative session is established and a collaborative session identifier is generated. The cloud center generates auscultation signal acquisition parameters based on the collaborative session identifier and sends the auscultation signal acquisition parameters to the edge intelligent agent gateway and the intelligent electronic stethoscope. The intelligent electronic stethoscope collects physiological signals according to the auscultation signal acquisition parameters and performs preprocessing, then outputs physiological audio data. The edge intelligent agent gateway performs feature analysis on the physiological audio data to obtain edge-side inference results, encapsulates the physiological audio data, the edge-side inference results, and the collaborative session identifier to form a media data stream, transmits the media data stream to the cloud center through the media stream channel, and simultaneously transmits control commands bound to the collaborative session identifier to the cloud center through the signaling channel. Based on the collaborative session identifier, the cloud center performs dual-link timing alignment of the received media data stream and the received control commands, and combines the edge-side inference results to perform deep inference on the timing-aligned physiological audio data to obtain auxiliary diagnosis and treatment decision results.

2. The remote collaborative auscultation data processing method based on dual-link timing synchronization according to claim 1, characterized in that, The auscultation signal acquisition parameters are generated based on the clinical acquisition mode, which includes pure heart sound mode, pure lung sound mode, and simultaneous heart and lung sound acquisition mode. When the clinical acquisition mode is the pure heart sound mode, the effective frequency band parameter of the auscultation signal acquisition parameter configuration is 20 Hz to 200 Hz; When the clinical acquisition mode is the pure lung sound mode, the effective frequency band parameter of the auscultation signal acquisition parameter configuration is 100 Hz to 500 Hz; When the clinical acquisition mode is the cardiopulmonary simultaneous acquisition mode, the effective frequency band parameter of the auscultation signal acquisition parameter configuration is 20 Hz to 2000 Hz.

3. The remote collaborative auscultation data processing method based on dual-link timing synchronization according to claim 1, characterized in that, The physiological signals include cardiopulmonary audio signals and low-frequency vibration signals; the preprocessing includes sequentially performing anti-aliasing pre-normalization, baseline drift correction, noise suppression, adaptive resampling, and pathological feature targeted enhancement. The anti-aliasing pre-normalization employs a two-stage combined analog and digital filter to remove aliasing clutter. The baseline drift correction is used to remove low-frequency noise and preserve the effective signal base waveform; The noise suppression is used to compress noise gain without signal amplitude distortion. The adaptive resampling is used to perform downsampling processing based on the auscultation signal acquisition parameters; The pathological feature targeted enhancement is used to amplify the gain of a preset pathological frequency band using an envelope extraction algorithm, and locks the temporal phase of the physiological audio-visual data during the gain amplification process.

4. The remote collaborative auscultation data processing method based on dual-link timing synchronization according to claim 1, characterized in that, The process of encapsulating the physiological audio data, the edge-side inference results, and the collaborative session identifier to form a media data stream includes: The collaborative session identifier is written into the least significant bit of the physiological audio data using implicit watermarking technology to preserve high-bit acoustic features; The edge-side inference result, the collaborative session identifier, sampling parameters, device number, and auscultation site timestamp are written into the extended header of the real-time transmission protocol to form the media data stream; The media streaming channel is a transmission channel using a real-time audio and video streaming media protocol, and the signaling channel is a transmission channel using a publish-subscribe mechanism.

5. The remote collaborative auscultation data processing method based on dual-link timing synchronization according to claim 1, characterized in that, The edge agent gateway has a built-in lightweight multilayer perceptron network model. The feature analysis of the physiological audio data to obtain the edge-side inference result includes: The physiological audio data is input into the lightweight multilayer perceptron network model for shallow feature perception to identify basic physiological indicators, including auscultation location on the body surface, baseline audio noise, baseline heart rate and respiratory rate, and the identified basic physiological indicators are used as the edge-side inference results.

6. The remote collaborative auscultation data processing method based on dual-link timing synchronization according to claim 1, characterized in that, The process of combining the edge-side inference results with the temporally aligned physiological audio data to perform deep inference and derive auxiliary diagnostic and treatment decision-making results includes: The time-aligned physiological audio data and the edge-side inference results are input into a dual-channel parallel network model; The short-term local pathological features of the physiological audio data are extracted by the one-dimensional convolutional neural network in the dual-channel parallel network model, and the long-term temporal rhythm dependence features of the physiological audio data are extracted by the multi-head attention mechanism in the dual-channel parallel network model. The short-term local pathological features, the long-term rhythm-dependent features, and the marginal inference results are fused to mine higher-order features of complex pathological murmurs, and the auxiliary diagnosis and treatment decision results containing disease subtyping and disease risk grading are generated based on the higher-order features of complex pathological murmurs.

7. The remote collaborative auscultation data processing method based on dual-link timing synchronization according to claim 6, characterized in that, The dual-channel parallel network model is configured with adaptive reinforcement learning reward logic, which is used to drive the dual-channel parallel network model to perform online autonomous iterative optimization. The adaptive reinforcement learning reward logic specifically includes: obtaining expert calibration matching factors, model output confidence coefficient, previous comprehensive training loss value, and difficult sample bias coefficient; assigning preset weighted hyperparameters to the expert calibration matching factors, the model output confidence coefficient, the previous comprehensive training loss value, and the difficult sample bias coefficient respectively, and then performing weighted calculations to obtain a comprehensive adaptive reward value. Based on the comprehensive adaptive reward value, the samples corresponding to the physiological audio data are assigned to either a positive reward sample set or a negative reward sample set, and incremental rolling training of the dual-channel parallel network model is completed based on the positive reward sample set and the negative reward sample set.

8. The remote collaborative auscultation data processing method based on dual-link timing synchronization according to claim 1, characterized in that, After obtaining the auxiliary diagnosis and treatment decision results, the method further includes a step of hierarchical data storage based on the collaborative session identifier: The physiological audio data in the media data stream is archived and stored in a persistent cloud database. The edge-side inference results and the auxiliary diagnosis and treatment decision results are cached in the high-speed database of the edge agent gateway; Obtain expert review opinions and final diagnostic conclusions, and archive the expert review opinions and final diagnostic conclusions into the main medical business database; The heterogeneous data in the cloud persistent database, the high-speed database, and the medical business master database are associated and anchored through the collaborative session identifier.

9. The remote collaborative auscultation data processing method based on dual-link timing synchronization according to claim 1, characterized in that, During the process of sending the auscultation signal acquisition parameters to the edge agent gateway and the intelligent electronic stethoscope, a hierarchical cyclic redundancy check mechanism based on the collaborative session identifier is executed: Generate a global cyclic redundancy check code bound to the collaborative session identifier, and assign independent sub-cyclic redundancy check codes to different levels of parameters in the auscultation signal acquisition parameters; When the edge agent gateway detects a verification anomaly based on the global cyclic redundancy check (CRC) code or the sub-CRC code, it automatically triggers the retransmission mechanism for the auscultation signal acquisition parameters.

10. A remote collaborative auscultation data processing system based on dual-link time synchronization, characterized in that, It includes multiple modules for performing the remote collaborative auscultation data processing method based on dual-link timing synchronization as described in any one of claims 1 to 9, specifically including: The collaborative session and parameter distribution module is used to establish a collaborative session and generate a collaborative session identifier. The cloud center generates auscultation signal acquisition parameters based on the collaborative session identifier and distributes the auscultation signal acquisition parameters to the edge intelligent agent gateway and the intelligent electronic stethoscope. The preprocessing and output module is used to collect physiological signals through the intelligent electronic stethoscope according to the auscultation signal acquisition parameters, perform preprocessing, and output physiological audio data. The initial screening and dual-link transmission module is used to perform feature analysis on the physiological audio data through the edge intelligent agent gateway to obtain edge-side inference results, encapsulate the physiological audio data, the edge-side inference results and the collaborative session identifier to form a media data stream, transmit the media data stream to the cloud center through the media stream channel, and simultaneously transmit control commands bound to the collaborative session identifier to the cloud center through the signaling channel. The timing alignment and inference module is used to perform dual-link timing alignment of the received media data stream and the received control command based on the collaborative session identifier through the cloud center, and to perform deep inference on the timing-aligned physiological audio data in combination with the edge-side inference results to obtain auxiliary diagnosis and treatment decision results.

Images