Iot-based auscultation system, control method, device and storage medium thereof

Abstract

The application discloses a kind of based on Internet of Things auscultation system and its control method, equipment and storage medium, the application relates to general control system, the auscultation system based on Internet of Things includes auscultation terminal, transmission module, Internet of Things box, control platform and remote server, the audio signal of the auscultation terminal is used to gather user;The transmission module is used to convert the audio signal into digital audio signal and upload to the control platform and the Internet of Things box;The Internet of Things box is used to generate first analysis result according to the received digital audio signal;The control platform is used to forward the digital audio signal to the remote server, and update the first model of the Internet of Things box;The remote server is used to receive digital audio signal, and generate second analysis result according to digital audio signal.The application can realize the technical effect of remote high-fidelity diagnosis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of general control systems, in particular to a auscultation system based on Internet of Things, a control method, equipment and storage medium thereof. BACKGROUND

[0002] With the continuous progress of modern medical technology, auscultation, as a basic and important clinical diagnosis method, plays a key role in the preliminary screening and diagnosis of diseases. The stethoscope mainly relies on air conduction sound, and the doctor listens to the physiological sound such as heart and lung sound, respiratory sound through the earphone and makes judgment by experience. However, the user needs to go to the institution to perform diagnosis, which leads to that the user cannot know the state of himself in real time. SUMMARY

[0003] The main purpose of the present application is to provide an auscultation system based on Internet of Things, a control method, equipment and storage medium thereof, which aims to solve the technical problem that the user needs to go to the institution to perform diagnosis, which leads to that the user cannot know the state of himself in real time.

[0004] To achieve the above purpose, the present application provides an auscultation system based on Internet of Things, which comprises an auscultation terminal, a transmission module, an Internet of Things box, a control platform and a remote server, wherein the auscultation terminal is used for collecting audio signals of a user;

[0005] The transmission module is used for converting the audio signals into digital audio signals and uploading them to the control platform and the Internet of Things box;

[0006] The Internet of Things box is used for generating a first analysis result according to the received digital audio signals;

[0007] The control platform is used for forwarding the digital audio signals to the remote server and updating the first model of the Internet of Things box;

[0008] The remote server is used for receiving digital audio signals and generating a second analysis result according to the digital audio signals.

[0009] In an embodiment, the transmission module comprises an auscultation sound collection and control submodule and an auscultation sound receiving and processing submodule, the auscultation sound collection and control submodule comprises an audio collection submodule and a signal sending end, the auscultation sound receiving and processing submodule comprises a signal receiving end and an adapter, and the audio collection submodule is in communication with the auscultation terminal.

[0010] In an embodiment, the Internet of Things-based auscultation system further comprises an analysis device, the adapter is in communication with the analysis device, the analysis device is a mobile terminal, a mobile consultation vehicle or a medical operation terminal, and the analysis device calls the Internet of Things box to analyze the digital audio signal to generate a first analysis result.

[0011] In an embodiment, the management platform is used to analyze historical digital audio signals to perform situation awareness, and if an abnormal situation is sensed, the abnormal situation is pushed to the Internet of Things box to prompt a user that an abnormality occurs.

[0012] To achieve the above object, the application provides a control method of an Internet of Things-based auscultation system, which comprises the following steps:

[0013] In response to a trigger operation of the auscultation terminal, the collected audio signal is converted into a digital audio signal through analog-to-digital conversion;

[0014] The digital audio signal, a user identifier and a terminal identifier of the auscultation terminal are encrypted into a target signal file through the transmission module and sent to the management platform and the Internet of Things box, and the Internet of Things box is networked with the auscultation terminal;

[0015] The management platform sends the target signal file to the remote server, and the Internet of Things box processes the target signal file based on a first model to generate a first analysis result;

[0016] In response to a second analysis result returned by the remote server for the target signal file, the first analysis result and the second analysis result are fused and processed into a state report.

[0017] In an embodiment, the step of converting the collected audio signal into a digital audio signal comprises:

[0018] The auscultation terminal collects the audio signal;

[0019] The audio signal is uploaded to a preset link through a radio frequency fixed frequency circuit of the signal sending end;

[0020] The adapter receives the audio signal of the preset link through the signal receiving end and converts it into the digital audio signal through analog-to-digital conversion.

[0021] In an embodiment, the step of uploading the audio signal to a preset link through a radio frequency fixed frequency circuit of the signal sending end comprises:

[0022] The audio signal is uploaded to the preset link through radio frequency fixed frequency circuit power amplification, filtering, operational amplification, amplification, π-type filtering and electrostatic processing.

[0023] The step of receiving the audio signal of the preset link and converting it into a digital audio signal includes:

[0024] Based on the frequency receiving antenna of the signal receiving end, the audio signal is processed through an operational amplifier, a power amplifier, electrostatic processing, and noise reduction to become the digital audio signal.

[0025] In an embodiment, the step of processing the target signal file based on the first model to generate a first analysis result includes:

[0026] The management and control platform sends the target signal file and the calling instruction of the first model to the Internet of Things box.

[0027] After the Internet of Things box receives the target signal file, it pre-processes the digital audio signal in the target signal file based on the first model, including noise reduction filtering, time-frequency domain conversion, and feature point extraction.

[0028] The extracted audio features are matched with the pre-stored standard heart-lung sound and / or respiratory sound feature library in the first model to identify the abnormal audio segment position and the abnormal type.

[0029] Based on the abnormal audio segment position and the abnormal type, combined with the historical health data associated with the user identifier, a first analysis result containing an abnormal feature identifier and a preliminary lesion probability is generated. In addition, to achieve the above-mentioned purpose, the present application also provides a control device of an Internet of Things-based auscultation system, which comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the control method of the Internet of Things-based auscultation system as described above.

[0030] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer readable storage medium, and the computer readable storage medium stores a program for implementing the control method of the Internet of Things-based auscultation system, the program for implementing the control method of the Internet of Things-based auscultation system being executed by a processor to implement the steps of the control method of the Internet of Things-based auscultation system as described above.

[0031] The application provides a control method of a auscultation system based on Internet of Things, the auscultation system comprising an auscultation terminal, a transmission module, an Internet of Things box, a management and control platform and a remote server, the application converts an audio signal collected into a digital audio signal by responding to a trigger operation of the auscultation terminal; the digital audio signal, a user identifier and a terminal identifier of the auscultation terminal are encrypted into a target signal file by the transmission module and sent to the management and control platform and the Internet of Things box, the Internet of Things box is networked with the auscultation terminal; the management and control platform sends the target signal file to the remote server, and the Internet of Things box generates a first analysis result based on a first model processing the target signal file; in response to a second analysis result returned by the remote server for the target signal file, the first analysis result and the second analysis result are fused and processed into a state report. The technical problem that users need to go to an institution for diagnosis in the related art, resulting in that users cannot understand their own state in real time is solved, and the technical effect of remote diagnosis is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0032] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings required to be used in the embodiments or prior art description will be briefly introduced. Obviously, those skilled in the art can obtain other drawings from these drawings without creative labor.

[0034] Figure 1 A schematic diagram of the auscultation system based on Internet of Things of the present application;

[0035] Figure 2 A flowchart provided by the control method of the auscultation system based on Internet of Things of the present application embodiment one;

[0036] Figure 3 A schematic diagram of the auscultation system provided by the control method of the auscultation system based on Internet of Things of the present application embodiment one;

[0037] Figure 4 A hardware structure schematic diagram involved in the control device of the auscultation system based on Internet of Things of the present application.

[0038] The object implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0039] It should be understood that the specific embodiments described herein are merely intended to explain the technical solutions of the present application, and are not intended to limit the present application.

[0040] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings of the specification and specific embodiments.

[0041] At present, the stethoscope mainly relies on air conduction sound, and the doctor listens to the physiological sound such as heart and lung sound, breathing sound through the earphone and makes judgment by experience. However, the user needs to go to the institution to perform diagnosis, which leads to that the user cannot know the own state in real time.

[0042] The main solution of the present application is: in response to the triggering operation of the auscultation terminal, the collected audio signal is converted into a digital audio signal; the digital audio signal, the user identifier and the terminal identifier of the auscultation terminal are encrypted into a target signal file by the transmission module, and sent to the management and control platform; the management and control platform schedules the Internet of Things box of the auscultation terminal networking, generates a first analysis result based on a first model processing the target signal file; in response to the second analysis result of the target signal file returned by the remote server, the first analysis result and the second analysis result are fused and processed into a state report.

[0043] The present application processes the audio signal collected by the auscultation terminal by combining the auscultation terminal, the transmission module, the Internet of Things box, the management and control platform and the remote server, processes the audio signal into a digital audio signal, and coordinates the local Internet of Things box and the remote server in parallel processing through the cloud, and then generates a state report by comprehensively processing the results of the two, thereby realizing the technical effect of remote diagnosis.

[0044] It should be noted that the execution subject of the present embodiment can be an auscultation system, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a control device of an auscultation system based on the Internet of Things capable of realizing the above functions, etc. The present embodiment does not make specific limitation thereto. The present embodiment and each of the following embodiments will be described below taking the auscultation system as an execution subject.

[0045] Based on this, referring to Figure 1The embodiment of the present application provides a auscultation system based on an Internet of Things, which comprises an auscultation terminal, a transmission module, an Internet of Things box, a control platform and a remote server, the auscultation terminal is used for collecting audio signals of a user, the transmission module is used for converting the audio signals into digital audio signals and uploading the digital audio signals to the control platform and the Internet of Things box, the Internet of Things box is used for generating a first analysis result according to the received digital audio signals, the control platform is used for forwarding the digital audio signals to the remote server and updating a first model of the Internet of Things box, and the remote server is used for receiving the digital audio signals and generating a second analysis result according to the digital audio signals.

[0046] Optionally, the transmission module comprises an auscultation sound collection and control sub-module and an auscultation sound receiving and processing sub-module, the auscultation sound collection and control sub-module comprises an audio collection sub-module and a signal sending end, the auscultation sound receiving and processing sub-module comprises a signal receiving end and an adapter, and the audio collection sub-module is in communication with the auscultation terminal.

[0047] In the embodiment, the adapter is a connector used for realizing signal transmission and protocol conversion between different devices, that is, one end of the adapter is in wired connection with a computer, a mobile diagnosis table and the like, and the other end is in wireless connection with a pacification device or an audio collection device. The adapter is used for receiving a lossless carrier signal, the carrier signal can be saved in a high-fidelity WAV format as a mother's cardiopulmonary sound at a local end through an analog signal, that is, a lossless sound, and the local end can be a computer, a mobile phone, a mobile diagnosis vehicle and the like. In the embodiment, the storage format of the collected audio is a WAV format, the WAV format can realize lossless sound quality, and can realize a sampling frequency of 48K, 96K and above.

[0048] Optionally, the auscultation system based on the Internet of Things further comprises an analysis device, the adapter is in communication with the analysis device, the analysis device is a mobile terminal, a mobile diagnosis vehicle or a medical operation terminal, and the analysis device calls the Internet of Things box to analyze the digital audio signals and generate a first analysis result.

[0049] The analysis device is installed with a software system, connected with the auscultation terminal through an audio signal receiving and processing submodule to obtain input data. The software system functions include: starting the sound card, directly connecting with the network cloud service platform, realizing synchronous full diagnosis recording and synchronous external broadcasting of the on-site end and the cloud end. Interacting with the cloud data information: auscultation data or information transmission, storage and cloud server AI data communication sample extraction, sampling, cleaning, deep learning, identification and feedback, such as extraction and identification of heart and lung sound, second heart and lung sound and prompt response. Feature information extraction and interaction: machine information, user information, AI interaction information setting and time sequence prompt, natural voice, heart and lung auscultation, other organ auscultation (vascular auscultation) and the like; optimal information transmission mode, low power consumption, low cost, easy identification, easy operation and high fidelity.

[0050] Optionally, the management and control platform is used to analyze historical digital audio signals to perform situation awareness, and if an abnormal situation is sensed, the abnormal situation is pushed to the Internet of Things box to prompt the user that an abnormality occurs.

[0051] As an optional implementation, at the user end, the auscultation terminal is provided to users who have daily physiological sound monitoring needs. The auscultation terminal is designed as a portable device for the user to use at any time. At the same time, an Internet of Things box and a transmission module can be deployed at the user's home. The Internet of Things box is used to preliminarily analyze the received digital audio signals, and the transmission module ensures stable connection between the auscultation terminal and the Internet of Things box. The audio acquisition submodule in the auscultation sound acquisition and control submodule of the transmission module is connected with the auscultation terminal to realize acquisition of the audio signal.

[0052] At the medical institution end, a management and control platform and a remote server are deployed. The management and control platform is responsible for data forwarding, model updating and situation awareness, and the remote server provides more in-depth analysis and diagnosis capabilities. In addition, an analysis device such as a mobile terminal, a mobile consultation vehicle or a medical operation terminal is provided, and a matching software system is installed on the analysis device, which is connected with the signal receiving end of the auscultation sound receiving and processing submodule of the transmission module through the adapter to realize data interaction with the auscultation terminal.

[0053] The user uses the auscultation terminal to collect physiological audio signals such as heart and lung sound and breath sound of the user. The audio signal collected by the auscultation terminal is transmitted to the auscultation sound acquisition and control submodule of the transmission module, and the audio acquisition submodule obtains the signal, and then the analog audio signal is sent to the signal receiving end of the auscultation sound receiving and processing submodule by the signal sending end. After the signal receiving end receives the analog audio signal, the adapter performs analog-to-digital conversion on the analog audio signal to convert it into a digital audio signal.

[0054] The converted digital audio signal is uploaded to the management and control platform and the Internet of Things box deployed in the user's home through the network respectively. At the same time, the software system on the analysis device starts the sound card and directly connects with the network cloud service platform, realizing synchronous full diagnosis recording and synchronous external broadcast of auscultation between the on-site end (user end) and the cloud end (remote service end), so as to ensure that the doctor can obtain the audio signal in real time.

[0055] After the Internet of Things box deployed in the user's home receives the digital audio signal, it analyzes the signal according to the preset first model to generate a first analysis result. The first model can be a basic diagnosis model trained based on a large amount of clinical data, which can preliminarily judge common physiological sound abnormalities, such as simple heart and lung sound abnormality prompt, heavy breathing sound, etc.

[0056] The analysis device obtains the first analysis result by calling the Internet of Things box. The software system on the analysis device can extract and interact with the feature information according to the first analysis result, for example, prompting the user with natural voice about the preliminary analysis situation, such as "detecting that the heart and lung sound has slight abnormality, please pay attention", and displaying relevant machine information, user information and AI interaction information on the device interface, and giving time sequence prompt.

[0057] The management and control platform forwards the received digital audio signal to the remote service end. The remote service end uses more powerful computing resources and more complex analysis algorithms, such as deep learning algorithm, to deeply analyze the digital audio signal and generate a second analysis result. The second analysis result will provide more accurate and detailed diagnosis information, such as specific disease type, disease severity, etc.

[0058] During the analysis process, the remote service end also interacts with the cloud data, transmits and stores the auscultation data or information, and extracts, samples, cleans, deeply learns, identifies and feeds back from the AI data of the cloud server. For example, for heart and lung sound signals, the second heart and lung sound is extracted and identified, and the identification result is prompted.

[0059] The management and control platform analyzes the historical digital audio signal to perform situation awareness. By analyzing the trend of audio signal change in a period of time, it is judged whether the user's physiological state is abnormal. If an abnormal situation is perceived, the management and control platform pushes the abnormal situation to the Internet of Things box in the user's home. After receiving the abnormal prompt, the Internet of Things box prompts the user to appear abnormal through the analysis device in the form of natural voice, interface pop-up window, etc., and suggests the user to take further measures, such as going to the hospital for detailed examination.

[0060] At the same time, the management and control platform also synchronizes the abnormal situation to the relevant medical staff of the medical institution, so that the medical staff can timely understand the user's situation and provide further medical advice and guidance for the user.

[0061] The management platform is responsible for updating the first model of the Internet of Things box in the user's home. As clinical data accumulates and medical research deepens, the management platform will collect new case data and diagnostic results to optimize and update the first model. By pushing the updated model to the Internet of Things box, the accuracy and effectiveness of the Internet of Things box in analyzing digital audio signals are improved.

[0062] In addition, the software system on the analysis device will also be optimized and upgraded in function according to actual use and user feedback, ensuring that the system always maintains the optimal information transmission mode, achieving the goal of low power consumption, low cost, easy identification, easy operation, and high fidelity, providing better use experience and diagnostic support for users and medical staff.

[0063] Further, by aggregating the data of the Internet of Things box in the region, disease risk assessment and group incidence trend prediction are realized to support public health decision-making. The management platform adds a "regional data aggregation module" to aggregate the first analysis results uploaded by each Internet of Things box in the region according to administrative divisions such as streets and communities. For example, results such as suspected respiratory abnormalities and irregular heart and lung rhythm are within the aggregation range. At the same time, user basic information is associated, including age, gender, medical history, etc., and all of which have been desensitized. Time-space sequence analysis algorithm is used to perform multi-dimensional statistics on aggregated data: time dimension: calculate the frequency of abnormal signals in different time periods. For example, the frequency difference between weekdays and weekends, and the frequency change during rush hours. Spatial dimension: generate a regional disease heat map to mark areas with high concentration of abnormal signals. Population dimension: analyze the incidence tendency of specific groups, such as people over 65 years old or specific occupational groups. Set three levels of warning thresholds: general attention, moderate warning, and high warning. When the proportion of a certain type of abnormal signal in a certain region exceeds the threshold value: push warning information and heat map to the regional health service center. For users in high-risk areas, push health tips through the Internet of Things box. For example, suggest reducing gatherings in areas with high incidence of respiratory diseases.

[0064] The management platform extracts season-related features from the Internet of Things box data. For example, increased respiratory sound abnormalities in winter, and allergic cough signal characteristics in summer. At the same time, combined with regional weather data such as temperature, humidity, and pollen concentration, an association model is constructed. Using LSTM and other time series prediction algorithms, the model is trained based on aggregated data from the past three years to predict disease prevalence trends in the next 1-3 months. For example, it can predict that the incidence of childhood bronchitis in the region will increase by 15% next month. Provide trend reports to the CDC to assist in developing vaccination plans or health education programs. Push the prediction results to regional medical institutions to allow them to allocate medical resources in advance. For example, increase the number of pediatric clinic medical staff.

[0065] Further, the model is optimized by using the massive data uploaded by the Internet of Things box, and the updated model is pushed to the terminal to improve the local analysis accuracy. Each family Internet of Things box uses the digital audio signals accumulated locally, such as the heart and lung sound data in the past three months, to perform lightweight training based on the basic model pushed by the management and control platform, and only uploads the model parameters to the management and control platform instead of the original data, so as to avoid privacy leakage. The "model aggregation module" of the management and control platform aggregates the parameters uploaded by all Internet of Things boxes by weighting, the weight is positively correlated with the data quality and sample size, generates a globally optimized model, and verifies the model accuracy by cross-validation. The cross-validation selects part of the labeled data of medical institutions. The management and control platform pushes the incremental model parameters to the Internet of Things box, only updates the difference, and reduces the transmission bandwidth. The Internet of Things box receives and automatically combines the parameters, completes the model update, and feeds back the update result to the platform.

[0066] The management and control platform classifies and labels the digital audio signals uploaded by the Internet of Things box according to disease types, such as cardiovascular and respiratory tract, and also classifies and labels according to user groups, such as infants and the elderly, to build a segmented data set. The labeling work is completed by a team of professional physicians on the remote server. For high-frequency diseases such as hypertensive heart disease, a "special recognition model" is trained to optimize the feature extraction algorithm, such as enhancing the recognition accuracy of the third heart and lung sound. For special groups, such as the attenuation problem of respiratory sound of obese users, a "group adaptation model" is trained to adjust the signal noise reduction parameters. The management and control platform pushes the adapted special model according to the historical data characteristics of the Internet of Things box, such as the user of this family is an old person: for the family of old users, the heart and brain blood vessel disease recognition model is updated first. For families with children, the pneumonia-related respiratory sound analysis model is updated.

[0067] The regional analysis scenario upgrades individual health data to group public health decision basis through the "aggregation-analysis-warning" link. The model iteration scenario realizes the conversion of data to model capability through the "local training-global optimization-precise pushing" mechanism, while considering privacy protection and terminal efficiency. The extension of the two scenarios extends the system from "individual health monitoring" to "regional health management" and "intelligent model evolution", and improves the overall ecological value.

[0068] Based on this, the control method of the auscultation system based on the Internet of Things is provided in Embodiment One of the present application, please refer to Figure 2 , the control method of the auscultation system based on the Internet of Things includes steps S10-S40:

[0069] Step S10, in response to the trigger operation of the auscultation terminal, the collected audio signal is converted into a digital audio signal.

[0070] In this embodiment, the auscultation terminal is a portable medical device integrated with a high-sensitivity microphone, which can be triggered to collect audio signals through physical buttons or a mobile phone APP. The audio signals refer to physiological acoustic signals generated by the activities of the human heart cavity, blood vessels or lungs, such as heart and lung sounds, and respiratory sounds. The analog-to-digital conversion is completed by the chip built-in the terminal, which retains high-frequency acoustic characteristics according to the set sampling rate, and the 16-bit quantization bit ensures the signal details.

[0071] Referring to Figure 3 , the auscultation system includes an auscultation terminal, a transmission module, an Internet of Things box, a control platform, and a remote server. There are several auscultation terminals, which send digital audio signals to the control platform through the transmission module. The control platform is connected to the Internet of Things box and the remote server at the same time. The remote server can be a community health service center, a medical institution, etc.

[0072] As an optional implementation, when the trigger signal is detected, the dual-microphone array synchronously collects the sound pressure signals with a 180° phase difference. The main microphone uses an electret capacitive sensor, and the environmental microphone is used for noise cancellation. The analog signal is amplified by a low-noise preamplifier to achieve a 40dB gain, and then passes through a second-order Butterworth low-pass filter to suppress high-frequency interference. Subsequently, the 32-bit ADC is over-sampled at a sampling rate of 128kHz, and the effective sampling rate is reduced to 16kHz by a digital decimation filter to generate a 32-bit floating-point PCM data stream. The data stream is packaged into data frames according to every 256 sampling points to generate a digital audio signal.

[0073] In step S20, the digital audio signal, the user identification, and the terminal identification of the auscultation terminal are encrypted into a target signal file by the transmission module, and sent to the control platform and the Internet of Things box. The Internet of Things box is networked with the auscultation terminal.

[0074] In this embodiment, the user identification is a unique number registered by the patient in the system (such as a mobile phone hash value), and the terminal identification is a unique serial number written into the device when it is manufactured (such as an IMEI code). The transmission module is used to encrypt and upload the digital audio signal to the cloud. The key is generated by the dynamic session key pre-negotiated by the control platform and the Internet of Things box. The target signal file is packaged in JSON format, including audio data, user identification, and terminal identification.

[0075] As an optional implementation, the transmission module uses blockchain storage technology to encrypt the digital audio signal, user identification, and terminal identification of the auscultation terminal into a target signal file. Before encryption, a timestamp and a hash value are added to the target signal file, which is uploaded to the control platform and recorded in the alliance chain node at the same time, ensuring that the data cannot be tampered with. At the same time, the transmission module sends the target signal file to the Internet of Things box.

[0076] Step S30, the management platform sends the target signal file to the remote server, and the Internet of Things box generates a first analysis result based on the first model processing the target signal file.

[0077] In this embodiment, the Internet of Things box is an edge computing gateway, and a lightweight CNN model, i.e., the first model, is deployed to classify heart and lung sounds. The first analysis result is the result analyzed by the Internet of Things box for the digital audio signal.

[0078] As an optional implementation, the management platform schedules the nearest edge node based on geographic location information, or schedules an edge node networking with the auscultation terminal. The Internet of Things box runs the optimized audio classification model through TensorFlowLite to extract time domain and frequency domain features of the signal, identify S1 / S2 heart and lung sound splitting, pathological noise and other features, and generate a structured analysis report containing heart rate variability and noise type, i.e., the first analysis result. At the same time, the management platform sends the target signal file to the remote server.

[0079] Step S40, in response to the second analysis result returned by the remote server for the target signal file, the first analysis result and the second analysis result are fused to generate a state report.

[0080] In this embodiment, the remote server is deployed with an LSTM-based time series analysis model and a medical expert rule engine, or is a device end with experts such as community health service centers and medical institutions. The second analysis result is the result analyzed by the expert for the digital audio signal, or the result analyzed by the time series analysis model and the medical expert rule engine for the digital audio signal. The state report is a result generated by combining the first analysis result and the second analysis result, which is more in line with the individual characteristics of the user.

[0081] As an optional implementation, the management platform uses evidence theory to fuse the results, weights and fuses the probability distribution output by the edge model and the diagnosis suggestion of the expert system, and generates a multi-dimensional state report containing heart rate and confidence index, which is pushed to the user terminal in real time.

[0082] For example, a user uses an intelligent stethoscope to collect heart and lung sound data, and the device automatically triggers collection through bioelectric signal detection. After the data is uploaded to the cloud after encryption, the management platform schedules the edge node of the community health service center to perform preliminary analysis and identifies the S3 gallop feature. At the same time, the deep learning model of the remote medical platform combines the electronic medical record of the patient to prompt the risk of left ventricular dysfunction. The system fuses the double analysis results to generate a red warning state report and pushes it to the patient's mobile phone and family doctor terminal.

[0083] In the embodiment, the auscultation system comprises an auscultation terminal, a transmission module, an Internet of Things box, a management and control platform, and a remote server. The auscultation system converts an audio signal collected into a digital audio signal in response to a triggering operation of the auscultation terminal; encrypts the digital audio signal, a user identifier, and a terminal identifier of the auscultation terminal into a target signal file through the transmission module, and sends the target signal file to the management and control platform; the management and control platform dispatches the Internet of Things box of the auscultation terminal network, processes the target signal file based on a first model to generate a first analysis result; in response to a second analysis result returned by the remote server for the target signal file, fuses the first analysis result and the second analysis result to process a status report. The technical problem that a user needs to go to an institution for diagnosis in the related art, so that the user cannot know the status in real time is solved, and a technical effect of remote diagnosis is achieved.

[0084] Based on any of the above embodiments, in the second embodiment of the present application, the auscultation system further comprises an adapter, the adapter is provided with a signal receiving end, the auscultation terminal is provided with a signal sending end, and step S10 comprises:

[0085] Step S11, the auscultation terminal collects the audio signal.

[0086] In the embodiment, the auscultation terminal is an intelligent module modified from a traditional stethoscope, a MEMS microphone array is built-in, and an adaptive beamforming technology is used to focus and collect cardiopulmonary sound signals.

[0087] As an optional implementation, the auscultation terminal detects contact pressure through a three-axis acceleration sensor, and triggers collection when the pressure value is in the range of 2 to 3 N. A double microphone array synchronously collects sound pressure signals with a phase difference of 180°. A main microphone uses a -38 dB sensitivity electret capacitive sensor, and an environmental microphone is used to collect background noise.

[0088] Step S12, the audio signal is uploaded to a preset link through a radio frequency fixed frequency circuit of the signal sending end.

[0089] As an optional implementation, the signal sending end integrates a 2.4 GHz radio frequency chip, uses a GFSK modulation mode, and the preset link is a private wireless channel based on an IEEE802.15.4 protocol.

[0090] Illustratively, after the audio signal is initially digitized by a 12-bit ADC at a sampling rate of 48 kHz, the audio signal is modulated into a 433 MHz carrier signal through a radio frequency fixed frequency circuit, direct sequence spread spectrum technology (DS-SS) is used to improve the anti-interference ability, and the data transmission rate is set to 250 kbps, so as to ensure the audio quality while reducing the power consumption.

[0091] As another optional implementation, a private protocol is used by the signal sending end and the signal receiving end to achieve lossless data transmission.

[0092] In step S13, the adapter receives the audio signal of the preset link through the signal receiving end and converts it into the digital audio signal through analog-to-digital conversion.

[0093] In this embodiment, the adapter is a rechargeable smart gateway device, the signal receiving end integrates a superheterodyne receiver circuit, and the analog-to-digital conversion uses a 24-bit sigma-delta ADC.

[0094] As an optional implementation, the signal receiving end locks the 433 MHz carrier signal through a phase-locked loop (PLL) circuit, recovers the baseband audio signal after mixing, intermediate frequency amplification and demodulation, and then performs high-precision analog-to-digital conversion at a sampling rate of 192 kHz through a 24-bit sigma-delta ADC, reduces the effective sampling rate to 44.1 kHz through a digital filter, generates a 32-bit floating-point PCM data stream, and finally uploads it to the management and control platform through a USB interface or Wi-Fi.

[0095] For example, a cardiologist uses a smart stethoscope to collect the heart and lung sounds of a patient. The stethoscope module transmits the audio signal to the clinic adapter through a 433 MHz radio frequency. After the adapter receives the signal, it performs high-precision analog-to-digital conversion, generates a digital audio file, and uploads it to the hospital information system. The doctor checks the real-time heart and lung sound waveform and AI analysis result through the remote server to quickly determine whether the patient has mitral stenosis.

[0096] This embodiment transfers the analog-to-digital conversion function to the adapter through a distributed collection architecture, reduces the size of the stethoscope terminal, and realizes lossless signal conversion.

[0097] Optionally, the step of uploading the audio signal to the preset link through the radio frequency fixed frequency circuit of the signal sending end comprises:

[0098] In step S121, the audio signal is uploaded to the preset link through the radio frequency fixed frequency circuit for power amplification, filtering, operational amplification, amplification, π-type filtering and electrostatic treatment.

[0099] In this embodiment, the radio frequency fixed frequency circuit refers to the radio frequency processing circuit integrated in the signal sending end of the stethoscope terminal, which is used for power amplification and signal optimization of the audio signal. Power amplification improves the signal transmission strength; filtering is used to remove noise interference; operational amplification is used for signal operational amplification; π-type filtering further filters out high-frequency noise through a π-type LC filter circuit; and electrostatic treatment prevents damage to the circuit caused by static electricity through an ESD electrostatic protection device. The preset link is a wireless transmission channel established based on a specific frequency band.

[0100] As an optional embodiment, the audio signal first enters the power amplifier of the radio frequency fixed frequency circuit, and the signal power is raised to a level suitable for long distance transmission; then through the band pass filter, the interference signals outside the frequency band are filtered out; then the signal is adjusted by the operational amplifier to ensure the stability of the signal amplitude; then the signal is amplified again to enhance the signal strength; then through the π type LC filter circuit, the residual high frequency noise is filtered out; finally through the ESD static protection device, the static treatment is carried out, and the signal is uploaded to the preset link through the 2.4GHz frequency band in the GFSK modulation mode.

[0101] As another optional embodiment of uploading the audio signal to the preset link through the radio frequency fixed frequency circuit of the signal sending end, the signal sending end realizes multi-directional transmission through the broadcast of 84Mhz.

[0102] Optionally, the step of receiving the audio signal of the preset link and converting it into the digital audio signal includes:

[0103] Step S131, based on the fixed frequency receiving antenna of the signal receiving end, the audio signal is processed into the digital audio signal through operational amplifier, power amplifier, static treatment and noise reduction.

[0104] In this embodiment, the fixed frequency receiving antenna is an antenna used by the adapter signal receiving end to receive signals of a specific frequency band. The operational amplifier performs operational amplification on the received weak signal; the power amplifier again raises the signal power; the static treatment is protected by the ESD device protection circuit; and the noise reduction removes the environmental interference through the digital noise reduction algorithm.

[0105] As an optional embodiment, after the fixed frequency receiving antenna receives the audio signal of the preset link, the signal is first sent into the operational amplifier for amplification processing to enhance the signal strength; then the power amplifier is used to further improve the signal power; then the ESD static protection device is used for static protection; finally, the adaptive digital noise reduction algorithm is used to dynamically adjust the parameters according to the environmental noise characteristics, remove the background noise interference, and then the built-in 24-bit ADC is used for analog-to-digital conversion to generate the digital audio signal.

[0106] For example, a user uses a smart auscultation terminal at home to collect cardiopulmonary sound signals, and the radio frequency fixed frequency circuit in the auscultation terminal sequentially performs power amplifier, filtering and a series of processing on the collected audio signal, and then sends it out through the preset link. The adapter placed in the living room receives the signal through the fixed frequency receiving antenna, converts it into a digital audio signal after operational amplifier and noise reduction processing, uploads it to the management and control platform for subsequent analysis and diagnosis, and finally the user receives a status report containing health status analysis on the mobile phone.

[0107] The embodiment transmits the audio signal after multiple processing by the radio frequency fixed frequency circuit, and optimizes and converts the received signal by the adapter, reduces the interference and loss in the signal transmission process, improves the transmission quality and stability of the audio signal, and provides a reliable data basis for subsequent accurate analysis and diagnosis.

[0108] Based on any of the above embodiments, in the third embodiment of the application, step S20 comprises:

[0109] Step S21, obtaining the terminal identification of the auscultation terminal and the user identification of the auscultation terminal login account.

[0110] In this embodiment, the terminal identification is a globally unique identification code (such as UUID or IMEI) fixed at the factory of the auscultation terminal, which is used for device identity authentication; the user identification is a unique account code (such as a mobile phone number hash value) generated when the user registers, which is associated with the personal health record.

[0111] As an optional implementation, when the auscultation terminal starts, the system automatically reads the terminal identification stored in the secure element (SE); after the user logs in to the APP through fingerprint or facial recognition, the system retrieves the user identification from the local encrypted database, ensuring that the data reading process meets the GDPR security standard.

[0112] Step S22, obtaining the data key of the Internet of Things box of the auscultation terminal networking.

[0113] In this embodiment, the data key is a symmetric encryption key previously negotiated between the Internet of Things box and the management platform, using a dynamic update mechanism (such as changing every 24 hours).

[0114] As an optional implementation, after the auscultation terminal establishes a connection with the Internet of Things box through Bluetooth or Wi-Fi, it sends a key request instruction. After verifying the legality of the terminal identification, the Internet of Things box retrieves the current data key from the secure storage module (TPM), encrypts it using the terminal public key, and returns it, ensuring that the key transmission process is theft-proof and tamper-proof.

[0115] Step S23, encrypting the digital audio signal, the user identification and the terminal identification into a first data packet based on the data key.

[0116] In this embodiment, AES-256-GCM algorithm is used for data encryption, and GCM mode provides authentication encryption function to ensure data integrity.

[0117] As an optional implementation, the system encapsulates the digital audio signal, the user identifier, and the terminal identifier into raw data in JSON format, performs AES-256-GCM encryption using the data key obtained from the Internet of Things box, generates a first data packet containing ciphertext, an authentication tag, and a random number, and calculates a data hash value (SHA-256) for integrity verification.

[0118] Step S24, the transmission module transmits the first data packet to the management and control platform through the encryption channel corresponding to the terminal identifier.

[0119] In this embodiment, the encryption channel is established based on the TLS1.3 protocol, and each terminal identifier corresponds to a unique session key.

[0120] As an optional implementation, the transmission module obtains the corresponding session key from the key management system after parsing the terminal identifier, and establishes a secure connection using the TLS1.3 protocol. After the first data packet is processed by framing, it is transmitted to the management and control platform through the encryption channel. Forward error correction (FEC) technology is enabled during transmission to ensure reliable data transmission in weak network environments.

[0121] For example, a diabetic patient uses an intelligent stethoscope at home to monitor heart health. After the device is started, the terminal identifier is automatically obtained, and the user logs in through face recognition on the APP to obtain the user identifier. The stethoscope connects to the Internet of Things box at home, obtains the data key after identity verification, and encrypts the collected digital audio signal to generate a first data packet. The transmission module securely uploads the data packet to the management and control platform through the pre-configured encryption channel, and the subsequent joint diagnosis by the AI model and the doctor and feedback of the results.

[0122] This embodiment realizes full-link encryption of data from the terminal to the cloud through multi-layer encryption mechanism and dynamic key management, effectively protecting the privacy and security of user medical data; based on the exclusive encryption channel of the terminal identifier and the transmission optimization technology, the security and reliability of data transmission are improved, providing a solid data security foundation for remote medical diagnosis.

[0123] Based on any of the above embodiments, in the fourth embodiment of the present application, the step of generating a first analysis result based on the target signal file based on the first model includes:

[0124] Step S31, the management and control platform sends the target signal file and the calling instruction of the first model to the Internet of Things box.

[0125] In this embodiment, the control platform is a distributed cluster with high-performance computing and storage capabilities, responsible for managing and scheduling system resources. The target signal file is an encrypted data packet containing user audio data, identifiers, and other information. The first model is a lightweight machine learning model deployed on the IoT box for audio signal analysis. The invocation command is a command sent by the control platform to the IoT box to start the model processing task.

[0126] As an optional implementation, after receiving the target signal file, the management platform selects a suitable IoT box for task allocation based on the IoT box's load status and geographical location. The target signal file and invocation instructions are then sent to the selected IoT box via a message queue (such as RabbitMQ) to ensure reliable and asynchronous data transmission.

[0127] Step S32: After receiving the target signal file, the IoT box preprocesses the digital audio signal in the target signal file based on the first model, including noise reduction filtering, time-frequency domain conversion, and feature point extraction.

[0128] In this embodiment, the IoT box is an edge computing device with certain computing and storage capabilities, used for preliminary processing of audio signals locally. Noise denoising filtering removes noise interference from the audio signal using digital filters, improving signal quality. Time-frequency domain conversion converts the time-domain audio signal to the frequency domain for better analysis of its frequency characteristics. Feature point extraction extracts representative feature points from the audio signal for subsequent matching and analysis.

[0129] As an optional implementation, after receiving the target signal file, the IoT box first decrypts and verifies it. Then, wavelet transform is used for denoising filtering to remove high-frequency noise and low-frequency interference. Next, the audio signal is converted into a time-frequency domain representation using short-time Fourier transform (STFT) to obtain a spectrogram. Finally, a peak detection algorithm is used to extract feature points from the spectrogram, such as the S1 and S2 peaks of heart and lung sounds.

[0130] Step S33: Match the extracted audio features with the pre-stored standard cardiopulmonary sound and / or respiratory sound feature library in the first model to identify the location and type of abnormal audio segments.

[0131] In this embodiment, the standard heart and lung sounds and / or breath sounds feature library is a pre-stored set of normal audio features in the first model, used for comparison with the extracted audio features. Matching determines the presence of an anomaly by calculating the similarity between audio features. The location of the abnormal audio segment refers to the specific time position in the audio signal where the anomaly occurs. The anomaly type refers to the classification of the anomaly determined based on the matching results, such as arrhythmia, shortness of breath, etc.

[0132] As an optional implementation, the Internet of Things box compares the extracted audio features with each feature in the standard feature library one by one, calculates the similarity by using a dynamic time warping (DTW) algorithm. When the similarity is lower than a certain threshold, it is determined as an abnormal audio segment, and the abnormal type is determined according to the feature matching result. For example, if the frequency and intensity of the heart and lung sound match the arrhythmia feature in the standard feature library within a certain time period, it is identified as an arrhythmia abnormality.

[0133] Step S34, based on the abnormal audio segment position and the abnormal type, combining the historical health data associated with the user identification, generating a first analysis result containing abnormal feature identification and preliminary lesion probability.

[0134] In this embodiment, the historical health data associated with the user identification refers to the past health information related to the user, such as medical records, examination reports, etc. The abnormal feature identification is a specific description and identification of the abnormal situation, in order to facilitate subsequent analysis and processing. The preliminary lesion probability is an estimated value of the lesion probability calculated according to the abnormal audio segment position, the abnormal type and the historical health data.

[0135] Based on any of the above embodiments, in the fifth embodiment of the present application, after the step of encrypting the digital audio signal, user identification and terminal identification of the auscultation terminal through the transmission module into a target signal file and sending it to the management and control platform, it includes:

[0136] Step A10, removing personal information contained in the digital audio signal in the target signal file to obtain a target digital signal.

[0137] In this embodiment, personal information includes but is not limited to user biometric data, geographic location information and identifiable identity timestamp. The removal operation is achieved by data desensitization algorithm, which processes the user identification by using irreversible hash desensitization (such as SHA-256), and strips the environmental dialogue sound that may be contained in the audio signal by using voice activity detection (VAD) technology.

[0138] As an optional implementation, the system scans the time domain waveform of the digital audio signal by using a natural language processing (NLP) model, identifies and filters voice segments containing keywords such as name and address; at the same time, the user identification field is desensitized with format reservation (such as converting "1381234" to "1381234" format but the actual value is irreversible confusion), to ensure that the target digital signal meets the requirements of the Health Medical Data Security Guide (GB / T39725-2020).

[0139] Step A20, training the second model with the target digital signal as training data.

[0140] In this embodiment, the second model is a deep learning model deployed in the cloud, which adopts a hybrid architecture of convolutional neural network (CNN) and recurrent neural network (RNN) to improve the generalization ability of heart-lung sound classification. The training process adopts a federated learning framework to ensure that the data does not leave the domain.

[0141] As an optional implementation, the target digital signal is segmented into 10-second segments, and then 40-dimensional feature vectors are extracted through Mel-frequency cepstral coefficients (MFCC) to construct the training data set. The Adam optimizer is used for end-to-end training, the loss function is the weighted sum of cross-entropy and focal loss (FocalLoss), the iteration period is set to 100 rounds, the batch size is 32, and the verification set proportion is 20%. During the training process, the migration performance of the model on the public data set (such as the PhysioNet heart-lung sound database) is monitored in real time to prevent overfitting.

[0142] Step A30, in response to the data synchronization instruction, updating the first model of the Internet of Things box based on the second model.

[0143] In this embodiment, the data synchronization instruction is generated periodically by the management and control platform (such as every day at 2 a.m.), or triggered based on model performance indicators (such as verification set accuracy improvement ≥1.5%). The update process adopts model difference compression technology, and only the weight parameter change amount is transmitted.

[0144] As an optional implementation, the management and control platform generates an incremental update package (DeltaPackage) by comparing the weight parameters of the second model and the local first model of the Internet of Things box. The update package is pushed to the target Internet of Things box using the gRPC protocol. After the box receives the update package, it first performs integrity check (SHA-512 hash matching), and then replaces the model file through a hot update mechanism, the entire process maintains uninterrupted business. After the update is completed, the Internet of Things box automatically runs test cases (such as pre-stored normal / abnormal heart-lung sound samples), verifies the correctness of the model function and returns the results to the cloud.

[0145] For example, a regional medical center collects 1000 cases of desensitized abnormal heart-lung sound data for training the cloud second model. The model improves the identification accuracy of mitral regurgitation from 88% to 93% through 3-day iterative training. After the management and control platform detects significant performance improvement, it sends a data synchronization instruction to all community health service centers in the region. After receiving the model update package, each box completes the upgrade during the non-diagnosis period, and the next day it can apply the new model for heart-lung sound analysis, which improves the early detection rate of valvular lesions in primary medical institutions by about 5%.

[0146] The embodiment constructs a closed-loop mechanism of "data desensitization-cloud training-edge update", realizes continuous evolution of the model under the premise of protecting user privacy, and solves the problem that the traditional edge computing model is difficult to dynamically optimize. The incremental update technology reduces the model upgrade traffic consumption, the hot update mechanism ensures that the basic medical services are not interrupted, and the practicability and scalability of the system are effectively improved.

[0147] Based on any of the above embodiments, in the sixth embodiment of the present application, in response to the second analysis result returned by the remote server for the target signal file, the step of fusing the first analysis result and the second analysis result into a state report comprises:

[0148] In step S41, first metadata of the first analysis result and second metadata of the second analysis result are obtained.

[0149] In the embodiment, the first metadata is data for describing the related information of the first analysis result, the generation timestamp records the specific time when the first analysis result is generated, the Internet of Things box identifier is used to specify which Internet of Things box generates the analysis result, and the model version number indicates the version of the first model used at that time. The second metadata is description information related to the second analysis result, the remote server identifier indicates which remote server the second analysis result comes from, the expert qualification level reflects the professional level of the expert participating in the analysis, and the analysis completion time records the time when the second analysis result is completed.

[0150] As an optional implementation, the system can directly extract the corresponding metadata information from the database storing the first analysis result and the second analysis result. The first metadata includes the generation timestamp, the Internet of Things box identifier, and the model version number, and the second metadata includes the remote server identifier, the expert qualification level, and the analysis completion time.

[0151] In step S42, if the first metadata and the second metadata pass the verification, the intersection interval between the abnormal feature identifier in the first analysis result and the manually annotated abnormal segment in the second analysis result is determined.

[0152] In the embodiment, passing the verification means that the first metadata and the second metadata meet certain rules and conditions in terms of time, source, etc. The abnormal feature identifier is a specific identifier of the abnormal situation in the first analysis result, and the manually annotated abnormal segment is an abnormal audio segment manually marked by experts in the second analysis result. The intersection interval is the range of the abnormal audio segment determined by both.

[0153] As an optional implementation, the system can verify whether the metadata passes by comparing time information, device identifiers, etc. in the first analysis result and the second analysis result. For abnormal feature identifiers and manually annotated abnormal segments, the intersection interval can be determined by comparison on the time axis.

[0154] In step S43, if the intersection interval meets the confidence requirement, the first analysis result and the second analysis result are assigned fusion weights.

[0155] In this embodiment, the confidence requirement refers to the reliability and credibility of the intersection interval meeting a certain standard. The fusion weight is a numerical value used to determine the proportion of the first analysis result and the second analysis result in fusion.

[0156] As an optional implementation, the system can determine whether the confidence requirement is met according to factors such as the length of the intersection interval and the degree of compliance with known standards. For the assignment of fusion weights, the accuracy history of the first analysis result and the second analysis result, the authority of experts, etc. can be used to determine.

[0157] In step S44, the first analysis result and the second analysis result are fused based on the fusion weights to generate the status report.

[0158] In this embodiment, fusion processing is to comprehensively calculate the first analysis result and the second analysis result according to the fusion weights to generate the final status report.

[0159] As an optional implementation, the system can perform weighted average on each index in the first analysis result and the second analysis result according to the fusion weights to obtain the corresponding content in the status report.

[0160] For example, it is assumed that the first analysis result indicates that there may be a certain heart disease, the abnormal feature identifier is "A", the second analysis result also relates to the disease in the manually annotated abnormal segment, and the intersection interval meets the confidence requirement. According to the fusion weights, the first analysis result and the second analysis result are fused to generate the status report, which indicates the possibility of the heart disease and gives corresponding suggestions.

[0161] This embodiment can comprehensively consider the information of the first analysis result and the second analysis result by obtaining and verifying the metadata, determining the intersection interval, assigning the fusion weights and performing fusion processing, to generate a more accurate and reliable status report, and provide more valuable health diagnosis and suggestions for users.

[0162] Optionally, the step of fusing the first analysis result and the second analysis result based on the fusion weights to generate the status report includes:

[0163] Step S441, cross-verify the abnormal audio segment positions and types, and determine the abnormal segment based on the intersection interval.

[0164] In this embodiment, cross-verification refers to comparing the abnormal audio segment positions and types in the first and second analysis results to ensure consistency and accuracy. The intersection interval refers to the time range of the abnormal audio segment identified by both the first and second analysis results.

[0165] As an optional implementation, the system first compares the abnormal audio segment positions and types in the first and second analysis results. If there is high consistency in the time range and type, the abnormal segment in the intersection interval is determined as the final abnormal segment. If there is a difference, further analysis is needed to determine the cause, which may be due to different analysis methods or data processing methods. In this case, the system can combine expert opinions or other relevant information to determine the final abnormal segment.

[0166] Step S442, based on the lesion probability value corresponding to the abnormal segment, the fusion weight is used for weighted calculation to obtain the fusion probability value.

[0167] In this embodiment, the lesion probability value refers to the estimated value of the possibility of the abnormal segment corresponding to the lesion in the first and second analysis results. The fusion weight is a weight value determined according to the reliability and authority of the first and second analysis results.

[0168] As an optional implementation, the system performs weighted calculation based on the lesion probability value corresponding to the abnormal segment in the first and second analysis results, and the pre-determined fusion weight. Specifically, the lesion probability value in the first analysis result is multiplied by its corresponding fusion weight, and the lesion probability value in the second analysis result is multiplied by its corresponding fusion weight. Then, the two products are added to obtain the fusion probability value.

[0169] Step S443, combine the historical health data associated with the user identification to correct the fusion probability value to obtain the final probability.

[0170] In this embodiment, the historical health data refers to past health information associated with the user identification, such as medical records, examination reports, etc. The corrected fusion probability value refers to adjusting the fusion probability value based on the historical health data to improve the accuracy of the diagnosis.

[0171] As an optional implementation, the system combines the fusion probability value with the historical health data associated with the user identifier. If the user has relevant medical history or other health problems, the fusion probability value is adjusted based on this information. For example, if the user has a history of heart disease, the fusion probability value is appropriately increased; if the user has no relevant medical history, the fusion probability value is appropriately reduced. In this way, the user's health status can be more accurately assessed.

[0172] At step S444, the state report is generated based on the final probability, the basic information, the abnormal audio segment location, and the abnormal type, in combination with the recommendation scheme.

[0173] In this embodiment, the final probability refers to the corrected fusion probability value, reflecting the likelihood of the user having a certain disease. The basic information refers to the user's basic information, such as age, gender, height, weight, etc. The abnormal audio segment location and the abnormal type refer to the time range and the type of the final determined abnormal segment. The recommendation scheme is a treatment recommendation or health management recommendation made based on the final probability and other relevant information.

[0174] As an optional implementation, the system generates a state report based on the final probability, the basic information, the abnormal audio segment location, and the abnormal type, in combination with the recommendation scheme. The state report includes the user's basic information, the abnormal audio segment location and the abnormal type, the final probability, and the recommendation scheme. The recommendation scheme can be customized according to the size of the final probability and other relevant information, for example, if the final probability is high, the user is recommended to undergo further examination or treatment; if the final probability is low, the user is recommended to take some health management measures, such as diet control, exercise, etc.

[0175] For example, a patient uses the intelligent stethoscope for examination, and both the first analysis result and the second analysis result identify an abnormal audio segment and believe that the abnormal segment may be related to a certain heart disease. The system cross- validates the abnormal audio segment location and the abnormal type, and determines the final abnormal segment based on the intersection interval. Then, based on the disease probability values corresponding to the abnormal segment in the first analysis result and the second analysis result, and the pre-determined fusion weight, the fusion probability value is calculated by weighted calculation. Next, the fusion probability value is corrected by combining the historical health data associated with the user identifier, to obtain the final probability. Finally, the state report is generated based on the final probability, the basic information, the abnormal audio segment location, and the abnormal type, in combination with the recommendation scheme. The state report shows that the patient's final probability is 70%, and the patient is recommended to undergo further examination and treatment.

[0176] The embodiment improves the accuracy of diagnosis by cross-validation of the position and type of abnormal audio segments, and determines the abnormal segment in combination with the intersection interval. The fusion probability value is obtained by weighted calculation based on the fusion weight, and the accuracy of diagnosis is further improved by combining the historical health data for correction. The state report is generated to provide the user with detailed health information and suggestions, helping the user to better manage his own health.

[0177] The application provides a control device of a auscultation system based on Internet of Things, the control device of the auscultation system based on Internet of Things comprises at least one processor and a memory in communication connection with the at least one processor; the memory stores instructions executable by the at least one processor; the instructions are executed by the at least one processor to enable the at least one processor to perform the control method of the auscultation system based on Internet of Things in the above embodiment one.

[0178] Reference is made below to Figure 4 which shows a structural schematic diagram of the control device of the auscultation system based on Internet of Things suitable for being used to implement the embodiments of the application. The control device of the auscultation system based on Internet of Things in the embodiments of the application can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDA, Personal Digital Assistant), tablet computers, vehicle-mounted terminals and the like, and fixed terminals such as digital TVs, desktop computers and the like. Figure 4 The control device of the auscultation system based on Internet of Things shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the application.

[0179] As Figure 4As shown, the control device of the IoT-based auscultation system can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1002 or loaded from a storage device 1003 into a random access memory (RAM) 1004. Various programs and data required for the operation of the control device of the IoT-based auscultation system are also stored in the random access memory 1004. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the control device of the IoT-based auscultation system to communicate wirelessly or wired with other devices to exchange data. Although the control device of the IoT-based auscultation system with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0180] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carrying computer program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0181] The control device of the auscultation system based on the Internet of Things provided in the application adopts the control method of the auscultation system based on the Internet of Things in the above embodiment, and can solve the technical problem that users need to go to an institution to perform diagnosis, resulting in that the users cannot understand their own state in real time. Compared with the prior art, the control device of the auscultation system based on the Internet of Things provided in the application has the same beneficial effects as the control device of the auscultation system based on the Internet of Things provided in the above embodiment, and other technical features in the control device of the auscultation system based on the Internet of Things are the same as the features disclosed in the above embodiment method, and will not be described here.

[0182] It should be understood that parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0183] The above describes only specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0184] The present application provides a computer readable storage medium having computer readable program instructions (i.e. computer programs) stored thereon, the computer readable program instructions being used to perform the control method of the auscultation system based on the Internet of Things in the above embodiment.

[0185] The computer readable storage medium provided in the application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium may include, but are not limited to, an electrical connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electrical wire, an optical cable, a radio frequency (RF), or any suitable combination of the above.

[0186] The above computer readable storage medium may be contained in the control device of the auscultation system based on the Internet of Things, or may exist separately without being assembled into the control device of the auscultation system based on the Internet of Things.

[0187] The above computer readable storage medium carries one or more programs, when the one or more programs are executed by the control device of the auscultation system based on the Internet of Things, the control device of the auscultation system based on the Internet of Things: in response to the trigger operation of the auscultation terminal, converts the collected audio signal into a digital audio signal; encrypts the digital audio signal, the user identifier, and the terminal identifier of the auscultation terminal into a target signal file through the transmission module, and sends it to the management platform; the management platform schedules the Internet of Things box of the auscultation terminal networking, processes the target signal file based on the first model to generate a first analysis result; in response to the second analysis result of the target signal file returned by the remote server, the first analysis result and the second analysis result are fused and processed into a state report.

[0188] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0189] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0190] The modules involved in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0191] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the control method of the auscultation system based on the Internet of Things, and can solve the technical problem that users need to go to an institution to be diagnosed, so that users cannot know their own state in real time. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the control method of the auscultation system based on the Internet of Things provided by the above-mentioned embodiments, and will not be described here.

[0192] The embodiment of the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the control method of the auscultation system based on the Internet of Things.

[0193] The computer program product provided by the present application can solve the technical problem that users need to go to an institution to perform diagnosis, so that the user cannot know the state of the user in real time. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as those of the control method of the auscultation system based on the Internet of Things provided by the above-mentioned embodiment, and are not described herein.

[0194] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings of the present application, or direct or indirect application in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A control method of an Internet of Things-based auscultation system, characterized by, The application is applied to an Internet of Things-based auscultation system, which comprises an auscultation terminal, a transmission module, an Internet of Things box, a management and control platform, and a remote server. The auscultation terminal is used to collect audio signals of a user. The transmission module is used to convert the audio signals into digital audio signals and upload them to the management and control platform and the Internet of Things box. The Internet of Things box is used to generate a first analysis result according to the received digital audio signals. The management and control platform is used to forward the digital audio signals to the remote server and update the first model of the Internet of Things box. The remote server is used to receive digital audio signals and generate a second analysis result according to the digital audio signals. The transmission module comprises an auscultation sound collection and control submodule and an auscultation sound receiving and processing submodule. The auscultation sound collection and control submodule comprises an audio collection submodule and a signal sending end. The auscultation sound receiving and processing submodule comprises a signal receiving end and an adapter. The audio collection submodule is in communication with the auscultation terminal. The control method of the Internet of Things-based auscultation system comprises the following steps: In response to a triggering operation of the auscultation terminal, the auscultation terminal collects audio signals; The audio signals are amplified, band-pass filtered, operational amplified, amplified, π-filtered by a radio frequency fixed-frequency circuit, and electrostatically treated by an ESD electrostatic protection device, and are uploaded to a preset link; The adapter receives the audio signals of the preset link through the signal receiving end and converts them into digital audio signals, specifically including: based on the fixed-frequency receiving antenna of the signal receiving end, the audio signals are processed into digital audio signals through operational amplification, power amplification, electrostatic treatment, and noise reduction; The digital audio signals, a user identifier, and a terminal identifier of the auscultation terminal are encrypted into a target signal file by the transmission module and are sent to the management and control platform and the Internet of Things box. The Internet of Things box and the auscultation terminal are networked; The management and control platform sends the target signal file to the remote server; and The management and control platform sends the target signal file and a calling instruction of the first model to the Internet of Things box; After receiving the target signal file, the Internet of Things box pre-processes the digital audio signals in the target signal file based on the first model, including noise reduction filtering, time-frequency domain conversion, and feature point extraction; The extracted audio features are matched with a pre-stored standard heart-lung sound and / or respiratory sound feature library in the first model to identify abnormal audio segment positions and abnormal types; Based on the abnormal audio segment positions and the abnormal types, combined with historical health data associated with the user identifier, a first analysis result containing an abnormal feature identifier and a preliminary lesion probability is generated; In response to a second analysis result returned by the remote server for the target signal file, the first analysis result and the second analysis result are fused and processed into a status report; The step of fusing the first analysis result and the second analysis result into a state report in response to the second analysis result of the target signal file returned by the remote server comprises: obtaining first metadata of the first analysis result and second metadata of the second analysis result; if the first metadata and the second metadata pass the verification, determining an intersection interval between an abnormal feature identifier in the first analysis result and an artificially labeled abnormal segment in the second analysis result; if the intersection interval meets a confidence requirement, assigning a fusion weight to the first analysis result and the second analysis result; fusing the first analysis result and the second analysis result into the state report based on the fusion weight. 2.The control method of the Internet of Things-based auscultation system of claim 1, wherein The auscultation system based on the Internet of Things further comprises an analysis device, the adapter is in communication with the analysis device, and the analysis device is a mobile terminal, a mobile consultation vehicle, or a medical operation terminal. The analysis device calls the Internet of Things box to analyze the digital audio signal and generate a first analysis result. 3.The control method of the Internet of Things-based auscultation system of claim 1, wherein The management and control platform is used to analyze historical digital audio signals to perform situation awareness. If an abnormal situation is sensed, the abnormal situation is pushed to the Internet of Things box to prompt a user that an abnormality occurs. 4.A control device of an Internet of Things (IoT) -based auscultation system, characterized by, The control device of the auscultation system based on the Internet of Things comprises a memory, a processor, and a computer program stored on the memory and executable on the processor. The computer program is configured to implement the steps of the control method of the auscultation system based on the Internet of Things according to any one of claims 1 to 3.

5. A storage medium, characterized by The storage medium is a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by the processor to implement the steps of the control method of the auscultation system based on the Internet of Things according to any one of claims 1 to 3.

Images